Vals, backed by Andreessen Horowitz, is looking to become the gold

San Francisco-based startup Vals has officially secured a significant $40 million in Series A funding, led by prominent venture capital firm Andreessen Horowitz. This substantial investment comes as Vals rapidly positions itself to establish a new "gold standard" for AI benchmarking, aiming to rectify the critical shortcomings of existing evaluation systems that struggle to keep pace with the swift evolution of artificial intelligence models. The company’s mission is to provide neutral and trustworthy assessments of AI capabilities, ensuring models deliver on their advertised promises in an increasingly AI-driven world.

The Urgent Need for Better AI Benchmarking

The current landscape of AI benchmarking is fraught with challenges. While these evaluations are meant to validate a model's performance and differentiate it in a competitive market, many legacy systems are outdated and ill-equipped to measure the advanced capabilities of modern AI. This inadequacy has led to a situation where companies can manipulate or "outwit" public benchmarks by training their models specifically against known tests, effectively undermining the credibility of the scores. Rayan Krishnan, Vals' 25-year-old co-founder, observed this growing disparity, noting that academic benchmarks were not "keeping up with that frontier advance."

Vals' Innovative Approach to Evaluation

Vals distinguishes itself by adopting a fundamentally different approach. Unlike traditional systems that often utilize publicly available tests, Vals maintains strict confidentiality regarding its specific evaluation materials, preventing models from being optimized to pass them. Furthermore, the company moves beyond abstract intelligence tests, such as those resembling a bar exam, to focus on an AI model's ability to execute complex, real-world tasks within specialized industries.

These include critical domains like law, finance, and coding, as well as emerging fields such as mental health, cybersecurity, biosecurity, and even applying the Geneva Convention. Krishnan emphasizes that their goal is to assess "what are the real impacts of the models," including analyzing potential negative implications if these models were deployed without proper oversight. Vals even has a benchmark on recursive self-improvement, underscoring its forward-looking methodology.

Visionary Leadership and Strong Backing

Formed in 2024, Vals was conceived by Krishnan, who brought a background of internships at Palantir and work with Microsoft and Stanford's AI lab. His firsthand observations on the limitations of existing benchmarks fueled the company’s inception. Following a period of rapid growth, Vals successfully secured a seed round last year, led by 8VC and Bloomberg Beta, before the recent $40 million Series A infusion from Andreessen Horowitz. This strong venture backing underscores the industry's recognition of the urgent need for more reliable and impartial AI evaluation.

Rapid Growth and Market Impact

Vals' innovative approach is clearly resonating in the market. The startup recently reported that its revenue has grown eightfold compared to last year, signaling strong demand for its services. The company has also seen significant expansion in its workforce, tripling its team from just eight employees at the start of the year to 25. With plans to hire an additional 10 to 15 people and relocate to a larger office in San Francisco, Vals is gearing up for continued growth.

Businesses are increasingly relying on Vals' evaluations as a crucial factor in their decisions when acquiring new AI models, recognizing the value of independent, in-depth assessments for troubleshooting and improvement. Vals has also extended its reach by launching a program dedicated to providing model evaluations to federal agencies.

Shaping the Future of AI Accountability

Krishnan envisions Vals' benchmarking system as integral to the future of the AI industry, particularly as major players prepare for public offerings. With companies like Anthropic slated to go public later this year and OpenAI potentially following suit, reliable, third-party evaluations will become paramount for establishing public trust and transparency. Krishnan predicts that Vals' benchmarks and evaluations will increasingly "drive their usage and be a central part of how these companies submit public filings or talk about the prospective investments they’re going to make in AI," effectively embedding independent assessment into the economic fabric of the rapidly expanding AI sector. The transition of AI models into core economic components necessitates a robust, trusted standard, a role Vals aims to fill.

FAQ

Q: Why are traditional AI benchmarking systems considered inadequate?

A: Many traditional AI benchmarking systems are older and were not designed to measure the advanced capabilities of modern AI models. Additionally, because some tests are publicly available, companies can train their models against these specific tests, which can lead to inflated scores that don't accurately reflect real-world performance.

Q: How does Vals' benchmarking approach differ from conventional methods?

A: Vals employs a proprietary system where test materials are not publicly disclosed, preventing models from being trained to pass specific exams. Instead of general knowledge, Vals focuses on evaluating AI models' ability to perform complex, industry-specific tasks (e.g., in law, finance, cybersecurity), and also analyzes potential negative implications, aiming for a more practical and trustworthy assessment of real-world impact.

Q: What is the significance of Andreessen Horowitz's investment in Vals?

A: The $40 million Series A investment led by Andreessen Horowitz highlights the venture capital community's belief in Vals' potential to solve a critical problem in the AI industry. This funding provides Vals with substantial resources to scale its operations, expand its team, and further develop its benchmarking technologies, solidifying its ambition to become the industry's gold standard for AI evaluation.