Vals, backed by Andreessen Horowitz, is looking to become the gold
Vals, a San Francisco startup, has secured $40 million in Series A funding led by Andreessen Horowitz to revolutionize AI benchmarking. The company aims to replace outdated, manipulable evaluation systems with rigorous, confidential, and task-specific assessments for AI models. Vals' approach focuses on real-world impact across industries, seeking to become the trusted standard for AI accountability and public trust.
San Francisco-based startup Vals has officially secured a significant $40 million in Series A funding, led by prominent venture capital firm Andreessen Horowitz. This substantial investment comes as Vals rapidly positions itself to establish a new "gold standard" for AI benchmarking, aiming to rectify the critical shortcomings of existing evaluation systems that struggle to keep pace with the swift evolution of artificial intelligence models. The company’s mission is to provide neutral and trustworthy assessments of AI capabilities, ensuring models deliver on their advertised promises in an increasingly AI-driven world.
The Urgent Need for Better AI Benchmarking
The current landscape of AI benchmarking is fraught with challenges. While these evaluations are meant to validate a model's performance and differentiate it in a competitive market, many legacy systems are outdated and ill-equipped to measure the advanced capabilities of modern AI. This inadequacy has led to a situation where companies can manipulate or "outwit" public benchmarks by training their models specifically against known tests, effectively undermining the credibility of the scores. Rayan Krishnan, Vals' 25-year-old co-founder, observed this growing disparity, noting that academic benchmarks were not "keeping up with that frontier advance."
Vals' Innovative Approach to Evaluation
Vals distinguishes itself by adopting a fundamentally different approach. Unlike traditional systems that often utilize publicly available tests, Vals maintains strict confidentiality regarding its specific evaluation materials, preventing models from being optimized to pass them. Furthermore, the company moves beyond abstract intelligence tests, such as those resembling a bar exam, to focus on an AI model's ability to execute complex, real-world tasks within specialized industries.
These include critical domains like law, finance, and coding, as well as emerging fields such as mental health, cybersecurity, biosecurity, and even applying the Geneva Convention. Krishnan emphasizes that their goal is to assess "what are the real impacts of the models," including analyzing potential negative implications if these models were deployed without proper oversight. Vals even has a benchmark on recursive self-improvement, underscoring its forward-looking methodology.
Visionary Leadership and Strong Backing
Formed in 2024, Vals was conceived by Krishnan, who brought a background of internships at Palantir and work with Microsoft and Stanford's AI lab. His firsthand observations on the limitations of existing benchmarks fueled the company’s inception. Following a period of rapid growth, Vals successfully secured a seed round last year, led by 8VC and Bloomberg Beta, before the recent $40 million Series A infusion from Andreessen Horowitz. This strong venture backing underscores the industry's recognition of the urgent need for more reliable and impartial AI evaluation.
Rapid Growth and Market Impact
Vals' innovative approach is clearly resonating in the market. The startup recently reported that its revenue has grown eightfold compared to last year, signaling strong demand for its services. The company has also seen significant expansion in its workforce, tripling its team from just eight employees at the start of the year to 25. With plans to hire an additional 10 to 15 people and relocate to a larger office in San Francisco, Vals is gearing up for continued growth.
Businesses are increasingly relying on Vals' evaluations as a crucial factor in their decisions when acquiring new AI models, recognizing the value of independent, in-depth assessments for troubleshooting and improvement. Vals has also extended its reach by launching a program dedicated to providing model evaluations to federal agencies.
Shaping the Future of AI Accountability
Krishnan envisions Vals' benchmarking system as integral to the future of the AI industry, particularly as major players prepare for public offerings. With companies like Anthropic slated to go public later this year and OpenAI potentially following suit, reliable, third-party evaluations will become paramount for establishing public trust and transparency. Krishnan predicts that Vals' benchmarks and evaluations will increasingly "drive their usage and be a central part of how these companies submit public filings or talk about the prospective investments they’re going to make in AI," effectively embedding independent assessment into the economic fabric of the rapidly expanding AI sector. The transition of AI models into core economic components necessitates a robust, trusted standard, a role Vals aims to fill.
FAQ
Q: Why are traditional AI benchmarking systems considered inadequate?
A: Many traditional AI benchmarking systems are older and were not designed to measure the advanced capabilities of modern AI models. Additionally, because some tests are publicly available, companies can train their models against these specific tests, which can lead to inflated scores that don't accurately reflect real-world performance.
Q: How does Vals' benchmarking approach differ from conventional methods?
A: Vals employs a proprietary system where test materials are not publicly disclosed, preventing models from being trained to pass specific exams. Instead of general knowledge, Vals focuses on evaluating AI models' ability to perform complex, industry-specific tasks (e.g., in law, finance, cybersecurity), and also analyzes potential negative implications, aiming for a more practical and trustworthy assessment of real-world impact.
Q: What is the significance of Andreessen Horowitz's investment in Vals?
A: The $40 million Series A investment led by Andreessen Horowitz highlights the venture capital community's belief in Vals' potential to solve a critical problem in the AI industry. This funding provides Vals with substantial resources to scale its operations, expand its team, and further develop its benchmarking technologies, solidifying its ambition to become the industry's gold standard for AI evaluation.
Related articles
Jensen Huang's Crisis Deepens Amid Ouster Reports and Second Affair
Nvidia CEO Jensen Huang is facing potential ouster amid revelations of two alleged extramarital affairs—one with a suspected Chinese spy, Alice Huang, and another with Nvidia VP Kimberly Powell. The scandal, which has caused Nvidia's stock to drop, has triggered national security concerns and a congressional inquiry into his ties to Chinese intelligence assets.
Jensen Huang accused of having another affair — with an Nvidia exec
Nvidia CEO Jensen Huang faces new accusations of an affair with a direct report, Kimberly Powell, escalating a crisis already simmering from his admitted affair with an alleged Chinese spy. These revelations prompted the Wall Street Journal to report that Nvidia's board is preparing to oust Huang, who co-founded the company in 1993, as congressional pressure and internal investigations intensify.
CNET's Wordle #1972 Guide: A Solid Daily Puzzle Companion
Quick Verdict CNET's daily offering of Wordle hints, answers, and strategic help for puzzle #1972 is a robust and reliable resource for both casual players and those seeking to refine their game. While the concept of a
AI-Powered "Microduck" Butler Signals Robotics' Future
WIRED Senior Writer Will Knight has unveiled a novel AI-powered "Microduck" robot, developed with an OpenAI model, designed to be a cat butler. This small, duck-like automaton represents a significant step in robotics, showcasing how advanced AI can be used to train specialized robots, potentially through virtual simulations, for practical and interactive roles in everyday life. The project highlights a future where AI-driven training simplifies robot development and expands their integration into our homes.
Seattle VC Firm Ascend Closes $17M Fund for Physical AI, Hardware
Seattle-based VC firm Ascend has successfully closed its $17 million third fund, overcoming a tough fundraising environment. The fund will strategically invest in early-stage "physical AI" and hardware startups across robotics, defense, and energy, tapping into what it calls Seattle’s "Third Act" of innovation. This significant raise includes its first institutional investor, FirstLook Partners.
policy: State Dept. official took Mediterranean trip paid for by tech
A preliminary financial disclosure reveals State Department official Sarah B. Rogers took an $11,900 Mediterranean vacation paid for by Stripe co-founder Patrick Collison. This news sparks ethical concerns regarding gifts from powerful tech figures to government officials, highlighting the growing scrutiny of such relationships and transparency in public service.