Can Safeworld convince people that GenAI robots won’t hurt them

Leading the charge in the burgeoning field of AI robot safety, Safeworld has emerged from stealth with over $12 million in seed funding. The startup aims to tackle the critical challenge of ensuring generative AI-powered robots operate safely alongside humans, addressing the inherent unpredictability that sets them apart from traditional algorithmic systems. This significant investment, spearheaded by Shine Capital and a16z Speedrun, underscores the urgent need for robust safety protocols as advanced robotics move towards wider deployment.

Generative AI is rapidly becoming the driving force behind modern robotics, offering unprecedented capabilities but also introducing a complex problem: a lack of the predictable architecture found in conventional algorithms. This unpredictability raises serious questions about how to guarantee the safety of these sophisticated machines, particularly when they interact with people in diverse and unstructured environments.

Addressing the Unpredictability of AI Robotics

Dr. Ding Zhao, director of the Safe AI lab at Carnegie Mellon University, a veteran in this domain, co-founded Safeworld alongside seasoned startup executive Kyle Wong and machine learning engineer Simo Rachidi. Their mission is to bridge the gap between advanced generative AI's probabilistic evaluations and the crucial element of public trust, both essential for widespread robot adoption.

Jonathan Lai, a partner at a16z Speedrun, emphasized the timeliness of Safeworld's mission. “The time to build an industry safety standard is now while robots are being designed and deployed,” Lai stated, highlighting the potential catastrophic consequences if safety incidents involving robots in households become common.

Safeworld’s core innovation lies in its specialized approach to evaluating robotic control systems. The company creates highly realistic digital simulations populated with detailed human models. This methodology is akin to the challenges faced by autonomous vehicle developers like Tesla, who must prepare for myriad unexpected road incidents. However, for robots, the complexity is amplified due to their operation in unstructured settings and varying facility-specific safety standards.

Simulating Real-World Human Interactions

Safeworld's simulations meticulously recreate real-world scenarios. For instance, in a factory setting, they might model a 'blind corner' to determine a robot's necessary speed and stopping distance to prevent collisions with humans. Kyle Wong illustrated this, asking, “If a human is carrying boxes, for example, will the robot detect the human or not?”

The simulations utilize models like Genesis or MuJoCo, inserting a robot driven by its actual software into a digital environment. Thousands of scenarios are then run, with human models interacting with the robot in unpredictable ways. Dr. Zhao noted the challenge lies in human unpredictability, stating that “tripping and falling is also a good example of something that we do a lot of testing with the simulation. Otherwise, you would have to go and trip and fall for the robot, which is like a hard thing to be doing all the time.”

Third-Party Validation and Industry Partnerships

While robot manufacturers develop their own internal testing tools, Safeworld’s founders believe in the critical need for a third-party validator. This not only provides an independent verification layer but also facilitates the crucial sharing of safety case information across competitors within the industry.

“A lot of people are underestimating one how hard some of these edge cases are going to be to solve,” Zhao cautioned. He stressed that concerns aren't about robots in isolated demos, but about widespread deployment among people who may have no prior experience operating them.

Gritt Robotics, a company developing AI brains for robots used in industrial-scale solar farm construction, has partnered with Safeworld. Vishal Dugar, CTO of Gritt Robotics, highlighted the empirical nature of their safety verification process. “The difficulty with most of our systems is it’s very hard to formally prove it by doing some math, writing some equations, and saying yeah, the system is verified to be safe,” Dugar explained.

Gritt Robotics' robots work closely with human laborers, making arm safety paramount. Dugar detailed the complexities: “Humans have many kinds of appearances. Their bodies can be in different configurations. They could be kneeling, standing. They could be tripping and falling potentially. They could be crouching. They could be running.” The robots must respond to this vast array of human behaviors and appearances, including variations in clothing, size, shape, height, and skin color.

The Road Ahead for Robot Safety

Safeworld is still in its early stages, alongside the broader field of generative AI in robotics. The company is actively exploring its optimal product model, considering whether to offer a platform for external users or a services-based approach. Despite these ongoing considerations, the team is confident they are addressing a fundamental and increasingly vital problem.

Dr. Zhao expressed strong conviction about the company's future, asserting, “We’ll probably be the first profitable company in this field. Because if anyone wants to deploy, they need to pay us to handle the situation.” As GenAI robots become more prevalent, Safeworld aims to be the essential layer of trust and safety that enables their widespread, secure integration into society.

FAQ

Q: What problem is Safeworld trying to solve?

A: Safeworld aims to solve the safety and trust challenges associated with generative AI-powered robots. These robots are inherently less predictable than traditional systems, posing risks when interacting with humans in complex, unstructured environments, and Safeworld provides a method to validate their safety.

Q: How does Safeworld test robot safety?

A: Safeworld tests robot safety through advanced digital simulations. They build virtual environments, insert robots running their actual software, and run thousands of scenarios with realistic human models exhibiting unpredictable behaviors like tripping or carrying objects, to evaluate the robot's responses.

Q: Who are Safeworld's founders and primary investors?

A: Safeworld was co-founded by Dr. Ding Zhao (from Carnegie Mellon's Safe AI lab), Kyle Wong, and Simo Rachidi. The company secured over $12 million in seed funding led by Shine Capital and a16z Speedrun, with additional investment from Box Group, Carnegie Mellon University Endowment, Innovation Endeavors, and SV Angel.