Generative AI-Powered Robot Safety Startup Emerges

A new startup is emerging to tackle a crucial challenge in the field of artificial intelligence: ensuring the safety of generative AI-powered robots.

Safeworld, founded by Dr. Ding Zhao and his team, aims to address the unpredictability of AI-driven robot behavior, which can be particularly concerning for humanoid robots designed to interact with humans. This is an area where traditional algorithms fall short, making it difficult to guarantee a robot's safety.

Dr. Zhao, who has dedicated nearly his entire career to safe artificial intelligence research at Carnegie Mellon University, is joined by veteran start-up executive Kyle Wong and machine learning engineer Simo Rachidi in this endeavor.

The key challenge lies in understanding the probabilistic nature of generative AI systems and assessing their risk levels, as well as building trust among users. These two aspects are crucial for deploying robots safely into homes and public spaces.

Safeworld has secured a significant seed round of over $12 million from notable investors, including Shine Capital, a16z Speedrun, Box Group, Carnegie Mellon University Endowment, Innovation Endeavors, and SV Angel.

Safeworld's approach to evaluating robotic control systems involves simulating real-world scenarios using human models in virtual environments. This method mimics the challenges faced by companies developing autonomous vehicles, such as Tesla and Wayve, which must ensure their vehicles respond appropriately to unexpected incidents on the road.

However, Zhao notes that robots face an even greater challenge due to the unstructured nature of their operating environments and varying safety standards across different facilities. For instance, a factory with blind corners poses unique hazards for robotic navigation, requiring careful consideration of speed and stopping distances to prevent collisions with human workers.

To address these concerns, Safeworld creates digital models of potential scenarios, such as a factory corner with a blind spot, and inserts simulations of the robot being evaluated. The simulation is driven by the robot's actual software, allowing Safeworld to test thousands of scenarios where human models interact with the robot.

One area of focus for Safeworld is testing how robots respond to unpredictable human behavior, such as carrying boxes or unexpected movements. This requires simulating a wide range of human actions and reactions to ensure the robot can adapt safely in various situations.

Safeworld also uses simulations to test more mundane but still critical scenarios, like tripping and falling, which are difficult to replicate in real-world testing due to their frequency and potential impact on robot safety and performance.

Safeworld's founders believe that their platform can provide a valuable service to robot manufacturers by validating their safety protocols and sharing information about critical cases between competitors.

This validation is crucial because it allows companies to identify potential issues early on in the development process, rather than waiting until they are deployed at scale. As Zhao noted, it's not just the robots themselves that pose a risk, but also how they interact with humans who may be unfamiliar with their operation.

One key partner for Safeworld is Gritt Robotics, which is developing AI brains for robots used in industrial settings such as solar farms and construction sites. The company's CTO, Vishal Dugar, sees the value in collaborating with Safeworld to develop safety simulations that can verify the reliability of his systems.

These simulations are essential because they allow companies like Gritt to test their robots' performance under various scenarios, including those where human workers may be present. By considering all possible outcomes, developers can identify potential blind spots and make necessary adjustments before deployment.

Dugar's company is working closely with Safeworld to create safety protocols that can be formally verified through mathematical equations. However, he notes that this approach has limitations and often requires empirical testing to ensure the system's overall safety.

The complexity of human behavior poses significant challenges for AI-powered robots designed to interact with humans safely. With people capable of exhibiting a wide range of physical configurations, from standing or kneeling to running or tripping and falling, robots must be able to respond appropriately in various situations.

To address this issue, Safeworld is developing a platform that can handle the diverse behaviors and appearances of humans. The company's team acknowledges that it's still early days for both their product and generative AI in robotics, but they're confident in tackling the problem at hand.

In addition to responding to human behavior, robots must also be able to recognize and adapt to various physical characteristics, including clothing, size, shape, height, skin color, and other factors. This requires a sophisticated system that can learn from experience and adjust its responses accordingly.

Safeworld's approach is focused on providing a solution for external users, but the company is still deciding whether to adopt a platform-based or services-oriented model. The team believes their product will be profitable because companies seeking to deploy AI-powered robots will need to pay Safeworld to ensure safe human-robot interactions.

While the details of Safeworld's business strategy are still being worked out, one thing is clear: the company is taking on a critical challenge that requires innovative solutions and careful consideration of the complexities involved.

Safeworld's ambitious goal of developing gen AI robots that can navigate and interact safely in public spaces is a complex challenge that requires innovative solutions.

While the company's business strategy is still being worked out, it's clear that Safeworld is taking on a critical challenge that demands careful consideration of the complex social and technical issues involved. The company must balance the potential benefits of AI robots with the need to ensure their safety and reliability in real-world settings.