Is Waymo ready for winter?

During a recent all-hands meeting, Waymo’s chief winter weather expert laid it out for all the company’s engineers and product managers: if Waymo wants to expand to new cities and new markets, it’s robotaxis will need to be able to confidently and safely handle the snow.

In its first few years of operation, Waymo has strategically stuck to cities with warmer, drier climates — places like Phoenix, Los Angeles, Atlanta, and Austin. But as it eyes a slate of East Coast cities, including Boston, New York City, and Washington, DC, for the next phase of its expansion, its abilities to handle more adverse weather will become a crucial test.

“Ha ha, good question,” said Robert Chen, the company’s product lead for weather, when I asked him when he hoped to validate the Waymo driver for winter operation. “This winter season is gonna be a really important season for us. I think that’s all I can probably say at this point.”

There’s an understanding that an inability to tackle winter roads could impede the usefulness of Waymo’s robotaxi service, which could limit its growth into new cities. Other, human-powered ridehailing services drive in all weather conditions. If it wants to compete, Waymo can’t just be a fair weather robotaxi company.

“I think here at Waymo, we really aspired to build this product and the service that people can rely on and use, not just eight or nine or ten months out of the year,” Chen said, “but something that they can really rely on and use whenever they want.”

In some ways, autonomous vehicles are a lot like human drivers: they preform best with good visibility, dry roads, and unobstructed sensors. When roads get icy, and the snow piles up, that’s when things get complicated. Waymo has faced a variety of challenges, from flash floods to Phoenix dust storms known as “haboobs,” but snow is uniquely challenging, he said.

But human drivers can make assumptions about road markings and signs that may elude even the most well-trained robot, said Phil Koopman, an expert in autonomous vehicle technology from Carnegie Mellon University. Snow can obstruct road markings and street signs, making it more difficult for the vehicle’s perception system to respond safely and accurately.

“You may only see a third of the stop sign, but you know it’s a stop sign,” Koopman said. “Machine learning can have trouble with that if it hasn’t been trained on partially obstructed stop signs.”

A system like Waymo’s, with its full stack of redundant hardware like lidar, radar, and cameras, should eventually be up to the task, Koopman added. Camera-only systems, like Tesla, may have more trouble.

“It’s gonna be easier for a multi-sensor platform because cameras are gonna have a lot of trouble with blowing snow,” he said. “But for sure, radar’s gonna really help you if there’s snow.”

Not only is it a technical challenge, but it’s also a data challenge as well. Snowy conditions are rare in Waymo’s dataset, making relevant data extremely sparse — often less than 5 percent or even a fraction of a percent of total driving data for more rare conditions, Chen said.

This scarcity has pushed Waymo to use innovative techniques, including advanced AI methods, to augment and analyze data for development and validation, which Chen said is helping the company make progress. Waymo has logged some miles in snowy climates, including in Truckee, California, Michigan, and Upstate New York. It’s also testing in Denver and Seattle — but there’s still more work to do.

Waymo’s fifth-generation system can tackle cold weather and light snow, but the sixth-generation Waymo Driver, which is set to roll out soon, is specifically being engineered and tested to handle severe winter conditions, Chen said.

In addition to better data, Waymo is exploring other solutions to help its robotaxis navigate slicker, slushier streets. For example, tiny mechanical wipers have been installed to help clear snow off the vehicle’s rooftop lidar sensor. More powerful heaters are also included to help defrost all the sensors. Waymo’s current system is already trained on icy road conditions and can handle challenges like black ice, Chen said, citing sub-zero temperatures in Austin last winter. Each vehicle in its fleet records this data while out on the road, like a mobile weather station, he said.

“Let’s say the vehicle encounters a slippery patch,” Chen said. “It’ll actually send that information to the rest of the fleet and now other other vehicles in the fleet know that that particular location is slippery.”

If conditions deteriorate to the point where roads are unsafe and most people are staying home, Waymo may pause service. Such decisions are rare, but if the public isn’t traveling, there is little need to keep vehicles on the road. And of course, each city is different: a light snow may bring some cities to a standstill, while in others, it would barely slow traffic.

Even when the snow melts, Waymo can continue developing its capabilities virtually, using advanced simulation models to replicate rare conditions. This is crucial for addressing its issues with data scarcity, Chen said. This is where AI is being integrated: generative and foundational models are being incorporated into the system. And Waymo is using layered models that can distinguish different kinds of snow — wet, powdery, slushy, etc. — and feed that information back into training pipelines.

It may be a while before Waymo’s customers experience their first snowy robotaxi rides. The company has said it will commence operations in Washington, DC, next year, but has yet to put a date on other East Coast cities. Waymo is also aiming to launch in London and Japan at a future date. But as the temperature plummets and flurries begin to swirl in the air, Chen and his team will be ready.

“The self-driving problem… is really hard on its own,” he said. “Now you add in these crazy weather conditions. It’s a pretty challenging task.“