Ford had to hire back former engineers to fix mistakes made by its automated systems

To celebrate its new status as No. 1 in JD Power’s initial quality ranking among mainstream automakers, Ford is opening up about the challenges it has faced in recent years, especially around its reliance on automated systems in production and design. It turns out that those automated systems were not as robust as previously assumed, requiring Ford to hire experienced technicians — sometimes bringing back former employees — to correct errors made by the company’s robots.

In Ford’s view, AI is both powerful and prone to pitfalls. Its effectiveness depends entirely on the quality of the data used to train the AI models. In addition, the automaker underestimated the value of the institutional knowledge accumulated by its more veteran engineers who had worked through multiple vehicle-development cycles. And this combination of phenomena led to a drop in quality in Ford’s vehicles.

“Mistakenly, we thought that by just introducing artificial intelligence and adjusting the design requirements that we had, that that would produce a high-quality product,” said Charles Poon, VP of vehicle hardware engineering, in a briefing this week with reporters.

According to Poon, some of the company’s most experienced personnel left before all of their accumulated knowledge could be fully transferred into Ford’s automated systems. That necessitated bringing back some of those employees to retrain those systems, or in some cases, mentor younger engineers who were currently struggling to maintain Ford’s vehicle quality. Poon said that Ford hired, promoted, or brought back over 350 experienced engineers to rebuild that layer of expertise. In addition to guiding younger engineers, they’ve also been tasked with improving the data collection and AI training that underpin Ford’s automated systems.

“That’s where some of our most experienced engineers have had experience solving and identifying those problems before they creep into the system,” Poon said.

Ford currently leads the industry in the number of recalls, and its quality ratings have slipped over the past several years. Those challenges became more pronounced recently, with the difficulties associated with the launches of the Explorer and Aviator, supply-chain disruptions during the covid pandemic, and the noticeable growth in the number of its vehicle recalls.

According to Ford’s COO Kumar Galhotra, the automaker eventually concluded that its approach to quality had become too fragmented. Different departments operated in silos, and the company relied heavily on a “find and fix” philosophy that focused on identifying defects after they appeared and correcting them as quickly as possible. While that approach could address immediate problems, it did not prevent those problems from occurring in the first place.

“We’re moving from that find-and-fix mentality to preventing issues before they occur,” Galhotra said. “We’re focused on enablers and early indicators versus outputs. Stop admiring the problem and start solving it.”

The transformation extends beyond vehicle hardware. Software and digital teams now work much more closely with vehicle engineering, manufacturing, and supply-chain teams, executives said. And Ford is now attempting to combine the speed and flexibility associated with software development with the rigor and validation requirements of automotive-grade engineering.

Historically, this wasn’t always the case. Ford was only discovering software bugs late in the process because it wasn’t fully leveraging the rapid iteration cycles available, Poon said. That said, the automaker couldn’t push out software updates as fast as consumer electronics companies with the mentality that it could “move fast and fix later,” Poon said. Vehicles, unlike smartphones, operate in a safety-critical environment where customers depend on software functioning correctly from the moment the vehicle is delivered. To fix this, Ford created a dedicated 40-person software quality assurance team with the sole responsibility of preventing problems before they occur.

But don’t think that Ford isn’t dedicated to integrating AI into more of its processes. The automaker says it has dramatically expanded its automated testing capabilities, adding more than 100,000 new AI-powered tests designed to identify edge cases and stress software systems under a wide range of conditions. Because the testing framework is highly automated, software changes can be rapidly revalidated even late in development, ensuring that modifications do not introduce new defects.

“Because these tests are highly automated, even if we have a late change in the software, we can rapidly run back through the entire validation process to guarantee it works perfectly well before it reaches the customer,” Poon said. “We’ve established software reliability as its own rigorous disciplines with strict metrics.”