Aurora, a self-driving car startup scheduled to debut on Nasdaq next week, is turning to Pixar’s senior team to help the computer simulation tools used to test and train its self-driving systems more like the real world.

Colrspace, a three-person computer graphics startup that has been operating secretly, is joining Aurora's perception team. Aurora will also own Colrspace's IP, especially combining CGI and machine learning technologies. A trio of Michael Fu, Allen Hemberger, and Alex Harvill developed a technique that can reconstruct 3D objects and materials from photos or images. In essence, it makes the simulation more "realistic", which Aurora and other self-driving car developers believe will help make the test more effective. (The following is a working example of Colrspace; TechCrunch changed mp4 to gif)

Fu, Hemberger and Harvill will join Aurora's perception team, which already includes people from 7D, a simulation startup founded by former Pixar software engineer Magnus Wrenninge. TechCrunch learned in 2019 that it had acquired 7D.

Although competitors such as Aurora and Argo AI, Cruise and Waymo regularly conduct real-world tests on closed routes and public roads, computer simulations are seen as an important tool for testing, training, and validating their self-driving car technology. Simulation can be used to help autopilot systems test various scenarios or replay what is happening in the real world. The ultimate goal is to train and evaluate the software stack to make it safe in the real world.

It is not uncommon for the largest AV companies to run thousands or even millions of simulations every day. For example, Aurora estimates that its simulator is equivalent to driving more than 50,000 trucks continuously. Aurora’s computer simulators are called virtual test suites that can run various driving conditions and common and abnormal (edge-case) scenarios to detect errors early before they are deployed to vehicles running on public roads. The data captured while driving on public roads will also be fed back to the simulator.

This year, Aurora invested more energy and resources to expand its simulation program. Aurora said this month that by the end of this year, its simulated driving mileage will exceed 9 billion miles, of which 6 billion miles will be completed in 2021.