Skip to main content

Tesla Autopilot is going to detect potholes and make mini-maps to remember them

Tesla’s Autopilot, which the automaker is trying to turn into a self-driving system, is going to detect potholes and make mini-maps to remember them, according to a new comment from CEO Elon Musk.

In order to achieve full self-driving, a system would have to be able to handle a wide range of different scenarios, including different weather and road conditions.

These conditions, like potholes, can sometimes be difficult for human drivers to handle, and some people find it improbable that self-driving systems will be able to appropriately navigate them.

Tesla is leveraging its large customer fleet equipped with Autopilot hardware to capture data on those corner cases and teach its neural network to handle them.

As we recently reported, Tesla is going through “a significant foundational rewrite in the Tesla Autopilot.” As part of the rewrite, Musk says that the “neural net is absorbing more and more of the problem.”

It will also include a more in-depth labeling system.

He added in an interview on the Third Row Podcast:

3D labeling is the next big thing — where the car can go through a scene with 8 cameras and paint a path in real-time, then label the path in 3D. This is probably a two to three orders of magnitude improvement in labeling efficiency and accuracy.

When asked if Tesla could use this new 3D labeling to create maps of potholes, Musk answered in the affirmative:

It means that Tesla vehicles could build maps that detect potholes and help other Tesla vehicles avoid them.

Musk didn’t say when the update is coming, but he did say that the Autopilot rewrite is almost completed.

Electrek’s Take

This is a big deal.

To be fair, plenty of human drivers are not attentive or quick enough to avoid some potholes, but a full self-driving system will be held to much higher standards and will have to avoid potholes

Last year, I logged in over 300 km (186 miles) of driving on the latest Autopilot update available in my Model 3 over a few hours.

The system performed extremely well, autonomously steering on the highway and adapting the speed to the traffic around the electric car.

With one hand on the steering wheel applying light pressure, Autopilot controlled the vehicle from point A to B almost without any input from me.

The only times I had to take over was to avoid large potholes, which are unfortunately a plague in Quebec, especially during the spring:

If Tesla can figure this out, it will be a significant step forward. And of course, the same system can be replicated for other things than potholes.

FTC: We use income earning auto affiliate links. More.

You’re reading Electrek— experts who break news about Tesla, electric vehicles, and green energy, day after day. Be sure to check out our homepage for all the latest news, and follow Electrek on Twitter, Facebook, and LinkedIn to stay in the loop. Don’t know where to start? Check out our YouTube channel for the latest reviews.



Avatar for Fred Lambert Fred Lambert

Fred is the Editor in Chief and Main Writer at Electrek.

You can send tips on Twitter (DMs open) or via email:

Through, you can check out Fred’s portfolio and get monthly green stock investment ideas.