Skip to main content

Watch how Tesla trains its neural networks for self-driving in 10 minutes

Andrej Karpathy, Tesla’s head of AI and computer vision, gave an interesting talk to get into how Tesla trains its neural networks for self-driving.

It results in an interesting overview of the concept in about 10 minutes.

Karpathy obtained his PhD from Stanford University in Machine Learning with a focus on Deep Learning for Computer Vision and Natural Language Processing in 2016.

In a short time, he made a name for himself in the space by teaching a new Stanford class on Convolutional Neural Networks for Visual Recognition that became very popular while he was doing his PhD.

He was working for Elon Musk’s new nonprofit AI research firm, OpenAI, since September 2016, and it looks like his research impressed Musk enough to hire him at Tesla to turn his neural net expertise into actual real-world applications.

That was two years ago, and over the last year, the engineer has taken a more public role in which he has been giving talks about what his team is up to.

The talks have been held at conferences where Tesla and Karpathy have been trying to recruit engineers for their project, and also at Tesla events to try to build investor and owner confidence into Tesla’s capability to deliver a full self-driving system.

Tesla’s approach is quite different from most of the rest of the industry. They are already deploying features in consumer vehicles and have been deploying the hardware needed for self-driving in all their vehicles for years.

They use this hardware in a large fleet of several hundreds of thousands of vehicles to collect data and train neural networks.

During a talk for PyTorch, an open-source machine learning library at Devcon, Karpathy gave a quick but very informative talk about how Tesla uses machine learning to train its neural networks toward the goal of full self-driving:

Electrek’s Take

I am always impressed by Karpathy. His talks are always informative and he seems extremely knowledgeable.

Something that I thought was really interesting in this presentation was the look at Tesla’s top-down view created through the neural net:

It’s really interesting that Tesla can create that out of its camera-based computer vision system.

Operation Vacation was also an interesting insight into Tesla’s plans. By automating most of the tasks related to training the networks, Karpathy says that his team could almost sit back and let their networks automatically improve as long as the new data keeps coming from the fleet:

Of course, I am sure they would then find other things to do to improve their system, but the pace of improvement would likely increase very fast if the system could automatically improve itself as more data comes in.

In short, I came out of that talk a little more confident about Tesla delivering full self-driving, albeit still being really skeptical about Elon’s timeline.

FTC: We use income earning auto affiliate links. More.

Black Friday
Stay up to date with the latest content by subscribing to Electrek on Google News. You’re reading Electrek— experts who break news about Tesla, electric vehicles, and green energy, day after day. Be sure to check out our homepage for all the latest news, and follow Electrek on Twitter, Facebook, and LinkedIn to stay in the loop. Don’t know where to start? Check out our YouTube channel for the latest reviews.

Comments

Author

Avatar for Fred Lambert Fred Lambert

Fred is the Editor in Chief and Main Writer at Electrek.

You can send tips on Twitter (DMs open) or via email: fred@9to5mac.com

Through Zalkon.com, you can check out Fred’s portfolio and get monthly green stock investment ideas.