Become a Sensor Fusion Engineer

Nanodegree Program

Learn to fuse lidar point clouds, radar signatures, and camera images using Kalman Filters to perceive the environment and detect and track vehicles and pedestrians over time.

Enroll now

Estimated time
4 Months
At 10 hours/week
Enroll by
April 26, 2023
Get access to classroom immediately on enrollment
Skills acquired
Point Cloud Data, Sensor Fusion, Kalman Filters

Built in collaboration with

What you will learn

Sensor Fusion Engineer
Estimated 4 Months
Learn to detect obstacles in lidar point clouds through clustering and segmentation, apply thresholds and filters to radar data in order to accurately track objects, and augment your perception by projecting camera images into three dimensions and fusing these projections with other sensor data. Combine this sensor data with Kalman filters to perceive the world around a vehicle and track objects over time.
Related Nanodegree programs
Prerequisite knowledge
You should have intermediate C++ knowledge, and be familiar with calculus, probability, and linear algebra.
1. Lidar
  Process raw lidar data with filtering, segmentation, and clustering to detect other vehicles on the road.
2. Cameras
  Fuse camera images together with lidar point cloud data. You'll extract object features, classify objects, and project the camera image into three dimensions to fuse with lidar data.
3. Radar
  Analyze radar signatures to detect and track objects. Calculate velocity and orientation by correcting for radial velocity distortions, noise, and occlusions.
4. Kalman Filters
  Fuse data from multiple sources using Kalman filters, and build extended and unscented Kalman filters for tracking nonlinear movement.

All our programs include

Real-world projects from industry experts
With real-world projects and immersive content built in partnership with top-tier companies, you’ll master the tech skills companies want.
Real-time support
On demand help. Receive instant help with your learning directly in the classroom. Stay on track and get unstuck.
Career services
You’ll have access to Github portfolio review and LinkedIn profile optimization to help you advance your career and land a high-paying role.
Flexible learning program
Tailor a learning plan that fits your busy life. Learn at your own pace and reach your personal goals on the schedule that works best for you.

Program offerings

Class Content
- Content Co-created with Mercedes-Benz
- Real-world projects
- Project reviews
- Project feedback from experienced reviewers
Student services
- Student community
- Real-time support
Career services
- Github review
- Linkedin profile optimization

Succeed with personalized services.

We provide services customized for your needs at every step of your learning journey to ensure your success.

Experienced Project Reviewers
Real-Time Support

Get timely feedback on your projects.

Personalized feedback
Unlimited submissions and feedback loops
Practical tips and industry best practices
Additional suggested resources to improve

1,400+
project reviewers
2.7M
projects reviewed
88/100
reviewer rating
1.1 hours
avg project review turnaround time

Learn with the best.

David Silver
Head of Curriculum
David Silver leads the Udacity Curriculum Team. Before Udacity, David was a research engineer on the autonomous vehicle team at Ford. He has an MBA from Stanford, and a BSE in Computer Science from Princeton.
Stephen Welch
Instructor
Stephen is a Content Developer at Udacity and has worked on the C++ and Self-Driving Car Engineer Nanodegree programs. He started teaching and coding while completing a Ph.D. in mathematics, and has been passionate about engineering education ever since.
Andreas Haja
Instructor
Andreas Haja is an engineer, educator and autonomous vehicle enthusiast with a PhD in computer science. Andreas now works as a professor, where he focuses on project-based learning in engineering. During his career with Volkswagen and Bosch he developed camera technology and autonomous vehicle prototypes.
Abdullah Zaidi
Instructor
Abdullah holds his M.S from the University of Maryland and is an expert in the field of Radio Frequency Design and Digital Signal processing. After spending several years at Qualcomm, Abdullah joined Metawave, where he now leads Radar development for autonomous driving.
Aaron Brown
Instructor
Aaron Brown has a background in electrical engineering, robotics and deep learning. Aaron has worked as a Content Developer and Simulation Engineer at Udacity focusing on developing projects for self-driving cars.

Top student reviews

4.7

4.7 stars

(303)

Daniel B.

5.0 stars

The first project gave a me very good start into the topic of object detection.

I think there is enough room left to dive even more into the topic, e.g. by having a follow-up course that connects by adding more sophisticated algorithms or a follow-up course about "environmental model analytics". I feel motivated to do this on my own, though. There is plenty of information and statements to give some hints how to improve the own implementation.

Udacity could think about adding measurements (they actually already have time measurements for the algorithms) to start a challenge to motivate people even more to improve their algorithms.

Peter L.

5.0 stars

I felt the first course on LiDAR Obstacle Detection was structured well and let me step through using the first concept and then delving into how it works. I'll see what I think as I progress, but I feel it would be useful to have a follow-on course that delves more deeply into some of the concepts that are briefly touched upon. For example, I think my favorite part of the first course was the PCA Bounding Boxes; I think I spent the most time on this part of the course, though it wasn't graded / reviewed.

Viktor M.

5.0 stars

In general first project was fine. But whole Lidar course had broken links to PCL site. So hard change dozen of links? As usually project has some challenge tasks, but no enough input and review contains nothing except rubric points with standard unpersonalized notes. Something like we enourage you do something beyond course plan, but it's your problem. By the way PCA doesn't work properly for Lidars because it capture just one side of most objects.

Hsin-Wen C.

5.0 stars

The first time I'm able to design and implemented multi-data structures, algorithms and visualize data for sensors on real self-driving Cars. Deep dive by hands-on implement the concept of ring buffer to handle the data buffer. Writing efficient, well-presented easy to understand code without using any unnecessarily complicated data structures to store additional data. I love Sensor Fusion Nanodegree Program which is very useful in my daily work.

shantam b.

5.0 stars

I just completed the LIDAR course and have to say that the project and the exposure provided towards point cloud processing is unparalleled. You can learn a lot about the point cloud library and implementing everything from scratch did add an additional layer of challenge but solidified my understanding of how point clouds are actually processed. Also everything is in C++ which makes it more like how it is done in the industry!

Eugen S.

5.0 stars

I appreciate the way how the Lidar topic was presented:

- Illustrative representation using laser beams from simulation - Explanation of the Lidar perception stack (segmentation, clustering, boxing) in a clear way - Illustration based on real Lidar data - obligation of implementation of the Lidar perception stack (segmentation, clustering) instead of using the PCL library --> this way you understand better the algorithms