So you’ve got the basics down from Part 1. Now you want to know what software to know for the specific role you want at the autonomous vehicle company you’re pursuing. As we mentioned previously, those roles are plentiful, and there are tens of tools available for each task in any given position.

Introduction

The following sections will breakdown the tools autonomous vehicle engineers use by their purpose. Again, these tools are constantly changing, and new ones are added daily. Per our methodology, when meticulously combing through every single careers page from 31 autonomous vehicle companies, we listed any tool or skill that appeared in more than 3 companies, and bolded ones that showed up in more than 10. We wrote a note about some of the most common. It didn’t matter how many times it showed up at a particular company, we wanted the tool to be used across the industry to include it. Therefore, a tool your company uses daily may not have made the cut.

Feel free to comment below and let us know if we forgot anything glaring, or if any new tool should be included that we forgot.

Tools of the Trade

Vehicle Design — Concept

If you’re looking at being the person in charge of designing the vehicle concept itself, or maybe creating marketing materials, these are the top tools for you:

Sketch, Adobe Illustrator, Adobe Photoshop, Microsoft Paint (Just Kidding)

Adobe Illustrator is an environment in which users can create and edit 2D images. It is considered one of the most popular software products for creating conceptual images of vehicles and general marketing images.

Figure 1: Bugatti Type 57 T Concept Rendering by Artur B. Nustas

Vehicle Design — CAE/CAD/PLM/FEA

If you’re interested in the mechanical design of the vehicle and its various components, you’ll need to know at least one of these. These tools are all Computer Aided Engineering tools (CAE). What used to be done with a pen and paper and a slide ruler is now done all on the computer, with fantastic visualization tools. Computer Aided Design (CAD) is 2D and 3D modeling of the vehicle and its parts. Product Lifecyce Management (PLM) is a comprehensive suite of tools that let you organize all the parts you design and purchase. Finite Element Analysis (FEA) is a computationally-intensive program that lets you test the stress and strain of bodies in software as the different forces and moments are applied to them.

CATIA, Solidworks, Pro/E, Autodesk 360, Enovia, STAR-CCM+, ANSA, Altair HyperWorks and OptiStruct, ANSYS, MSC NASTRAN, Abaqus, Polarion

CATIA is a very popular 3D engineering design tool. It lets you create and edit 3D images, and then simulate how they will be built and stressed. It is especially popular in the automotive industry.

Figure 2: 3D Image Created with CATIA (Source)

Multibody Vehicle Dynamics and Vehicle Model Simulation

If you want to design how the car actually drives, this section is for you. These software tools help engineers design suspension, brakes, and deliver power to the wheels. This is software lets engineers build out representative models of the vehicle in software and test it in real-life driving scenarios, to optimize its performance. It’s also how the engineers know where the limit of the vehicle is in any given situation, which is instrumental in letting the autonomous software know how to navigate emergency scenarios. This is the same software that race car engineers use!

LS-DYNA, MSC ADAMS, CarSim, CarMaker, Dymola, OptimumG, SusProg3D, Oktal SKANeR

MSC ADAMS is a software tool for solving and visualizing all of the math that does into multibody dynamics. In particular, it is an extremely powerful tool for multibody vehicle dynamics that can provide engineers immediate feedback on their vehicle designs.

Figure 3: MSC ADAMS Example Use (Source)

Analog and Digital Hardware Development

There’s a ton of tools that engineers use to design the circuitry that goes into autonomous vehicles, and we won’t go into much detail here. Just know, these are the two most common tools we see in this space:

LTSpice, Altium

Vehicle Software Development — General Knowledge

There’s a lot of software that falls into the “general knowledge” bucket that having a knowledge of will substantially help certain job functions. These are cross-team, so we included them here. We don’t anticipate anyone knows all of these, but it would be good to have an understanding of what at least some of them do before walking into a job interview:

Docker, CMake, Shell, Bash, Perl, JavaScript, Node.js, React, Go, Rust, Java, Redux, Scala, R, Ruby, Rest API, gRPC, protobuf, Julia, HTML5, PHP

Docker is a virtualization layer for distributing applications. It is a much simpler and streamlined mechanism for developing and distributing software due to the fact it can “containerize” all of the dependencies for that application without all the “fat” that makes applications slow and cumbersome. There is no need to setup separate VMs when using Docker.

Figure 4: VMs vs Containers using Docker Engine (Source)

Vehicle Software Development — Programming ROS

Most autonomous vehicle teams use ROS to control the vehicle, as stated in the previous section. If you are using ROS, it’s important to know these two tools to make your life easier:

RVIZ, PCL

Vehicle Software Development — Programming CPUs/MCUs

If you’re programming a central processing unit (CPU) or microcontroller unit (MCU) to drive your autonomous vehicle, it’s important to have these skills:

C, MISRA C, Embedded C, RTOS

RTOS stands for “Real-Time Operating System,” and is a operating system architecture that allows processes to happen deterministically, or always at a set interval without delays. This is necessary for highly reliable CPU/MCU systems because latency and jitter can be the difference between executing a life-saving maneuver on time or not.

Figure 5: Example Embedded System running RTOS (Source)

Vehicle Software Development Programming FPGAs/ASICs

If you’re programming an FPGA to drive the control system on your vehicle, it’s important to have an understanding of at least one of these Hardware Description Languages (HDL), which is a low-level programming language that lets you control hardware directly. Many of these tools are used in designing tiny, dedicated silicon chips, called Application-Specific Integrated Circuits (ASIC):

Verilog, VHDL, DSP, Cadence, Synopsys, Xilinx Platform Studio (ISE and XPS)

Velilog is a software language that allows the developer direct access to the hardware registers, thus making it a HDL. It is used to design and test analog and digital circuits for FPGAs, ASICs, and even some MCUs.

Vehicle Software Development — Programming GPUs (GPGPUs)

GPUs are getting a lot of press lately because of their ability to process images and point clouds in a massively parallel process. Often the skill that will be advertised on job offerings is “General Purpose computing on Graphics Processing Units” (GPGPU). These tools will help you do just that:

CUDA, OpenCL, OpenGL, DirectX, DirectCompute, Vulkan

CUDA (Compute Unified Device Architecture) is NVIDIA’s toolkit programmers use to program their GPUs. It’s a platform and API that gives the user direct access to all the bells and whistles of the GPU. CUDA works with C and C++ and integrates with OpenCL. If you’re doing image processing, CUDA is a must know.

Figure 6: NVIDIA CUDA Domain Specific Libraries (Source)

Vehicle Software Development — LIDAR, Radar, Cameras, Perception Sensor Fusion

How does your autonomous vehicle “see” the world? Is it LiDAR, Radar, or just cameras? Maybe something else, like ultrasonic sensors? Well, the following are the most common software tools and skills associated with hardware that lets you take what the sensor “sees” and make it useful information for the rest of the software stack:

Velodyne Development Kit, ZED Stereo Camera SDK, Scanse LIDAR SDK (shutdown), SLAM

SLAM stands for Simultaneous Localization And Mapping. “It is the computational problem of constructing or updating a map of an unknown environment while simultaneously keeping track of an agent’s location within it” (Wikipedia). This is the core approach to determining where a vehicle is in space and where it needs to go next. This is where you hear about those famous machine learning buzz words, like “extended Kalman filter.”

Vehicle Software Development — Machine Learning, AI, Deep Learning

Here’s where all the crazy research is happening. If you’re into AI, Machine Learning, or Deep Learning, you’re probably intimately familiar with these tools. If you want to be in this space, you better start learning, because creating the algorithms necessary to do much of this from scratchncan be quite the task!

TensorFlow, Keras, Torch/PyTorch, CAFFE, Apache MXNet, Theano, CNTK

TensorFlow is an open source software library developed by Google that’s become the de facto standard for leveraging Machine Learning and Neural Network algorithms. It can run on GPUs and CPUs. TensorFlow creates easy-to-visualize stateful dataflow graphs (shown below), that are referred to as “tensors,” hence the name.

Figure 7: Example TensorFlow Playground (Source)

Software Test Frameworks

So, you wrote all your software, how do you make sure it works. Many times, engineers will build out their own “Unit Test Frameworks” to feed in simulated data to their software and check their answers. They stress all of the corner cases to make sure nothing breaks. Here’s a few tools that the industry uses to make this easier:

Mocha, SysML, Jasmine, Jest

Game and Physics Engines for Simulation

Here’s another super popular area for jobs. Many times, engineers want to see their software run on the autonomous vehicle as a simulation. They want to physically see the car driving in an environment, simulated. This is a challenging thing to do, take data variables and make them into pictures and moving images. It’s even more complicated to simulate the physics of the real world. These powerful tools allow engineers to make these simulations and test our the software they create with life-like visuals:

Unity 3D, Unreal Engine, CryEngine, Lumberyard, Bullet, Havok, PhysX

Unity 3D is a gaming engine for 2D and 3D visualizations. This is often used for video game development, but is very applicable for real-world simulations, since it includes a “physics engine.” This physics engine let’s you create real world scenarios and test out vehicles without solving for all the math involved at every point. It also helps create representative visuals of how the vehicle will navigate the road.

Figure 8: Unreal Engine Car Configuration (Source)

Vehicle Communication Protocols

These aren’t so much software tools, as specific protocols you should be aware of if you’re programming the vehicle. There are a bunch of tools that let you see what is being sent back and forth along the communication interfaces to the specific parts of the vehicle that will be discussed later, but you should be aware of the various protocols that exist:

CAN, LIN, FlexRay, Ethernet (Automotive Ethernet), SPI/I2C, TSN (Time-Sensitive Networking), TCP/IP, WLAN (Wifi), Bluetooth, 5G, Cryptography Primitives and Cryptoschemes

CAN (Controller Area Network bus) is a 2-wire digital communication protocol that is standard in the automotive world. Almost everything that is controlled by the vehicle’s onboard computers communicates via CAN.

Data Storage

You should be familiar with methods to interface to the data storage onboard the vehicle, namely:

Redundant Array of Independent Disks (RAID), Network Attached Storage (NAS)

RAID is a data virtualization technology that let’s you increase the speed of writing/reading from large chunks of memory and/or creates redundancy of the data in memory. It is necessary to employ when dealing with large amounts of data that are stored locally that also cannot be susceptible to read/write errors.

Databases

So once you take that data off the hardware onboard the vehicle, you need a place to put it. These softwares help you do just that, by managing all of the data in an organized database that has a certain way it records and extracts data:

HBase, NoSQL, MongoDB, PostgreSQL, SQL, MySQL, DynamoDB,HDP, Cloudera, EMR, Cassandra, Vertica

NoSQL is a non-relational database used for cloud storage and retrieval of data. NoSQL is often used instead of SQL due to it’s performance and ability to handle large amounts of data in real-time.

Figure 9: NoSQL vs SQL from Intellipaat

Streaming technologies

Stream processing is a computer programming architecture or paradigm that allows developers to process large chunks of data in parallel, with the help of multi-core computers, GPUs, or even FPGAs. This technology is necessary for speeding up data crunching dramatically.

Apache Kafka, Storm, Flink, Spark Streaming

Apache Kafka is a common streaming technology written in Java and Scala used for high throughput, low latency handling of real-time data. Since performance is so key with streaming platforms, many users also employ monitoring applications alongside Kafka.

Batch technologies

Similar to streaming technologies, batch technologies provide a high performance methodology to handle large sets of data. The difference is that batch technologies distribute data among multiple hardware resources and solve them in parallel. This is particularly useful for redundancy and reliability

Apache Hadoop, MapReduce, Apache Spark, Hive, Presto, Impala

Apache Hadoop is an open source series of software libraries that assumes that hardware failures are common, so employs a strategy to distribute data sets among multiple resources, typically on a server farm.

Here is a good simple article on Batch vs Streaming Technologies by

.

Serialization

Serialization is a method for taking large chunks of data and processing them one-by-one, typically to be stored or analyzed. This is useful for piping data between applications.

Avro, Parquet, JSON

Vehicle Test — MIL, SIL, HIL, In-Vehicle Test

Test Engineering roles are plentiful, and that’s because it’s not easy to make sure everything works all the time on the autonomous vehicles. But there are tools that do speed up the process. These software tools provide environments, templates, and architectures for validating anything on the vehicle, whether it’s just on an engineers bench or in a high-volume manufacturing setting:

NI LabVIEW, NI TestStand, NI VeriStand, dSpace HIL Simulation Systems, dSpace RTMaps, Proemion PETools CAN Tools, Vector CANalyzer, Vector CANape, Vector Capl

dSpace HIL Simulation Systems are a software and hardware suite of tools for taking software, models, or hardware, and generating inputs and outputs to that “thing” under test. This can be simulated as fake data in software, or even real-life electrical signals and physical inputs/outputs for HIL. dSpace is so popular in this space because they have been in the automotive space for so long and are intricately tied to automotive ECU design.

Figure 10: National Instruments HIL Architecture (Source)

Data Visualization and Analysis

After all that test and simulation, you have tons and tons of data in different formats. It’s quite the task to consolidate that data, put it in a format that’s easily consumable, analyze it to determine what was going on, and make intelligent conclusions about what to do next based on that analysis. There’s a few tools available to help with this today, and there are some new tools on the horizon 😊

Microsoft Excel, NI DIAdem, Splunk, Datadog, Logz.io, ELK Stack, Looker, Tableau

Microsoft Excel is used by everyone, and many times when it shouldn’t be. There are often better tools for aggregating and visualizing data, but people tend to use the devil they know…

Web Services

Cloud computing platforms are all the rage these days,and for good reason. People no longer have to set up servers in their offices, which are large and costly. Now you can rent server space online through any one of the following cloud computing platforms. Some are even specially tailored for machine learning!

Azure and Azure ML, Google Cloud and Google AI, Amazon Web Services (AWS)

Amazon Web Services (AWS) is an on-demand cloud computing platform. It is so popular beacuse it is extensible from individual use up to large corporation use.

Source Code Control (Others)

In addition to Git, which has already been talked about, these two source code control tools are the next most-common:

Perforce, Subversion (SVN)

Issue Tracking Products

Jira is by and far the most common issue-tracking tool. It is used for bug tracking, service requests, and project management. It is a paid product.

Requirements Management

When anyone takes on a problem, they first document what will constitute successful accomplishment of that task. Those are considered the “requirements” of that task. When the problem gets big and challenging, there are often tons of requirements that must be fulfilled at certain times, and at times at conflict with one another. That’s why there are softwares to help manage that process.

Rational DOORS, JAMA, Rhapsody

IBM’s Rational Doors (Dynamic Object-Oriented Requirements System) “is a group of requirements management tools that allow you to capture, trace, analyze and manage changes across the development lifecycle” (Source). Over time it has become the standard for large, complex software project requirements management.

Other Automotive Topics

Here are a few automotive topics, regulations, and standards you aught to be aware of, just to make sure you can speak intelligently about vehicles in general:

Automotive SPICE, SixSigma (DFMEA, HARA, FTA, FMEA), AUTOSAR, ASPICE, ISO26262, other Standards (ISO, IEEE, ANSI, ASTM, SAE, NHTSA, etc)

AUTOSAR (AUtomotice Open System ARchitecture) is a conglomerate of companies that came together to standardize the software architecture of ECUs. This made it possible for Tier 1 and even OEMs to develop their ECUs and have them work with standard tools for test and servicing.

Figure 11: ECU Design with AUTOSAR (Source)

Conclusion

That’s a lot of softwares to wrap your head around. It would be silly to try to learn them all. Simply pick the area that you would like to pursue and get really good at at least one tool from that area. The beauty of the roles at autonomous vehicle companies is that they’ll often use the clause “or like software/skill,” meaning that as long as you know one of the like tools your skill will be transferable.

Where do you go to learn the tools? That depends on the nature of the tool. Open source software typically has a fantastic community and many of their users are self-taught. Proprietary tools can be learned through university or continuous education programs. Some tools you’ll just have to learn on the job.

There are some great online programs, namely

’s Self Driving Car Nanodegree Program (Paid, two levels available) and MIT’s Deep Learning for Self-Driving Cars (free, self-paced) program by Lex Fridman.

Let us know what we forgot, and happy coding!

What Software Do Autonomous Vehicle Engineers Use? Part 1/2