• The cost of hardware in AVs, in particular LiDAR , have made companies innovate towards a camera only vision future
• While regulations will drive regional choices, the advancement in end-to-end, transformer-based AI models are enabling vision only perception systems
• LiDAR cost and form factors are too reducing, which may still yet make LiDAR relevant in future autonomous vehicle hybrid sensor suites

Access to the full report here: Re-thinking Autonomous Vehicle Sensors: Evolution to camera-based vision only future?

As autonomous vehicle (AV) technology evolves, a growing debate surrounds the future of sensor suites. Should the AVs of the future rely solely on cameras or will LiDAR and radar continue to play an essential role?

Historically, most AVs have used a multi-sensor approach: combining LiDAR, radar, and cameras to perceive and interpret their surroundings. This redundancy ensures high reliability, especially in complex driving environments. However, the cost and complexity of these systems have driven some companies to explore a camera-only, vision-based strategy underpinned by artificial intelligence.

Cost is the leading reason companies have made moves to do away with LiDAR. LiDAR sensors have traditionally been large, expensive, and power-hungry. These drawbacks have pushed some automakers toward using camera-only systems that are cheaper and lighter. Tesla and its CEO Elon Musk are the main proponents of the vision-only narrative. The company argues that since human drivers rely solely on vision, cars should be able to do the same with the right neural networks. Tesla deploys powerful end-to-end, transformer-based AI models so that cameras alone can enable full autonomy.

Evolution Pathway of Sensor Prices and Vision Only Perception

Regulations are also influencing this shift. In the U.S. and Europe, regulatory frameworks tend to be outcome-based and sensor-agnostic, allowing automakers the flexibility to decide how their systems are built as long as they meet safety and performance requirements. This opens the door for companies like Tesla to pursue vision-only systems without facing significant regulatory resistance.

In contrast, China is taking a more prescriptive approach. With formalized LiDAR standards and widespread adoption by local automakers, the country appears to favor multi-modal sensor suites that include LiDAR as a critical component. This divergence is likely to shape regional strategies.

Vison-only systems have significant technical and safety challenges that they must overcome. Camera-based systems, are known to struggle with edge cases that involve unusual and unpredictable scenarios such as low-light and adverse weather conditions. LiDAR and radar provide crucial redundancy in such edge cases, generating more precise 3D maps. In fact, the multi-modal sensor suites have been used to train AI models for use with camera-based system to validate AI models against a more accurate ground truth.

LiDAR, also once prohibitively expensive, is becoming increasingly viable as suppliers aggressively reduce costs. Solid-state LiDAR designs with smaller form factors also come with significantly lower production costs. In China, automakers are already equipping mass-market vehicles with LiDAR, thanks in part due to the aggressive cost reduction efforts by local manufacturers like Robosense and Hesai.

Lower cost LiDAR is expected to drive the decision by some OEMs to retain it for enhanced safety and redundancy, while some, like Tesla are working to remove LiDAR entirely, though the success of this strategy is dependent on how quickly camera-only systems can be proven safe and reliable under real-world conditions.

Additionally, the advances in radar, for example 4D imaging radar developed by the likes of Mobileye and distributed aperture radar (DAR) by Zendar, promise improved perception capabilities to complement cameras and reduce dependance on LiDAR

In the near to medium term, hybrid sensor suites that include cameras, LiDAR, and radar are expected to remain the standard, especially in robotaxis and commercial AVs. These systems offer greater robustness and help ensure vehicles have redundant perception capabilities, which is vital for gaining public trust and regulatory approval.

However, for some consumer-grade AVs, where cost and scalability are paramount, the balance may shift over time in favour of camera-only systems that can match or exceed the safety performance of multi-modal approaches. The reduced hardware cost could be too compelling to ignore, though this would only be possible if the technical and validation hurdles are overcome for camera-only systems.

The AV industry’s sensor strategy sits at a crossroads. On the one hand, we see a push toward vision-only systems driven by AI advancements and hardware cost savings. On the other, radar and LiDAR continue to evolve, becoming smaller, cheaper, and more practical for mass adoption. Different regions are pursuing different paths, and the ultimate direction will likely be determined by who can prove their approach is safest, most scalable, and most cost-effective.