Physical AI Autonomous Vehicles

Overview

A large-scale, multi-sensor dataset for AV research released by NVIDIA. It contains driving data collected globally across a wide variety of geographies, road types, and environmental conditions, intended to support development and evaluation of end-to-end driving systems, perception, scene understanding, and related AV tasks.

Modality: Video, LiDAR, radar
- Multi-camera and LiDAR coverage for all clips; radar coverage for a subset of clips
Annotations | Labels: Calibration files, sensor metadata, time-aligned signals, and machine-generated annotations; core dataset consists of raw sensor captures
Size: 1,727 hours of driving data comprising 310,895 clips (each 20 seconds in length)
Format:
- Camera data: MP4 video
- LiDAR and radar data: parquet or zipped binary files
- Metadata in structured formats (timestamps, calibration, environmental attributes)
Structure:
- Clips grouped into chunk directories
- Separate folders per sensor type
- Metadata supports filtering by sensor availability, geography, weather, and other conditions

Typical Uses

Train/eval of AV perception models (object detection, scene understanding)
Multi-sensor fusion for driving tasks
End-to-end driving
Simulation, scenario mining

Notable Features

Geographic, environmental, and traffic diversity:
- About 50% of data from US + 50% from 24 EU countries
- Captures diverse traffic patterns, road types, weather, time of day
Specifically designed for "Physical AI"

Limitations

Access requires accpetance of NVIDIA's dataset license agreement
Data volume & complexity (~= 1 TB)
Radar coverage not present for all clips

Access

License / Source Information

Dataset Owner: NVIDIA Corportation
Dataset may only be used for AV-related use cases; can be commercial or non-commercial as long as the license terms are abided by (license agreement on huggingface)
Publication Date: 2025-10-28