aerial-autonomy-stack
Aerial autonomy stack (AAS) is a “batteries included”, flight-proven, open software and the simplest way to:
- Develop multi-drone autonomy—with PX4, ArduPilot, Betaflight, and ROS2
- Simulate faster-than-real-time perception and control—with YOLO and 3D LiDAR
- Deploy in real drones—with JetPack, DeepStream, and NVIDIA Orin
For an example bill of materials, read BOM.md; for motivation, read RATIONALE.md
https://github.com/user-attachments/assets/57e5bc91-8bee-4bae-8f81-a9aacef471e7
Features (click to expand)
- PX4, ArduPilot, and Betaflight multi-vehicle simulation (quadrotors, quadplane VTOLs, and tailsitters)
- ROS2 action-based autopilot interface (via XRCE-DDS, MAVROS, or MSP)
- YOLO (with ONNX GPU Runtimes) and LiDAR Odometry (with KISS-ICP)
- 3D worlds for perception-based simulation
- Steppable Gymnasium environment and faster-than-real-time, multi-instance simulation
- Gazebo’s wind effects and waves plugins
- Dockerized simulation based on Ubuntu with CUDA and cuDNN
- Dockerized deployment based on Ubuntu with CUDA, cuDNN, TensorRT and DeepStream
- Windows 11 compatibility via WSL
- Multi-Jetson-in-the-loop (HITL) simulation to test NVIDIA- and ARM-based on-board compute
- Dual network to separate simulated sensors (
SIM_SUBNET) and inter-vehicle comms (AIR_SUBNET)
- Zenoh inter-vehicle ROS2 bridge
- PX4 Offboard interface (e.g. CTBR/
VehicleRatesSetpoint for agile, GNSS-denied flight)
- PX4 ROS 2 Interface Library external flight modes
- ArduPilot Guided interface (i.e.
setpoint_velocity, setpoint_accel references)
- Betaflight MSP Override interface (i.e. RC sticks in ACRO, ANGLE, or ALTHOLD + POS HOLD)
- Logs analysis with
plot_logs.py (.ulg, .bin); SITL logs and MCAP rosbags (RECORD_ROSBAG=true) are copied to the host at the end of each simulation
1. Installation
AAS is developed with Ubuntu 24.04 and nvidia-driver-615-open on an i7-11 with RTX 3060
Read REQUIREMENTS_UBUNTU.md (or REQUIREMENTS_WSL.md for Windows 11) to install the requirements
sudo apt update && sudo apt install -y git xterm xfonts-base wget unzip
git clone https://github.com/JacopoPan/aerial-autonomy-stack.git
cd aerial-autonomy-stack/tools_and_docs/
./tests/check_requirements.sh # AAS requires nvidia-driver, docker, and nvidia-container-toolkit
./sim_build.sh # The 1st build takes ~45' with good internet (`Ctrl + c` and restart if needed, cached stages will be preserved)
ghcr.io pre-built images (click to expand)
# ghcr.io images are re-built from `main` every Friday night
for name in aircraft ground simulation; do
docker pull ghcr.io/jacopopan/${name}-image:latest
docker tag ghcr.io/jacopopan/${name}-image:latest ${name}-image:latest
done
2. Multi-drone Simulation
Start AAS:
cd aerial-autonomy-stack/tools_and_docs/
NUM_QUADS=1 NUM_VTOLS=1 WORLD=swiss_town RTF=3 PLOT=true ./sim_run.sh # Start a simulation, check the script for more options (note: ArduPilot SITL checks take ~30s of simulated time before being ready to arm)
# Simulation options:
# AUTOPILOT=px4, ardupilot, betaflight
# HEADLESS/CAMERA/LIDAR/PX4_ROS2_LIB=true, false
# NUM_QUADS/NUM_VTOLS/NUM_TAILS=0, 1, ...
# ODOM=none, openvins, fastlio, superodom, mimosa (all except "none" require BUILD_ADVANCED_ODOM=true)
# WORLD=impalpable_greyness, apple_orchard, crematoria, shibuya_crossing, swiss_town, waterworld
# RTF=1, 2, ... (real-time-factor, use 0 for "as fast as possible", PX4_ROS2_LIB=true or AUTOPILOT=betaflight set RTF=1)
# INSTANCE=0, 1, ... (integer ID to run multiple parallel simulations)
# PLOT/RECORD_ROSBAG=true, false (plotting requires pymavlink, pyulog, pymap3d)
There are 3 different ways to autonomously fly the drones (plus USB gamepad for manual flight and QGroundControl/Betaflight App for operator supervision)
- From the
Ground’s terminal, fly all drones in a synchronized formation with dtc_controller_node:
ros2 run drone_traffic_controller dtc_controller --ros-args -p use_sim_time:=true
- From any
QUAD/VTOL/TAIL terminal, fly its own behavior tree mission (e.g., yalla.yaml):
ros2 run mission mission --conops yalla.yaml --ros-args -r __ns:=/Drone$DRONE_ID -p use_sim_time:=true
- From any
QUAD/VTOL/TAIL terminal, use ROS2 actions for px4_offboard/ardupilot_guided/betaflight_rc_override controllers:
cancellable_action "ros2 action send_goal /Drone${DRONE_ID}/takeoff_action \
autopilot_interface_msgs/action/Takeoff '{takeoff_altitude: 30.0}'"
cancellable_action "ros2 action send_goal /Drone${DRONE_ID}/offboard_action \
autopilot_interface_msgs/action/Offboard \
'{controller_name: att-test, max_duration_sec: 10.0}'"
# Add or re-implement offboard controllers in `px4_offboard.cpp`, `ardupilot_guided.cpp`, `betaflight_rc_override.cpp`
AAS also includes px4-ros2-interface-lib support, run:
PX4_ROS2_LIB=true ./sim_run.sh
# Then, take-off using QGroundControl and switch to one of the "AAS <controller>" modes implemented in px4_offboard.cpp
To add and fly more custom modes, from a PX4 QUAD terminal run:
ros2 run px4_custom_mode_template px4_custom_mode_template --ros-args -r __ns:=/Drone${DRONE_ID} -p use_sim_time:=true -p setpoint_type:=goto # Use setpoint_type:=trajectory, attitude, or rates to fly the same mode with a different PX4 control loop
# Then, take-off using QGroundControl and switch to mode "AAS Template" (implemented in aircraft/aircraft_ws/src/px4_custom_mode_template/)
WORLDs:
(i) apple_orchard, a GIS world created using BlenderGIS
/ (ii) impalpable_greyness, an empty world with simple shapes
/ (iii) shibuya_crossing, a 3D world adapted from cgtrader
/ (iv) swiss_town, a photogrammetry world courtesy of Pix4D / pix4d.com
/ (v) waterworld, a dynamic world using the asv_wave_sim wave plugin
/ (vi) crematoria, based on DARPA SubT’s simple_cave_02 (CC BY 4.0)
Edit sensor_config.yaml, then run sim_build.sh, to customize the sensor parameters
Use ROS2 drone and gimbal control primitives from CLI (click to expand)
# Takeoff action (quads and VTOLs/tailsitters)
cancellable_action "ros2 action send_goal /Drone${DRONE_ID}/takeoff_action autopilot_interface_msgs/action/Takeoff '{takeoff_altitude: 40.0, vtol_transition_heading: 330.0, vtol_loiter_north: 200.0, vtol_loiter_east: 100.0, vtol_loiter_alt: 120.0}'"
# Land (at home) action (quads and VTOLs/tailsitters), note: on ArduPilot, overwrites params RTL_ALT/Q_RTL_ALT
cancellable_action "ros2 action send_goal /Drone${DRONE_ID}/land_action autopilot_interface_msgs/action/Land '{landing_altitude: 60.0, vtol_transition_heading: 60.0}'"
# Orbit action (quads and VTOLs/tailsitters), note: on ArduPilot, overwrites param CIRCLE_RATE, not implemented on Betaflight
cancellable_action "ros2 action send_goal /Drone${DRONE_ID}/orbit_action autopilot_interface_msgs/action/Orbit '{east: 500.0, north: 0.0, altitude: 150.0, radius: 200.0}'"
# Reposition service (quads only)
ros2 service call /Drone${DRONE_ID}/set_reposition autopilot_interface_msgs/srv/SetReposition '{east: 50.0, north: 100.0, altitude: 60.0}'
# Offboard action (Specify the flight behavior via `controller_name`, e.g., "traj-test" for PX4 or "vel-test" for ArduPilot and Betaflight)
cancellable_action "ros2 action send_goal /Drone${DRONE_ID}/offboard_action autopilot_interface_msgs/action/Offboard '{controller_name: traj-test, max_duration_sec: 5.0}'"
# SetSpeed service (ephemeral, limited by the autopilot params, for quads applies from the next command, not effective on ArduPilot VTOLs, not implemented on Betaflight)
ros2 service call /Drone${DRONE_ID}/set_speed autopilot_interface_msgs/srv/SetSpeed '{speed: 3.0}'
# Gimbal status and position control (in radians)
ros2 topic echo /gimbal_state
ros2 topic pub -1 /gimbal_pitch_cmd std_msgs/msg/Float64 "{data: 1.57}"
Add or disable wind effects, in the Simulation's terminal (click to expand)
python3 /aas/simulation_resources/scripts/gz_wind.py --from_west 0.0 --from_south 3.0
python3 /aas/simulation_resources/scripts/gz_wind.py --stop_wind
# Note: only use with the quadrotor models as the simulated airspeed sensors on VTOLs/tailsitters are not Gazebo wind-aware
Develop within live containers (click to expand)
Launching the sim_run.sh script with DEV=true does not start the simulation and mounts folders, [aircraft|ground]_ws/src as volumes to more easily track, commit, push changes while building and testing them within the containers:
cd aerial-autonomy-stack/tools_and_docs/
DEV=true ./sim_run.sh # Starts one simulation-image, one ground-image, and one aircraft-image where the *_ws/src/ folders are mounted from the host
To build changes—made on the host—in the Ground or QUAD terminal:
cd /aas/aircraft_ws/ # Or cd /aas/ground_ws/
colcon build --symlink-install
To start the simulation, in the QUAD terminal:
tmuxinator start -p /aas/aircraft.yml.erb
In the Ground terminal:
tmuxinator start -p /aas/ground.yml.erb
In the Simulation terminal:
tmuxinator start -p /aas/simulation.yml.erb
To end the simulation, in each terminal detach Tmux with Ctrl + b, then d; kill all lingering processes with tmux kill-server && pkill -f gz
BUILD_ADVANCED_ODOM=true ./sim_build.sh adds to the aircraft-image several VIO and LIVO packages, including openvins, rovio, fastlio, superodom, and mimosa
3. Deployment on Jetson
AAS is tested on a Holybro Jetson Baseboard with Pixhawk 6X and NVIDIA Orin NX 16GB
The default quad is a Holybro X650 with the IMX219 camera and the Livox Mid-360S LiDAR
Read SETUP_AVIONICS.md and BOM.md to setup the requirements on the Jetson and configure the Pixhawk/FCU
Read SETUP_CHRONY.md to let the Jetson timesync to the ground-image computer when w/o internet
git clone https://github.com/JacopoPan/aerial-autonomy-stack.git
cd aerial-autonomy-stack/tools_and_docs/
./deploy_build.sh # Build for arm64, on Jetson Orin NX the first build requires ~50' to build onnxruntime-gpu with TensorRT support from source
# Build options:
# PREBUILT_DEPS=true, false (use a pre-built image on ghcr.io and only build stages aircraft-manifests/aircraft-dev-image)
On a Jetson Orin, start the aircraft-image:
cd aerial-autonomy-stack/tools_and_docs/
DRONE_ID=1 CAMERA=true LIDAR=false AIR_SUBNET=10.223 HEADLESS=true ./deploy_run.sh # The first run of `./deploy_run.sh` requires ~5-10' to build the TensorRT cache
# Deployment options:
# DRONE_TYPE=quad, vtol, tail
# AUTOPILOT=px4, ardupilot, betaflight
# DRONE_ID=1, 2, ... (ROS_DOMAIN_ID of the drone, matching the MAV_SYS_ID/SYSID_THISMAV of the autopilot)
# HEADLESS/CAMERA/LIDAR/PX4_ROS2_LIB/RECORD_ROSBAG/RECORD_VIDEO=true, false (recordings are saved on the host's ~/Downloads/ folder)
# REMOTE_VIDEO_STREAMS=true, false (cam_0 stream to the ground container, toggle at runtime with topic /enable_remote_video_streams)
# ODOM=none, openvins, fastlio, superodom, mimosa (all except "none" require BUILD_ADVANCED_ODOM=true)
On a laptop, start the ground-image (QGC, Zenoh, SSH, and GStreamer):
cd aerial-autonomy-stack/tools_and_docs/
./sim_build.sh # Build all images for amd64, including ground-image
GROUND=true QUAD_IDS=1 AIR_SUBNET=10.223 HEADLESS=false ./deploy_run.sh # Optionally, use offline QGC maps placing file qgcMapCache.qgctiledb in the host's ~/Downloads/ folder
HITL Simulation (click to expand)
Note: HITL simulation validates the Jetson compute and the inter-vehicle network.
Use USB2.0 ASIX AX88772A Ethernet adapters to add network interfaces (besides the one between Orin and 6X on the board).
Lock each interface’s “MAC Address” under “Network Options” -> “Identity”.
Set up a LAN on an arbitrary SIM_SUBNET with netmask 255.255.0.0 (e.g. 172.30.x.x, gateway 172.30.1.1, using a GL.iNet Flint 2 router) between:
- One simulation computer, with IP
[SIM_SUBNET].90.100
- One ground computer, with IP
[SIM_SUBNET].90.101
N Jetson Baseboards with IPs [SIM_SUBNET].90.1, …, [SIM_SUBNET].90.N
Optionally, set up the real-world AIR_SUBNET with netmask 255.255.0.0 (e.g. 10.223.x.x, if using Doodle Labs Mesh Rider Nano radios) between:
- One ground computer, with IP
[AIR_SUBNET].90.101
N Jetson Baseboards with IPs [AIR_SUBNET].90.1, …, [AIR_SUBNET].90.N
First, start the ground container:
# On the computer with IPs ending in 90.101
HITL=true GROUND=true QUAD_IDS=1,2 SIM_SUBNET=172.30 AIR_SUBNET=10.223 HEADLESS=false ./deploy_run.sh
Then, start all aircraft containers, one on each Jetson (e.g. via SSH from the ground container):
# On the Jetson with IPs ending in 90.1
HITL=true DRONE_ID=1 SIM_SUBNET=172.30 AIR_SUBNET=10.223 ./deploy_run.sh # Add HEADLESS=false if a screen is connected to the Jetson
# On the Jetson with IPs ending in 90.2
HITL=true DRONE_ID=2 SIM_SUBNET=172.30 AIR_SUBNET=10.223 ./deploy_run.sh # Add HEADLESS=false if a screen is connected to the Jetson
Finally, start the simulation container:
# On the computer with IPs ending in 90.100
HITL=true NUM_QUADS=2 SIM_SUBNET=172.30 PLOT=true ./sim_run.sh
Note: running only the last 3 commands with GND_CONTAINER=false puts the Zenoh bridge on the SIM_SUBNET, removing the need for the optional AIR_SUBNET and the computer with IP ending in 90.101
Once done, detach Tmux (and remove the containers) with tmux kill-server
4. Gymnasium RL Environment
Using a Python venv or a conda environment is optional but recommended (click to expand)
wget https://repo.anaconda.com/archive/Anaconda3-2025.12-2-Linux-x86_64.sh # Or a newer version in https://repo.anaconda.com/archive/
bash Anaconda3-2025.12-2-Linux-x86_64.sh # Install; start a new terminal
conda config --set auto_activate_base false # Turn off auto initialization of (base); start a new terminal
conda update --all -n base -c defaults # Update to the latest conda version
conda create -n aas python=3.12 # Latest Python version beyond "bugfix" status https://devguide.python.org/versions/
Install the aas-gym package (after completing the steps in “Installation”):
conda activate aas # If using Anaconda
cd aerial-autonomy-stack/aas-gym/
pip3 install -e .
Examples:
conda activate aas # If using Anaconda
cd aerial-autonomy-stack/tools_and_docs/
python3 gym_run.py --mode step # Manually step AAS @1Hz
python3 gym_run.py --mode speedup # Speed-up test @50Hz
python3 gym_run.py --mode vectorenv-speedup # Vectorized speed-up test @50Hz
Appendix A: Citation
@INPROCEEDINGS{panerati2026aas,
author={Jacopo Panerati and Sina Sajjadi and Sina Soleymanpour and Varunkumar Mehta and Iraj Mantegh},
booktitle={2026 International Conference on Unmanned Aircraft Systems (ICUAS)},
title={{aerial-autonomy-stack}---a faster-than-real-time, autopilot-agnostic, {ROS2} framework to simulate and deploy perception-based drones},
year={2026}}
Appendix B: Architecture
%%{init: {'theme': 'base', 'flowchart': {'wrappingWidth': 200}, 'themeVariables': { 'fontFamily': 'monospace'}}}%%
flowchart TB
subgraph aas [" "]
subgraph sim ["#nbsp;simulation#nbsp;container#nbsp;(amd64)"]
sitl("[N x] PX4 || ArduPilot ||#nbsp;Betaflight#nbsp;SITL"):::resource
gz(Gazebo Sim):::resource
subgraph models [Models]
drones(aircraft_models/):::resource
worlds(simulation_worlds/):::resource
end
drones --> gz
worlds --> gz
sitl <--> |"gz_bridge || ardupilot_gazebo <br/> || aeroloop_gazebo"| gz
end
subgraph gnd ["#nbsp;ground#nbsp;container#nbsp;(amd64)"]
mlrouter{{mavlink_router}}:::bridge
ground_system(ground_system):::algo
dtc_controller(dtc_controller):::algo
qgc(QGroundControl):::resource
bf_app("Betaflight#nbsp;App"):::resource
zenoh_gnd{{zenoh_bridge}}:::bridge
ground_system --> |"/tracks"| zenoh_gnd
dtc_controller --> |"/dtc_commands"| zenoh_gnd
mlrouter <--> qgc
mlrouter --> ground_system
end
subgraph air ["[N#nbsp;x]#nbsp;aircraft#nbsp;container(s)#nbsp;(amd64,#nbsp;arm64)"]
subgraph perception [Perception]
yolo_py(yolo_py):::algo
kiss_icp(kiss_icp):::algo
livo_pkgs(livo_pkgs):::algo
end
zenoh_air{{zenoh_bridge}}:::bridge
subgraph control [Control]
dtc_client(dtc_client):::algo
mission(mission):::algo
offboard_control(offboard_control):::algo
autopilot_interface(autopilot_interface):::algo
state_sharing(state_sharing):::algo
end
detection_split( ):::splitNode
track_split( ):::splitNode
ap_link{{"uxrce_dds || MAVROS#nbsp;|| MSP"}}:::bridge
kiss_icp -.-> |"/TBD"| ap_link
livo_pkgs <-.-> |"/imu_data <br/> /TBD"| ap_link
zenoh_air --> |"/tracks <br/> /state_drone_N"| track_split
track_split --> offboard_control
track_split --> mission
zenoh_air --> |"/dtc_commands"| dtc_client
ap_link --> state_sharing
ap_link <--> autopilot_interface
yolo_py --> |"/detections"| detection_split
detection_split --> offboard_control
detection_split --> mission
offboard_control --> |"/ctrl_ref"| autopilot_interface
mission --> |"ros2 action"| autopilot_interface
dtc_client --> |"ros2 action"| autopilot_interface
zenoh_air <--> |"/state_drone_n"| state_sharing
end
lidar_split( ):::splitNode
camera_split( ):::splitNode
end
gz --> |"/lidar_points <br/> DDS/UDP [SIM_SUBNET]"| lidar_split
lidar_split --> kiss_icp
lidar_split --> livo_pkgs
gz --> |"gz_gst_bridge <br/> RTP/UDP [SIM_SUBNET]"| camera_split
camera_split --> livo_pkgs
camera_split --> yolo_py
sitl <--> |"XRCE-DDS/UDP || MAVLink/UDP <br/> || MSP/TCP [SIM_SUBNET]"| ap_link
sitl <--> |"MAVLink/UDP [SIM_SUBNET]"| mlrouter
sitl <--> |"MSP/TCP [SIM_SUBNET]"| bf_app
zenoh_gnd <-.-> |"Zenoh/TCP <br/> [AIR_SUBNET]"| zenoh_air
classDef bridge fill:#ffebd6,stroke:#f5a623,stroke-width:2px;
classDef algo fill:#e1f5fe,stroke:#0277bd,stroke-width:2px;
classDef resource fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px;
classDef splitNode fill:#cccccc,stroke:#666666,stroke-width:2px;
classDef blueStyle fill:#e1f0ff,stroke:#666,stroke-width:2px; class aas blueStyle;
classDef whiteStyle fill:#f9f9f9,stroke:#666,stroke-width:1px,stroke-dasharray: 5 5; class air,gnd,sim whiteStyle;
classDef greyStyle fill:#eeeeee,stroke:#666,stroke-width:1px,stroke-dasharray: 5 5; class perception,control,models greyStyle;
linkStyle 22,23,24,25,26,27,28,29,30 stroke:teal,stroke-width:3px; linkStyle 31 stroke:blue,stroke-width:4px;
Repository structure (click to expand)
aerial-autonomy-stack
│
├── aas-gym
│ └── src
│ └── aas_gym
│ └── aas_env.py # aerial-autonomy-stack as a Gymnasium environment
│
├── aircraft
│ ├── aircraft_ws
│ │ └── src
│ │ ├── autopilot_interface # Ardupilot/PX4/Betaflight high-level actions (Takeoff, Orbit, Offboard, Land)
│ │ ├── drone_traffic_client # Subscriber of topic `/dtc_commands` enforcing high-level actions from the ground
│ │ ├── imu_publisher # Multiplexer between PX4/DDS, ArduPilot/MAVROS, and Betaflight/MSP sensor topics
│ │ ├── mission # Orchestrator of the actions in `autopilot_interface`
│ │ ├── offboard_control # Low-level references for the Offboard action in `autopilot_interface`
│ │ ├── state_sharing # Publisher of the `/state_sharing_drone_N` topic broadcasted by Zenoh
│ │ └── yolo_py # GStreamer video acquisition and publisher of YOLO bounding boxes
│ │
│ └── aircraft.yml.erb # Aircraft docker tmux entrypoint
│
├── ground
│ ├── ground_ws
│ │ └── src
│ │ ├── drone_traffic_controller # Publisher of topic `/dtc_commands` broadcasted by Zenoh
│ │ └── ground_system # Publisher of topic `/tracks` broadcasted by Zenoh
│ │
│ └── ground.yml.erb # Ground docker tmux entrypoint
│
├── simulation
│ ├── simulation_resources
│ │ ├── aircraft_models
│ │ │ ├── alti_transition_quad # ArduPilot VTOL model
│ │ │ ├── iris_with_ardupilot # ArduPilot quad model
│ │ │ ├── iris_with_betaflight # Betaflight quad model
│ │ │ ├── swan_k1_hwing # ArduPilot tailsitter model
│ │ │ ├── sensor_camera # Camera model
│ │ │ ├── sensor_gimbal # 3D gimbal used with sensor_camera
│ │ │ ├── sensor_lidar # LiDAR model
│ │ │ ├── standard_vtol # PX4 VTOL model
│ │ │ ├── x500 # PX4 quad model
│ │ │ ├── quadtailsitter # PX4 tailsitter model
│ │ │ └── sensor_config.yaml # Intrinsics and extrinsics for all sensor and vehicle models
│ │ └── simulation_worlds
│ │ ├── apple_orchard.sdf
│ │ ├── crematoria.sdf
│ │ ├── impalpable_greyness.sdf
│ │ ├── shibuya_crossing.sdf
│ │ ├── swiss_town.sdf
│ │ └── waterworld.sdf
│ │
│ └── simulation.yml.erb # Simulation docker tmux entrypoint
│
└── tools_and_docs
├── docker
│ ├── aircraft.dockerfile # Docker image for aircraft simulation and deployment
│ ├── ground.dockerfile # Docker image for ground system simulation and deployment
│ └── simulation.dockerfile # Docker image for SITL and HITL simulation
│
├── deploy_build.sh # Build `aircraft.dockerfile` for arm64/Orin
├── deploy_run.sh # Start the aircraft docker on arm64/Orin or the ground docker on amd64 (deploy or HITL)
│
├── gym_run.py # Examples for the Gymnasium aas-gym package
│
├── sim_build.sh # Build all dockerfiles for amd64/simulation
└── sim_run.sh # Start the simulation (SITL or HITL)
Dependencies management (click to expand)
External repositories (cloned by sim_build.sh and deploy_build.sh):
You’ve done a man’s job, sir. I guess you’re through, huh?
评论