LiDAR_fog_sim

Created by Martin Hahner at the Computer Vision Lab of ETH Zurich.

Fog Simulation on Real LiDAR Point Clouds
for 3D Object Detection in Adverse Weather

by Martin Hahner, Christos Sakaridis, Dengxin Dai, and Luc van Gool

Accepted at ICCV 2021.
Please visit our paper website for more details.

Overview

.
├── file_lists                          # contains file lists for pointcloud_viewer.py
│   └── ...
├── integral_lookup_tables              # contains lookup tables to speed up the fog simulation
│   └── ... 
├── extract_fog.py                      # to extract real fog noise* from the SeeingThroughFog dataset
├── fog_simulation.py                   # to augment a clear weather pointcloud with artificial fog (used during training)
├── generate_integral_lookup_table.py   # to precompute the integral inside the fog equation
├── pointcloud_viewer.py                # to visualize entire point clouds of different datasets with the option to augment fog into their scenes
├── README.md
└── theory.py                           # to visualize the theory behind a single LiDAR beam in foggy conditions

* Contains returns not only from fog, but also from physical objects that are closeby.

Datasets supported by pointcloud_viewer.py:

• H3D
• A2D2
• KITTI
• LyftL5
• Pandaset
• nuScenes
• Argoverse
• ApolloScape
• SeeingThroughFog  :arrow_left: works best
• WaymoOpenDataset (via waymo_kitti_converter)

License

This software is made available for non-commercial use under a Creative Commons License.
A summary of the license can be found here.

Acknowledgments

This work is supported by Toyota via the TRACE project.

Furthermore, we would like to thank the authors of SeeingThroughFog for their great work.
In this repository, we use a fork of their original repository to visualize annotations and compare to their fog simulation. Their code is licensed via the MIT License.

Citation(s)

If you find this work useful, please consider citing our paper.

@inproceedings{HahnerICCV21,
  author = {Hahner, Martin and Sakaridis, Christos and Dai, Dengxin and Van Gool, Luc},
  title = {{Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather}},
  booktitle = {IEEE International Conference on Computer Vision (ICCV)},
  year = {2021},
}

You may also want to check out our latest work (Oral at CVPR 2022)
LiDAR Snowfall Simulation for Robust 3D Object Detection.

@inproceedings{HahnerCVPR22,
  author = {Hahner, Martin and Sakaridis, Christos and Bijelic, Mario and Heide, Felix and Yu, Fisher and Dai, Dengxin and Van Gool, Luc},
  title = {{LiDAR Snowfall Simulation for Robust 3D Object Detection}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2022},
}

Getting Started

Setup

1. Install anaconda.

2. Create a new conda environment.

conda create --name foggy_lidar python=3.9 -y

3. Activate the newly created conda environment.

conda activate foggy_lidar

4. Install all necessary packages.

conda install matplotlib numpy opencv pandas plyfile pyopengl pyqt pyqtgraph quaternion scipy tqdm -c conda-forge -y
pip install pyquaternion

5. Clone this repository (including submodules).

git clone [email protected]:MartinHahner/LiDAR_fog_sim.git --recursive
cd LiDAR_fog_sim

Usage

How to run the script that visualizes the theory behind a single LiDAR beam in foggy conditions:

python theory.py

How to run the script that visualizes entire point clouds of different datasets:

python pointcloud_viewer.py -d <path_to_where_you_store_your_datasets>

Note:

You may also have to adjust the relative paths in pointcloud_viewer.py (right at the beginning of the file) to be compatible with your datasets relative folder structure.

Disclaimer

The code has been successfully tested on

• Ubuntu 18.04.5 LTS
• macOS Big Sur 11.2.1
• Debian GNU/Linux 9.13

using conda 4.9.2.

Contributions

Please feel free to suggest improvements to this repository.
We are always open to merge usefull pull request.