Skip to content

Latest commit

 

History

History
93 lines (85 loc) · 5.12 KB

history.md

File metadata and controls

93 lines (85 loc) · 5.12 KB
Raw

History

There are many different version of TransFuser by now, and we sometimes see the wrong papers getting cited. Here is a short history over the different transFuser versions and how to cite them correctly:

TransFuser (CVPR 2021)

The first version of TransFuser. The paper Multi-Modal Fusion Transformer for End-to-End Autonomous Driving introduced the architecture back in 2021. The code is still available and can be found here. The models were developed in the early days of the CARLA leaderboard code / community, where dataset quality was quite poor. There is not much of a point to compare to this model any more unless you want to make your life easy.

@inproceedings{Prakash2021CVPR,
  author = {Prakash, Aditya and
            Chitta, Kashyap and
            Geiger, Andreas},
  title = {Multi-Modal Fusion Transformer for End-to-End Autonomous Driving},
  booktitle = {Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2021}
}

TransFuser+ (Master Thesis University of Tübingen 2021)

The thesis Expert Drivers for Autonomous Driving investigated the data quality issue of the original TransFuser work. It proposes a stronger automatic labeling algorithm. Together with auxiliary training, this pushed the performance of TransFuser a lot without changing the architecture. The document is available online but not formally published as it is only relevant to the core CARLA community. We would still appreciate it if you cite it where relevant. The code and models are not directly released, but the relevant code was published as part of the PAMI project.

@mastersthesis{Jaeger2021Thesis, 
	author = {Bernhard Jaeger}, 
	title = {Expert Drivers for Autonomous Driving}, 
	year = {2021}, 
	school = {University of Tübingen}, 
}

TransFuser (T-PAMI 2022)

The journal update to the CVPR paper. The paper TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving is published in Transaction on Pattern Analysis and Machine Intelligence. At the core it is still the TransFuser architecture, but it features better data, sensors, backbones, training and a rigorous set of ablations that shows what is important and what is not. The paper features a version called Latent TransFuser, which is a camera only TransFuser that replaces the LiDAR input by a positional encoding. The final models are roughly 4x better on the CARLA leaderboard than the CVPR TransFuser. This is the model you should be comparing to when you want to claim your method outperforms TransFuser. Code, models and data are available online.

@article{Chitta2022PAMI,
  author = {Chitta, Kashyap and
            Prakash, Aditya and
            Jaeger, Bernhard and
            Yu, Zehao and
            Renz, Katrin and
            Geiger, Andreas},
  title = {TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving},
  journal = {Pattern Analysis and Machine Intelligence (PAMI)},
  year = {2022},
}

TransFuser++ (ICCV 2023)

The ICCV 2023 paper Hidden Biases of End-to-End Driving Models offers some explanations why TransFuser and related approaches work so well. It also contains the latest and best performing models in the TransFuser family, called TransFuser++ (as well as a WP variant that uses waypoints as output representation). Code is available online. TransFuser++ is what you need to outperform to claim state-of-the-art performance on leaderboard 1.0.

@article{Jaeger2023ICCV,
  title={Hidden Biases of End-to-End Driving Models},
  author={Bernhard Jaeger and Kashyap Chitta and Andreas Geiger},
  booktitle={Proc. of the IEEE International Conf. on Computer Vision (ICCV)},
  year={2023}
}

TransFuser++ (Tech Report & Master Thesis)

Adapted version of TransFuser++ for the needs of the CARLA leaderboard 2.0 and PDM-Lite expert. The model reached 2nd place in the CVPR 2024 CARLA challenge. The technical report was submitted as part of the challenge to describe our solution. The model released in this repository is a reproduction of the challenge model. This model was created as part of a master thesis project which described the approach in more depth.

@Misc{Zimmerlin2024TECHREPORT,
  title={Hidden Biases of End-to-End Driving Datasets},
  author={Julian Zimmerlin and Jens Beißwenger and Bernhard Jaeger and Andreas Geiger and Kashyap Chitta},
  howpublished = {\textsc{url:}~\url{https://opendrivelab.github.io/Challenge%202024/carla_Tuebingen%20AI.pdf}},
  year={2024}
}

@mastersthesis{Zimmerlin2024thesis,
  title={Tackling CARLA Leaderboard 2.0 with End-to-End Imitation Learning},
  author={Julian Zimmerlin},
  school={University of Tübingen},
  howpublished={\textsc{url:}~\url{https://kashyap7x.github.io/assets/pdf/students/Zimmerlin2024.pdf}},
  year={2024}
}