Machine Learning for Autonomous Driving

Saturday December 14th, Vancouver Convention Centre, Vancouver


Welcome to the NeurIPS 2019 Workshop on Machine Learning for Autonomous Driving!

Autonomous vehicles (AVs) offer a rich source of high-impact research problems for the machine learning (ML) community; including perception, state estimation, probabilistic modeling, time series forecasting, gesture recognition, robustness guarantees, real-time constraints, user-machine communication, multi-agent planning, and intelligent infrastructure. Further, the interaction between ML subfields towards a common goal of autonomous driving can catalyze interesting inter-field discussions that spark new avenues of research, which this workshop aims to promote. As an application of ML, autonomous driving has the potential to greatly improve society by reducing road accidents, giving independence to those unable to drive, and even inspiring younger generations with tangible examples of ML-based technology clearly visible on local streets.

All are welcome to submit and/or attend! This will be the 4th NeurIPS workshop in this series. Previous workshops in 2016, 2017, and 2018 enjoyed wide participation from both academia and industry.


Paper submissions due: 17th September 2019 at 23:59 Anywhere on Earth
Author notifications: 1st October 2019
DiDi Challenge submission due: 30th November 2019
Argo Challenge submission due: 1st December 2019
Camera-ready papers due: 1st December 2019

Date: Saturday 14th December 2019
Room: 1, 2, and 3 (East Building, Meeting Level)


  • Vladlen Koltun

    9:00 AM

    Vladlen Koltun

    Director of Intelligent Systems Lab, Intel

  • Igor Gilitschenski

    10:30 AM

    Towards Robust Interactive Autonomy

    Igor Gilitschenski

    Postdoctoral Researcher, MIT

  • Chris Baker

    11:00 AM

    Human-inspired AI for autonomous driving

    Chris Baker

    Chief Scientist,

  • Raquel Urtasun

    1:30 PM

    Raquel Urtasun

    Associate Professor, UToronto / Chief Scientist at Uber

  • Eric Wolff

    2:30 PM

    Data, detection, and prediction at Aptiv.

    Eric Wolff

    Principal Research Scientist at nuTonomy (Aptiv)

  • Jaime Fisac

    5:00 PM

    Safety and Interaction: the Game Theory of Autonomous Vehicles

    Jaime Fisac

    UC Berkeley / Princeton, Waymo

  • Cathy Wu

    5:30 PM

    Mixed Autonomy Traffic: A Reinforcement Learning Perspective

    Cathy Wu

    Assistant Professor, MIT


Camera-ready deadline: 1st December 2019 at 23:59 Anywhere on Earth
Submission website:
Submission format: either extended abstracts (4 pages) or full papers (8 pages) anonymously using:
 •  neurips_2019_ml4ad.tex -- LaTeX template
 •  neurips_2019_ml4ad.sty -- style file for LaTeX 2e
 •  neurips_2019_ml4ad.pdf -- example PDF output
References and appendix should be appended into the same (single) PDF document, and do not count towards the page count.

We invite submissions on machine learning applied to autonomous driving, including (but not limited to):
 •  Supervised scene perception and classification
 •  Unsupervised representation learning for driving
 •  Behavior modeling of pedestrians and vehicles
 •  Gesture recognition
 •  Uncertainty propagation through AV software pipelines
 •  Metrics for autonomous driving
 •  Benchmarks for autonomous driving
 •  Real-time inference and prediction
 •  Causal modeling for multi-agent traffic scenarios
 •  Robustness to out-of-distribution road scenes
 •  Imitative driving policies
 •  Transfer learning from simulation to real-world (Sim2Real)
 •  Coordination with human-driven vehicles
 •  Coordination with vehicles (V2V) or infrastructure (V2I)
 •  Explainable driving decisions
 •  Adaptive driving styles based on user preferences


Q: Are dual submissions OK?
A: Yes.

Q: Will there be archival proceedings?
A: No. Neither 4 or 8 page submissions will be indexed nor have archival proceedings.

Q: Should submitted papers be anonymized?
A: Yes. If accepted, we will ask for a de-anonymized version to link on the website.

Q: What are the poster specifications?
A: Portrait: 36W x 48H inches (90W x 122H cm) on lightweight paper, not laminated.

Q: Where can I print my poster in Vancouver?
A: Options include and


Please note schedule is currently subject to small changes - schedule will be finalized closer to December 14th

Time Event Talk Title
8:50 AM Welcome
9:00 AM Vladlen Koltun Vladlen Koltun Intel
9:30 AM Posters + Coffee
10:30 AM Igor Gilitschenski Igor Gilitschenski MIT Towards Robust Interactive Autonomy
11:00 AM Chris Baker Chris Baker iSee Human-inspired AI for autonomous driving
11:30 AM ArgoAI Challenge
12:00 PM Lunch
1:30 PM Raquel Urtasun Raquel Urtasun UToronto, Uber
2:00 PM DiDi Challenge
2:30 PM Eric Wolff Eric Wolff nuTonomy Data, detection, and prediction at Aptiv.
3:00 PM Johannes LehnerContributed Talk 1 Patch Refinement - Localized 3D Object Detection
3:15 PM Apratim BhattacharyyaContributed Talk 2 Conditional Flow Variational Autoencoders for Structured Sequence Prediction
3:30 PM Posters + Coffee
4:30 PM Harkirat Singh BehlContributed Talk 3 Meta Learning Deep Visual Words for Fast Video Object Segmentation
4:45 PM Daniele RedaContributed Talk 4 Urban Driving With Conditional Imitation Learning
5:00 PM Jaime Fisac Jaime Fisac UC Berkeley / Princeton, Waymo Safety and Interaction: the Game Theory of Autonomous Vehicles
5:30 PM Cathy Wu Cathy Wu MIT Mixed Autonomy Traffic: A Reinforcement Learning Perspective


Argo Challenges:

We are hosting the first Argoverse competitions for 3d tracking and motion forecasting. The prize pool is $5,000 and winners will be invited to intern at Argo and present their methods at the workshop. More details can be found here.

Open for submissions through: 1st December 2019

DiDi Challenge:

We are also hosting the D²-City Detection Domain Adaptation Challenge, a transfer learning challenge for object detection. The prize pool is $1,700 and winners will be invited to present their methods at the workshop.

Open for submissions through: 30th November 2019


Accepted Full Papers:

Self-Driving like a Human driver instead of a Robocar: Personalized comfortable driving experience for autonomous vehicles
Il Bae, Jaeyoung Moon, Junekyo Jhung, Ho Suk, Taewoo Kim, Hyungbin Park, Jaekwang Cha, Jinhyuk Kim, Dohyun Kim, Shiho Kim

End-to-End Deep Path Planning and AutomaticEmergency Braking Camera Cocoon-based Solution
Eslam Mohamed Bakr, Mohammed Abdou, Omar Nasr

Spatial Influence-aware Reinforcement Learning for Intelligent Transportation System
Wenhang Bao, Xiao-Yang Liu

Meta Learning Deep Visual Words for Fast Video Object Segmentation
Harkirat Singh Behl, Mohammad Najafi, Anurag Arnab, Philip Torr

DynaNet: Neural Kalman Dynamical Model for Motion Estimation and Prediction
Changhao Chen, Chris Xiaoxuan Lu, Bing Wang, Niki Trigoni, Andrew Markham

Joint Triangulation and Mapping via Differentiable Sensor Fusion
Jonathan P. Chen*, Fritz Obermeyer*, Vladimir Lyapunov, Lionel Gueguen, Noah Goodman

SoildNet: Soiling Degradation Detection in Autonomous Driving
Arindam Das

Visibility Guided NMS: Efficient Boosting of Amodal Object Detection in Crowded Traffic Scenes
Nils Gählert, Niklas Hanselmann, Uwe Franke, Joachim Denzler

Urban Driving With Conditional Imitation Learning
Jeffrey Hawke, Richard Shen, Corina Gurau, Siddharth Sharma, Daniele Reda, Nikolay Nikolov, Przemyslaw Mazur, Sean Micklethwaite, Nicholas Griffiths, Amar Shah, Alex Kendall

Hybrid Sensor Fusion Framework for Perception in Autonomous Vehicles
Babak Shahian Jahromi, Theja Tulabandhula, Sabri Cetin

End-to-End Delay Analysis and Optimization of Object Detection Module for Autonomous Driving
Wootae Jeon, Kyungtae Kang, Jong-Chan Kim

Learning Adaptive driving behavior using Recurrent Deterministic Policy Gradients
Meha Kaushik, Nirvan Singhania, Madhava Krishna, Kaustubh Mani

Patch Refinement - Localized 3D Object Detection
Johannes Lehner, Andreas Mitterecker, Thomas Adler, Markus Hofmarcher, Bernhard Nessler, Sepp Hochreiter

Social Attention for Autonomous Decision-Making in Dense Traffic
Edouard Leurent*, Jean Mercat*

Radar and Camera Early Fusion for Vehicle Detection in Advanced Driver Assistance Systems
Teck Yian Lim, Amin Ansari

Benchmarking Robustness in Object~Detection: Autonomous Driving when Winter is Coming
Claudio Michaelis, Benjamin Mitzkus, Robert Geirhos, Evgenia Rusak, Oliver Bringmann, Alexander S Ecker, Matthias Bethge, Wieland Brendel

Improving Movement Prediction of Traffic Actors using Off-road Loss and Bias Mitigation
Matthew Niedoba, Henggang Cui, Kevin Luo, Darshan Hegde, Fang-Chieh Chou, Nemanja Djuric

Boosting semantic segmentation with multi-task self-supervised learning for autonomous driving applications
Jelena Novosel, Prashanth Viswanath, Bruno Arsenali

Simulation-based reinforcement learning for real-world autonomous driving
Błażej B Osiński, Adam Jakubowski, Piotr Miłoś, Paweł Zięcina, Christopher Galias, Silviu Homoceanu, Henryk Michalewski

Understanding Natural Language Parking Instructions and Grounding for Self-driving Cars
Nana Otawara, Hiroshi Tsukahara, Ichiro kobayashi, Atsushi Keyaki

Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learning
Praveen Palanisamy

RST-MODNet: Real-time Spatio-temporal MovingObject Detection for Autonomous Driving
Hazem Rashed, Mohamed Ramzy, Senthil Yogamani, Ahmad ElSallab

CNNs for Interference Mitigation and Denoising in Automotive Radar Using Real-World Data
Johanna Rock, Mate Toth, Paul Meissner, Franz Pernkopf

Single-step Options for Adversary Driving
Nazmus Sakib, Hengshuai Yao, Shangling Jui, Zhang Hong

Unsupervised Neural Sensor Models for Synthetic LiDAR Data Augmentation
Ahmad El Sallab*, Ibrahim Sobh*, Mohamed Zahran, Mohamed Shawky

Scribble Supervised Annotation Algorithms of Panoptic Segmentation for Autonomous Driving
Ruobing Shen, Bo Tang, Ismail Ben Ayed, Thomas Guthier

Approaching Neural Network Uncertainty Realism
Joachim Sicking*, Matthias Fahrland*, Alexander Kister, Stefan Eickeler, Fabian Hüger, Stefan Rueping, Peter Schlicht, Tim Wirtz

Efficient Black-box Assessment of Autonomous Vehicle Safety
Aman Sinha*, Matthew E OKelly*, Justin Norden*

Robust Imitative Planning: Planning from Demonstrations Under Uncertainty
Panagiotis Tigkas*, Angelos Filos*, Rowan McAllister, Nicholas Rhinehart, Sergey Levine, Yarin Gal

Quadratic Q-network for Learning Continuous Control for Autonomous Vehicles
Pin Wang, Hanhan Li, Ching-Yao Chan

Inverse Reinforcement Learning with ModelPredictive Control
Jinxin Zhao, Liangjun Zhang

Accepted Abstracts:

Learning to Drive using Waypoints
Tanmay Agarwal*, Hitesh Arora*, Tanvir Parhar*, Shubhankar V Deshpande, Jeff Schneider

Prediction by Imagination: A prediction method for handling low-probable actions
Ershad Banijamali, Mohsen Rohani

Conditional Flow Variational Autoencoders for Structured Sequence Prediction
Apratim Bhattacharyya, Michael Hanselmann, Mario Fritz, Bernt Schiele, Christoph-Nikolas Straehle

Joint Interaction and Trajectory Prediction for Autonomous Driving using Graph Neural Networks
Donsuk Lee, Yiming Gu, Jerrick Hoang, Micol Marchetti-Bowick

On the Role of Receptive Field in Unsupervised Sim-to-Real Image Translation
Nikita Jaipuria, Shubh Gupta, Praveen Narayanan, Vidya N. Murali

Semantic Domain Adaptation for Deep Classifiers via GAN-based Data Augmentation
Amitangshu Mukherjee, Ameya A Joshi, Chinmay Hegde, Soumik Sarkar

StarNet: Targeted Computation for Object Detection in Point Clouds
Jiquan Ngiam*, Benjamin Caine*, Wei Han, Brandon Yang, Yuning Chai, Pei Sun, Yin Zhou, Xi Yi, Ouais Alsharif, Patrick Nguyen, Zhifeng Chen, Vijay Vasudevan, Jonathon Shlens

A Reinforcement Learning Based Algorithm for Multi-hop Ride-sharing: Model-free Approach
Ashutosh Singh, Abubakr O Alabbasi, Vaneet Aggarwal

Safer End-to-End Autonomous Driving via Conditional Imitation Learning and Command Augmentation
Renhao Wang, Adam Scibior, Frank Wood



  • Rowan McAllister

    Rowan McAllister

    is a postdoctoral researcher at UC Berkeley working on probabilistic models for safe motion planning with autonomous vehicles.

  • Nicholas Rhinehart

    Nicholas Rhinehart

    is a Ph.D. student at Carnegie Mellon University, focusing on understanding, forecasting, and control with computer vision and machine learning.

  • Fisher Yu

    Fisher Yu

    is a postdoctoral researcher at UC Berkeley working in image recognition, interactive data processing systems, and high-level understanding of dynamic 3D scenes.

  • Li Erran Li

    Li Erran Li

    is head of machine learning at and an adjunct professor at Columbia University.

  • Anca Dragan

    Anca Dragan

    is an Assistant Professor at UC Berkeley and runs the InterACT Lab, focusing on algorithms for human-robot interaction.

Program Committee

   We thank those who help make this workshop possible!

        •  Yehya Abouelnaga
        •  Elmira Amirloo
        •  Hossein Azizpour
        •  Arjun Nitin Bhagoji
        •  Maciej Brzeski
        •  Sebastian Bujwid
        •  Wei-Lun Chao
        •  Weikai Chen
        •  Xinyun Chen
        •  Chen Chen
        •  Fang-Chieh Chou
        •  Henggang Cui
        •  Neema Davis
        •  Zihan Ding
        •  Nemanja Djuric
        •  Alexey Dosovitskiy
        •  Hesham M. Eraqi
        •  Andrien Gaidon
        •  Yang Gao
        •  Paweł Gora
        •  Mikael Henaff
        •  Ravi Kiran
        •  Edouard Leurent
        •  Zhuwen Li
        •  Xiaoyuan Liang
        •  Xiao-Yang Liu

        •  Jun Luo
        •  Sergio Macua
        •  Miquel Martí
        •  Marcin Możejko
        •  Lukas Neumann
        •  Thi Nguyen
        •  Gaurav Raina
        •  German Ros
        •  Ahmad El Sallab
        •  Axel Sauer
        •  Mark Schutera
        •  Ibrahim Sobh
        •  Zhaoen Su
        •  Anthony Tompkins
        •  Stephen Tu
        •  Andrea Vedaldi
        •  Dequan Wang
        •  Bin Wang
        •  Hua Wei
        •  Peyman Yadmellat
        •  Xinchen Yan
        •  Hengshuai Yao
        •  Mohamed Zahran
        •  Jiakai Zhang
        •  Yuzheng Zhuang


East Meeting Room 1,2,3 in the Vancouver Convention Center, 999 Canada Pl, Vancouver, BC V6C 3C1, Canada