Machine Learning for Autonomous Driving

NeurIPS 2021 Virtual Workshop

December 13th

About

Welcome to the NeurIPS 2021 Workshop on Machine Learning for Autonomous Driving!

Autonomous vehicles (AVs) offer a rich source of high-impact research problems for the machine learning (ML) community; including perception, state estimation, probabilistic modeling, time series forecasting, gesture recognition, robustness guarantees, real-time constraints, user-machine communication, multi-agent planning, and intelligent infrastructure. Further, the interaction between ML subfields towards a common goal of autonomous driving can catalyze interesting inter-field discussions that spark new avenues of research, which this workshop aims to promote. As an application of ML, autonomous driving has the potential to greatly improve society by reducing road accidents, giving independence to those unable to drive, and even inspiring younger generations with tangible examples of ML-based technology clearly visible on local streets. All are welcome to attend! This will be the 6th NeurIPS workshop in this series. Previous workshops in 2016, 2017, 2018, 2019 and 2020 enjoyed wide participation from both academia and industry.

Attending:
1. Register for NeurIPS
2. Watch talks live from our NeurIPS Portal and ask questions in the "Chat" window (begins 7:50am PST on December 13th)
3. Chat with authors during the GatherTown poster sessions (9:30am, 3:00pm PST)

Post Event: all videos now avilable here

Dates

Papers

→Submission due: 1st October 2021
→Reviewing starts: 4th October 2021
→Reviewing ends: 18th October 2021
→Notification: 22nd October 2021
→Video upload: 1st November 2021
→Camera Ready: 5th November 2021

Challenge

→Open: 15th September 2021
→Close: 19th November 2021
→Notification: 22nd November 2021
→Video submission: 1st December 2021

Workshop

→Workshop event: 13th December 2021

Speakers

Submissions

Submission deadline: 1st October 2021 at 23:59 Anywhere on Earth
Submission website: https://cmt3.research.microsoft.com/ML4AD2021
Submission format: either extended abstracts (4 pages) or full papers (8 or 9 pages) anonymously using:
 •  neurips_2021_ml4ad.tex -- LaTeX template
 •  neurips_2021_ml4ad.sty -- style file for LaTeX 2e
 •  neurips_2021_ml4ad.pdf -- example PDF output
References and appendix should be appended into the same (single) PDF document, and do not count towards the page count.

We invite submissions on machine learning applied to autonomous driving, including (but not limited to):
 •  Supervised scene perception and classification
 •  Unsupervised representation learning for driving
 •  Behavior modeling of pedestrians and vehicles
 •  Gesture recognition
 •  Uncertainty propagation through AV software pipelines
 •  Metrics for autonomous driving
 •  Benchmarks for autonomous driving
 •  Real-time inference and prediction
 •  Causal modeling for multi-agent traffic scenarios
 •  Robustness to out-of-distribution road scenes
 •  Imitative driving policies
 •  Transfer learning from simulation to real-world (Sim2Real)
 •  Coordination with human-driven vehicles
 •  Coordination with vehicles (V2V) or infrastructure (V2I)
 •  Explainable driving decisions
 •  Adaptive driving styles based on user preferences

FAQ

Q: Will there be archival proceedings?
A: No. Neither 4 or 8 or 9 page submissions will be indexed nor have archival proceedings.

Q: Should submitted papers be anonymized?
A: Yes. If accepted, we will ask for a de-anonymized version to link on the website.

Q: My papers contains ABC, but not XYZ, is this good enough for a submission?
A: Submissions will be evaluated based on these reviewer questions.

Schedule

Monday, December 13th, 2021. All times are in Pacific Time. Current time is

Time Event Talk Title
07:50 Welcome
08:00 Jeff Schneider Jeff Schneider Carnegie Mellon University Reinforcement Learning for Self Driving Cars
08:40 Alex Kendall Alex Kendall Wayve AV2.0: Learning a Globally Scalable Driving Intelligence
09:20 Best Paper Oral Christopher Diehl UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planning
09:30 Poster Session
10:30 Break
11:00 Rose Yu Rose Yu University of California San Diego Physics-Guided AI for Modeling Autonomous Vehicle Dynamics
11:40 Matthew Johnson-Roberson Matthew Johnson-Roberson University of Michigan The Ongoing Research in University of Michigan & Ford Center for Autonomous Vehicles (FCAV)
12:20 CARLA Challenge German Ros CARLA Autonomous Driving Challenge 2021
13:20 Break
13:30 Katie Driggs-Campbell Katie Driggs-Campbell University of Illinois at Urbana-Champaign Fantastic Failures and Where to Find Them: Designing Safe, Robust Autonomy
14:10 Claire Tomlin Claire Tomlin University of California, Berkeley Modeling Other Agents
14:50 Spotlight Talk Letian Wang Object-Level Targeted Selection via Deep Template Matching
14:53 Spotlight Talk Suraj Nandkishor Kothawade Hierarchical Adaptable and Transferable Networks (HATN) for Driving Behavior Prediction
14:56 Spotlight Talk Mingze Wang Incorporating Voice Instructions in Model-Based Reinforcement Learning for Self-Driving Cars
15:00 Poster Session
16:00 Mark Palatucci Mark Palatucci Waymo Learning Driving Agents from Simulation
16:40 Songan Zhang Songan Zhang Ford Autonomous Vehicle Decision-Making Policy Fast Adaptation using Meta Reinforcement Learning
17:20 Sarah Tang Sarah Tang Waymo Robotics for an ML-Driven World
18:00 Shifts Challenge Andrey Malinin Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks
18:20 Closing Remarks

Challenge




The CARLA Autonomous Driving Challenge 2021 winners will present their solutions as part of the workshop. Details here.


Invited Talks


Learning from All Vehicles
Dian Chen, Philipp Krähenbühl
video | slides

Imitation with Transformer-Based Sensor Fusion
Kashyap Chitta, Zehao Yu, Bernhard Jaeger, Katrin Renz, Andreas Geiger
video | slides

GRIAD: General Reinforced Imitation for Autonomous Driving
Raphael Chekroun, Marin Toromanoff, Sascha Hornauer, Fabien Moutarde
video | slides

Papers


Best Paper


UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planning
Christopher Diehl, Timo Sievernich, Martin Krüger, Frank Hoffmann, Torsten Bertram
paper | poster H3 | video

Spotlight Talks


Object-Level Targeted Selection via Deep Template Matching
Suraj Kothawade, Donna Roy, Michele Fenzi, Elmar Haussmann, Jose M. Alvarez, Christoph Angerer
paper | poster D3 | video

Hierarchical Adaptable and Transferable Networks (HATN) for Driving Behavior Prediction
Letian Wang, Yeping Hu, Liting Sun, Wei Zhan, Masayoshi Tomizuka, Changliu Liu
paper | poster C1 | video

Incorporating Voice Instructions in Model-Based Reinforcement Learning for Self-Driving Cars
Mingze Wang, Ziyang Zhang, Grace Hui Yang
paper | poster E1 | video

Accepted Papers And Abstracts


Real-time Generalized Sensor Fusion with Transformers
Aayush Ahuja, Jen Li, Shili Xu, Vivek Rane, Ashesh Jain, Balazs Kovacs
paper | poster B1 | video

Circular-Symmetric Correlation Layer
Bahar Azari, Deniz Erdoğmuş
paper | poster A3 | video

MTL-TransMODS: Cascaded Multi-Task Learning for Moving Object Detection and Segmentation with Unified Transformers
Eslam Mohamed Bakr, Ahmad El Sallab
paper | poster B2 | video

PKCAM: Previous Knowledge Channel Attention Module
Eslam Mohamed Bakr, Ahmad El Sallab, Mohsen A. Rashwan
paper | poster B3 | video

Fast Polar Attentive 3D Object Detection on LiDAR Point Clouds
Manoj Bhat, Steve Han, Fatih Porikli
paper | poster A0 | video

Improved Object Detection in Thermal Imaging Through Context Enhancement and Information Fusion: A Case Study in Autonomous Driving
Junchi Bin*, Ran Zhang*, Chengkai Zhang*
paper | poster A2 | video

Compressing Sensor Data for Remote Assistance of Autonomous Vehicles using Deep Generative Models
Daniel Bogdoll*, Johannes Jestram*, Christin Scheib*, Moritz Wittig*, Jonas Rauch*, J. Marius Zöllner
paper | poster H2 | video

Self-supervised Sun Glare Detection CNN for Self-aware Autonomous Driving
Yiqiang Chen, Feng Liu, Ke Pei
paper | poster H0 | video

NSS-VAEs: Generative Scene Decomposition for Visual Navigable Space Construction
Zheng Chen, Lantao Liu
paper | poster C0 | video

Monocular 3D Object Detection by Leveraging Self-Supervised Visual Pre-training
Can Erhan, Anil Ozturk, Burak Gunel, Nazim Kemal Ure
paper | poster A1 | video

PolyTrack: Tracking with Bounding Polygons
Gaspar Faure, Hughes Perreault, Guillaume-Alexandre Bilodeau, Nicolas Saunier
paper | poster E3 | video

Does Thermal Data Make the Detection Systems More Reliable?
Shruthi Gowda, Bahram Zonooz, Elahe Arani
paper | poster D1 | video

A Scenario-Based Platform for Testing Autonomous Vehicle Behavior Prediction Models in Simulation
Francis Indaheng, Edward Kim, Kesav Viswanadha, Jay Shenoy, Jinkyu Kim, Daniel J. Fremont, Sanjit A. Seshia
paper | poster F1 | video

Watch Out for the Risky Actors: Assessing Risk in Dynamic Environments for Safe Driving
Saurabh Jha, Yan Miao, Zbigniew Kalbarczyk, Ravishankar K. Iyer
paper | poster G2 | video

Reinforcement Learning as an Alternative to Reachability Analysis for Falsification of AD Functions
Tobias Johansson, Angel Molina Acosta, Alexander Schliep, Paolo Falcone
paper | poster D0 | video

Efficient Unknown Object Detection with Discrepancy Networks for Semantic Segmentation
Ryo Kamoi, Takumi Iida, Kaname Tomite
paper | poster H1 | video

DriverGym: Democratising Reinforcement Learning for Autonomous Driving
Parth Kothari, Christian Perone, Luca Bergamini, Alexandre Alahi, Peter Ondruska
paper | poster E2 | video

ORDER: Open World Object Detection on Road Scenes
Deepak Kumar Singh*, Shyam Nandan Rai*, K J Joseph, Rohit Saluja, Vineeth N Balasubramanian, Chetan Arora, Anbumani Subramanian, C.V. Jawahar
paper | poster C3 | video

Spatio-Temporal Gated Transformers for Efficient Video Processing
Yawei Li 1, Babak Ehteshami Bejnordi, Bert Moons, Tijmen Blankevoort, Amirhossein Habibian, Radu Timofte, Luc Van Gool
paper | poster F3 | video

TITRATED: Learned Human Driving Behavior without Infractions via Amortized Inference
Vasileios Lioutas, Adam Ścibior, Frank Wood
paper | poster F0 | video

Meta-Guided Metric Learner for Overcoming Class Confusion in Few-Shot Road Object Detection
Anay Majee, Anbumani Subramanian, Kshitij Agrawal
paper | poster G3 | video

How Far Can I Go?: A Self-Supervised Approach for Deterministic Video Depth Forecasting
Sauradip Nag*, Nisarg A. Shah*, Anran Qi*
paper | poster F2 | video

Switching Recurrent Kalman Networks
Giao Nguyen-Quynh, Philipp Becker, Chen Qiu, Maja Rudolph
paper | poster E0 | video

Self-Supervised Pretraining for Scene Change Detection
Vijaya Raghavan T. Ramkumar, Prashant Bhat, Elahe Arani, Bahram Zonooz
paper | poster D2 | video

A Step Towards Efficient Evaluation of Complex Perception Tasks in Simulation
Jonathan Sadeghi, Blaine Rogers, James Gunn, Thomas Saunders, Sina Samangooei, Puneet Kumar Dokania, John Redford
paper | poster G1 | video

AA3DNet: Attention Augmented Real Time 3D Object Detection
Abhinav Sagar
paper | poster I0 | video

Offline Reinforcement Learning for Autonomous Driving with Safety and Exploration Enhancement
Tianyu Shi, Dong Chen, Kaian Chen, Zhaojian Li
paper | poster B0 | video

Temporal Transductive Inference for Few-Shot Video Object Segmentation
Mennatullah Siam, Konstantinos G. Derpanis, Richard P. Wildes
paper | poster G0 | video

Scalable Primitives for Generalized Sensor Fusion in Autonomous Vehicles
Sammy Sidhu, Linda Wang, Tayyab Naseer, Ashish Malhotra, Jay Chia, Aayush Ahuja, Ella Rasmussen, Qiangui Huang, Ray Gao
paper | poster C2 | video

Organizers


  • Xinshuo Weng

    Xinshuo Weng    xinshuow@cs.cmu.edu

    is a PhD student at Carnegie Mellon University working on 3D Computer Vision and Graph Neural Networks in the context of autonomous driving.


  • Jiachen Li

    Jiachen Li    jiachen_li@stanford.edu

    is a Postdoctoral Scholar at Stanford University working on relational reasoning and graph neural networks for trajectory forecasting and decision making of multi-agent systems.


  • Nicholas Rhinehart

    Nicholas Rhinehart    nrhinehart@berkeley.edu

    is a postdoctoral researcher at UC Berkeley, focusing on understanding, forecasting, and control with computer vision and machine learning.


  • Daniel Omeiza

    Daniel Omeiza    daniel.omeiza@cs.ox.ac.uk

    is a PhD student at the University of Oxford working on explainability in autonomous vehicles.


  • Ali Baheri

    Ali Baheri    ali.baheri@mail.wvu.edu

    is an assistant professor at West Virginia University working on machine learning, control, and data-driven optimization.


  • Rowan McAllister

    Rowan McAllister    rowan.mcallister@tri.global

    is a research scientist at Toyota Research Institute working on probabilistic models and motion planning for autonomous vehicles.

Challenge Organizers

Volunteers

A special thanks to workshop volunteers Letian Wang, Bahar Azari, Shah Nisarg Anish, and Sauradip Nag for their help hosting this virtual workshop!

Program Committee

   We thank those who help make this workshop possible!