By ICLR 2021 Senior Program Chair Katja Hofmann (Microsoft Research), and Program Co-Chairs Ivan Titov (U. Edinburgh / U. Amsterdam), Naila Murray (Facebook AI Research), and Alice Oh (KAIST)

ICLR 2021 is now behind us and looking back, we thoroughly enjoyed the spotlight and poster presentations, our keynote talks and live Q&As, the socials and mentoring sessions, and the opportunities to catch up with our colleagues around the world at the ICLR cafe. Videos of all keynotes and oral sessions are available to the public at https://slideslive.com/iclr-2021 and at the virtual conference site https://iclr.cc/virtual/2021/calendar. As we reflect on the last 12 months spent preparing for ICLR 2021, we wanted to share with you some of our observations and lessons learned.

The first is that ICLR continues to be a growing community. To illustrate this, for ICLR 2021 we received 3014 full submissions compared to 2604 in 2020, and at the conference there were 6194 registrations, compared to 5622 registrations in 2020. This growth requires continuously reinforcing and refining our community practices, including our submission authoring and review process, our codes of conduct and ethics guidelines, and our diversity and inclusion efforts.

When preparing the review process last year, there was a great deal of uncertainty around the number of submissions to expect, given the extraordinary circumstances we all faced. We also faced a compressed timeline due to our decision to set the submission deadline after the NeurIPS 2020 acceptance decisions. Given these factors, we introduced a mandatory abstract submission stage, which allowed us to get a sense of whether our reviewer and area chair pools were sufficiently large and topically diverse. We also adopted a Code of Ethics this year, and we’ll soon publish a separate blog post dedicated to our experiences putting the CoE into practice at ICLR 2021.

Our conference schedule featured round-the-clock live content, in order to be inclusive to attendees in different time zones around the world. For the spotlight and poster sessions in particular, we struck a balance between topical coherence and the preferred session time-slots selected by presenters. We generally found a good balance, although sessions falling into some time periods were over-subscribed (and we thank the authors for their flexibility in these cases!).

This was our second virtual ICLR and, having learned a lot from our successful 2020 experience, we were pleased to find that the 2021 conference went quite smoothly. For glitches that did arise, our 200-strong volunteer team, as well as the EventHosts and SlidesLive teams, were quick to respond and resolve. Conference attendees hailed from more than 60 countries around the world, and every continent except for Antarctica (maybe next year!). Throughout the week, there were 16,624 views of our different livestreams. We were very pleased with the degree of engagement in the mentoring program using the Mementor portal, which was organized by Emtiyaz Khan, Marc Deisenroth, and Hendrik Strobelt. In total, about 1950 people signed up for Mementor and there were 42 mentoring sessions throughout the conference week. It would be very helpful to us to hear about your experiences at ICLR 2021 so we encourage those of you who attended to fill out the survey sent by email.

ICLR 2021, as with every edition, is a large team effort, and the program committee would once again like to thank everyone who made it possible, including the other organizing committee members, the authors, reviewers, area chairs, volunteers, keynote speakers, the OpenReview team, the Event Hosts team, and the SlidesLive team. Lastly, we would like to wish the incoming general chair Katja Hofmann (Microsoft), senior program chair Yan Liu (University of Southern California), and program chairs Yejin Choi (University of Washington), Marc Deisenroth (UCL), and Chelsea Finn (Stanford University), every success in putting together an amazing ICLR 2022.

The countdown has started — we’re less than 1 week away from kicking off ICLR 2021!

On our side, we cannot wait to welcome you to this year’s virtual conference. The program will be packed with inspiring talks, Q&As, and plenty of opportunity for discussion. Here we give you a first look of what to expect, together with tips on how to make the most of attending the virtual format.

Central to the conference program and planning are our paper authors and attendees. We aim to make the fantastic conference content as easily accessible as possible, and provide a virtual environment where you can engage in the way that works best for you.

From the first day of the conference, all asynchronous content will be made available on the virtual conference site. This includes pre-recorded videos for all 860 accepted papers. You can browse papers, search for specific topics, and discover inspiring content through the virtual interface. Also from day one, we enable rocket chat where you can discuss papers and ask questions to the authors. We encourage you to engage throughout the event.

Complementing the asynchronous content and interaction is our ICLR 2021 live program. This is a round-the-clock program consisting of talks and live Q&As, poster sessions and socials. Our goal: no matter where you are, and no matter what times you attend the conference, there will be inspiring content and plenty of opportunities for engagement for you. Oral sessions with talks and live Q&As alternate with poster sessions and socials to allow you to make the most of the time you spend learning and discussing.

If you need help in accessing any content, or if you have questions regarding the conference, there will be continuous helpdesk support — thanks to the efforts of 200 selected volunteers from all over the world.

The 2021 ICLR live program at a glance

At a glance — here’s an overview of the live content:

The full conference schedule is now available on virtual ICLR. Below is an overview of the exciting content to look out for.

Opening and closing: in the conference opening you will meet the organizing teams and learn more about the conference program, while the closing rounds out the conference with a sneak peak of what’s to come.

Invited talks: hear from some of the leading researchers in the ICLR and related communities, who share their perspectives on the field, inspiring ideas, and outlook on the future. You can read more about our invited speakers in our recent blog post, and find them in the ICLR 2021 program Invited Talks overview.

Oral sessions: papers accepted for Oral or Spotlight presentation will be featured here. Sessions consist of talks, as well as live Q&As with all authors. Authors look forward to your questions, which you can submit until and throughout the relevant session through rocket chat.

Outstanding Paper Award sessions: two sessions are reserved for the Outstanding Paper Award winners. The format of these sessions follows that of talks and live Q&As. You can read about the winning papers in our blog post Announcing ICLR 2021 Outstanding Paper Awards. The Outstanding Paper Award sessions take place on May 5, 4PM PDT and May 6, midnight PDT.

Poster sessions: held in Gathertown, these are your opportunity to engage with authors, ask questions and discuss ideas. All 860 accepted papers will be presented here by the authors.

Townhall: the conference organizers and special guests share thoughts about the conference format and wider questions about the future direction of the ICLR community. The townhall session takes place on May 4, 4PM PDT, and will feature short talks and time for live Q&A with all attendees. You can post your questions through rocket chat — we look forward to this discussion with you.

Socials: a great way to meet other attendees, socials are community organized events that take place throughout the conference. We have 28 fantastic Socials with something for everybody: Affinity Group Socials, Competitions, Talks & Panels, Game Events, Group Discussion, as well as places for simply meeting people and socializing. Check out the Socials overview to find those you want to join.

Workshops: taking place on the day after the main conference, workshops provide a more focused venue to discuss emerging directions and exciting trends. You can learn more about workshops on our workshop overview page.

Mentoring Sessions: We will use Mementor to enable ad-hoc mentorship sessions. These are spontaneous web calls that allow mentors and mentees to connect to talk about a specific topic in ML or about research life in general.

Throughout the conference, please keep in mind the ICLR Code of Conduct. It is designed to encourage respectful scientific exchange for all attendees, so it is crucial that all conference interactions adhere to it. For more details, please visit Code of Conduct (iclr.cc).

If you’re ready to dive in, we recommend registering today at https://iclr.cc/, and Get Started at virtual ICLR 2021.

A warm welcome and see you 3–7 May at virtual ICLR 2021!

We are thrilled to announce the winners of the ICLR 2021 Outstanding Paper Awards. While there are 860 excellent papers in our program, and many of them of exceptional quality, we would like to highlight 8 papers that are especially noteworthy.

Award winners (in alphabetical order):

• Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with 1/n Parameters by Aston Zhang, Yi Tay, Shuai Zhang, Alvin Chan, Anh Tuan Luu, Siu Hui, and Jie Fu
• Complex Query Answering with Neural Link Predictors by Erik Arakelyan, Daniel Daza, Pasquale Minervini, and Michael Cochez
• EigenGame: PCA as a Nash Equilibrium by Ian Gemp, Brian McWilliams, Claire Vernade, and Thore Graepel
• Learning Mesh-Based Simulation with Graph Networks by Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter Battaglia
• Neural Synthesis of Binaural Speech from Mono Audio by Alexander Richard, Dejan Markovic, Israel D. Gebru, Steven Krenn, Gladstone Alexander Butler, Fernando Torre, and Yaser Sheikh
• Optimal Rates for Averaged Stochastic Gradient Descent under Neural Tangent Kernel Regime by Atsushi Nitanda, and Taiji Suzuki
• Rethinking Architecture Selection in Differentiable NAS by Ruochen Wang, Minhao Cheng, Xiangning Chen, Xiaocheng Tang, and Cho-Jui Hsieh
• Score-Based Generative Modeling through Stochastic Differential Equations by Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole

Congratulations to the authors on this fantastic achievement!

The paper awards were selected as follows:

• based on paper reviews, we constructed a shortlist of candidate papers;
• the shortlisted papers were sent to the Outstanding Paper Committee for further review;
• the experts were asked to assess not only technical quality but also the potential for impact, including bringing in a new perspective, opening up an exciting direction, and making a strong contribution to addressing an important problem;
• the highest-ranking papers were selected for the awards.

Thank you to the members of the Outstanding Paper Award Committee for their hard work!

The papers will be presented during two Outstanding Paper sessions (May 5, 16.00–17.00 PDT and May 6, 00:00–01:00 PDT).

To attend the event and participate in these and many other exciting sessions, register at https://iclr.cc/Register/view-registration.

Outstanding Paper Award Committee

• Adam White, University of Alberta
• Aditya Krishna Menon, Australian National University
• Adriana Romero, Facebook
• Amir Globerson, Tel Aviv University
• André Martins, Instituto Superior Técnico / Unbabel
• Franziska Meier, Facebook
• Irina Higgins, DeepMind
• Michael S Ryoo, Stony Brook State University of New York
• Mingyuan Zhou, The University of Texas at Austin
• Natalia Neverova, Facebook
• Panayotis Mertikopoulos, French National Center for Scientific Research
• Piotr Koniusz, Australian National University
• Rémi Flamary, Ecole Polytechnique
• Tengyu Ma, Stanford University
• Tie-Yan Liu, Microsoft
• Yale Song, Microsoft

It’s time for a big drum-roll … today we are announcing our invited speakers for ICLR 2021! Much activity precedes this exciting moment. First, we collect a long list of candidate speakers, many of whom are suggestions from the community, and especially our ICLR Area Chairs. Second, we in the program committee consider all suggestions to take into account how each would contribute to widening the range of perspectives at the event. Finally, we invite speakers (and cross our fingers for them to accept our invitation), before confirming talk details with each one of them.

Without further ado, we are incredibly honored to have the following amazing speakers contribute their unique perspectives to ICLR 2021:

• Lourdes Agapito
  Perceiving the 3D World from Images and Video
• Michael Bronstein
  Geometric Deep Learning: the Erlangen Programme of ML
• Kyu Jin Cho
  Soft bodied robots for human centered design of robots for everyday life
• Yejin Choi
  Commonsense Intelligence: Cracking the Longstanding Challenge in AI
• Alexei (Alyosha) Efros
  Self-Supervision for Learning from the Bottom Up
• Timnit Gebru
  Moving beyond the fairness rhetoric in machine learning
• Kate Saenko
  Is My Dataset Biased?
• Manuela Veloso
  AI in Finance: Scope and Examples

At ICLR 2021, each invited speaker will give a half-hour talk, followed by a live Q&A. This is a unique opportunity to learn from our speakers’ expertise and insights, understand their perspective on key challenges and opportunities, and get inspired to seize these opportunities.

I personally cannot wait to see the invited talks and participate in the live Q&A for all speakers at ICLR 2021 from May 3 to May 7, 2021. To join the event, register now at: https://iclr.cc/Register/view-registration.

See you at the conference!

Read on to learn more about each of our invited speakers:

Lourdes Agapito
Perceiving the 3D World from Images and Video

Lourdes Agapito holds the position of Professor of 3D Vision at the Department of Computer Science, University College London (UCL). Her research in computer vision has consistently focused on the inference of 3D information from single images or videos acquired from a single moving camera. She received her BSc, MSc and PhD degrees from the Universidad Complutense de Madrid (Spain). In 1997 she joined the Robotics Research Group at the University of Oxford as an EU Marie Curie Postdoctoral Fellow. In 2001 she was appointed as Lecturer at the Department of Computer Science at Queen Mary University of London. From 2008 to 2014 she held an ERC Starting Grant funded by the European Research Council to focus on theoretical and practical aspects of deformable 3D reconstruction from monocular sequences. In 2013 she joined the Department of Computer Science at University College London and was promoted to full professor in 2015. She now heads the Vision and Imaging Science Group, is a founding member of the AI centre and co-director of the Centre for Doctoral Training in Foundational AI. Lourdes serves regularly as Area Chair for the top Computer Vision conferences (CVPR, ICCV, ECCV) was Program Chair for CVPR 2016 and will serve again for ICCV 2023. She was keynote speaker at ICRA 2017, the top robotics conference, and is a member of the executive committee of the British Machine Vision Association. In 2017 she co-founded Synthesia, the London based synthetic media startup responsible for the AI technology behind the Malaria no More video campaign that saw David Beckham speak 9 different languages to call on world leaders to take action to defeat Malaria.

Michael Bronstein
Geometric Deep Learning: the Erlangen Programme of ML

Michael Bronstein is a professor at Imperial College London, where he holds the Chair in Machine Learning and Pattern Recognition, and Head of Graph Learning Research at Twitter. He also heads ML research in Project CETI, a TED Audacious Prize-winning collaboration aimed at understanding the communication of sperm whales. Michael received his PhD from the Technion in 2007. He has held visiting appointments at Stanford, MIT, and Harvard, and has also been affiliated with the Institute for Advanced Study at TUM (as a Rudolf Diesel Fellow, 2017–2019) and Harvard (as a Radcliffe fellow, 2017–2018). Michael is the recipient of the Royal Society Wolfson Research Merit Award, Royal Academy of Engineering Silver Medal, five ERC grants, two Google Faculty Research Awards, and two Amazon AWS ML Research Awards. He is a Member of the Academia Europaea, Fellow of IEEE, IAPR, BCS, and ELLIS, ACM Distinguished Speaker, and World Economic Forum Young Scientist. In addition to his academic career, Michael is a serial entrepreneur and founder of multiple startup companies, including Novafora, Invision (acquired by Intel in 2012), Videocites, and Fabula AI (acquired by Twitter in 2019). He has previously served as Principal Engineer at Intel Perceptual Computing and was one of the key developers of the Intel RealSense technology.

Kyu Jin Cho
Soft bodied robots for human centered design of robots for everyday life

Kyu Jin Cho is a Professor and the Director of Soft Robotics Research Center and Biorobotics Lab at Seoul National University. He received his Ph.D. in mechanical engineering from MIT and his B.S and M.S. from Seoul National University. He was a postdoctoral fellow at Harvard Microrobotics Laboratory before joining SNU in 2008. He has been exploring novel soft bio-inspired robot designs, including a water jumping robot, various shape changing robots and soft wearable robots for the disabled. He has received the 2014 IEEE RAS Early Academic Career Award for his fundamental contributions to soft robotics and biologically inspired robot design. He has published a Science paper on the water jumping robot and several papers in Science Robotics with novel robot designs. He serves RAS as associate VP of Publication Activities Board, and will serve as VP of the RAS Technical Activities Board next year.

Yejin Choi
Commonsense Intelligence: Cracking the Longstanding Challenge in AI

Yejin Choi is a Brett Helsel associate professor at the Paul G. Allen School of Computer Science & Engineering at the University of Washington and also a senior research manager at AI2 overseeing the project Mosaic. Her research interests include language grounding with vision, physical and social commonsense knowledge, language generation with long-term coherence, conversational AI, and AI for social good. She is a co-recipient of the AAAI Outstanding Paper Award in 2020, a recipient of Borg Early Career Award (BECA) in 2018, among the IEEE’s AI Top 10 to Watch in 2015, a co-recipient of the Marr Prize at ICCV 2013, and a faculty advisor for the Sounding Board team that won the inaugural Alexa Prize Challenge in 2017. Her work on detecting deceptive reviews, predicting the literary success, and interpreting bias and connotation has been featured by numerous media outlets including NBC News for New York, NPR Radio, New York Times, and Bloomberg BusinessWeek. She received her Ph.D. in Computer Science from Cornell University.

Alexei (Alyosha) Efros
Self-Supervision for Learning from the Bottom Up

Alexei (Alyosha) Efros is a professor of computer science at UC Berkeley and member of the BAIR lab. Prior to that, he was nine years on the faculty of Carnegie Mellon University, and has also been affiliated with École Normale Supérieure/INRIA and University of Oxford. His research is in the area of computer vision and computer graphics, especially at the intersection of the two. He is particularly interested in using data-driven techniques to tackle problems where large quantities of unlabeled visual data are readily available. Efros received his PhD in 2003 from UC Berkeley. He is a recipient of the Sloan Fellowship (2008), Guggenheim Fellowship (2008), Okawa Grant (2008), SIGGRAPH Significant New Researcher Award (2010), 3 PAMI-TC Helmholtz Test-of-Time Prizes (1999,2003,2005), the ACM Prize in Computing (2016), and Diane McEntyre Award for Excellence in Teaching Computer Science (2019). He likes Paris and gelato.

Timnit Gebru
Moving beyond the fairness rhetoric in machine learning

Timnit Gebru was recently fired by Google for raising issues of discrimination in the workplace. Prior to that she was a co-lead of the Ethical AI research team at Google Brain. She received her PhD from the Stanford Artificial Intelligence Laboratory, studying computer vision under Fei-Fei Li, and did a postdoc at Microsoft Research, New York City in the FATE (Fairness Accountability Transparency and Ethics in AI) group, where she studied algorithmic bias and the ethical implications underlying projects aiming to gain insights from data. Timnit also co-founded Black in AI, a nonprofit that works to increase the presence, inclusion, visibility and health of Black people in the field of AI.

Kate Saenko
Is My Dataset Biased?

Kate Saenko is an Associate Professor of Computer Science at Boston University and a consulting professor for the MIT-IBM Watson AI Lab. She received a PhD from MIT and was a postdoc at UC Berkeley and Harvard. Her research is on representation learning for visual and language data, with a focus on generalizable, explainable, data-efficient and power-efficient AI models.

Manuela Veloso
AI in Finance: Scope and Examples

Manuela M. Veloso is the Head of J.P. Morgan AI Research, which pursues fundamental research in areas of core relevance to financial services, including data mining and cryptography, machine learning, explainability, and human-AI interaction. J.P. Morgan AI Research partners with applied data analytics teams across the firm as well as with leading academic institutions globally. Professor Veloso is on leave from Carnegie Mellon University as the Herbert A. Simon University Professor in the School of Computer Science, and the past Head of the Machine Learning Department. With her students, she had led research in AI, with a focus on robotics and machine learning, having concretely researched and developed a variety of autonomous robots, including teams of soccer robots, and mobile service robots. Her robot soccer teams have been RoboCup world champions several times, and the CoBot mobile robots have autonomously navigated for more than 1,000km in university buildings. Professor Veloso is the Past President of AAAI, (the Association for the Advancement of Artificial Intelligence), and the co-founder, Trustee, and Past President of RoboCup. Professor Veloso has been recognized with multiple honors, including being a Fellow of the ACM, IEEE, AAAS, and AAAI. She is the recipient of several best paper awards, the Einstein Chair of the Chinese Academy of Science, the ACM/SIGART Autonomous Agents Research Award, an NSF Career Award, and the Allen Newell Medal for Excellence in Research. Professor Veloso earned a Bachelor and Master of Science degrees in Electrical and Computer Engineering from Instituto Superior Tecnico in Lisbon, Portugal, a Master of Arts in Computer Science from Boston University, and Master of Science and PhD in Computer Science from Carnegie Mellon University. See www.cs.cmu.edu/~mmv/Veloso.html for her scientific publications.

Please note that all talk details are preliminary and may change as the talks and program are finalized.

By ICLR 2021 Senior Program Chair Katja Hofmann (Microsoft Research Cambridge) and Program Co-Chairs Naila Murray (Facebook), Alice Oh (KAIST), Ivan Titov (U. Edinburgh)

Acceptance notifications for ICLR 2021 were released on January 12. Out of the 2997 full submissions that were received, a total of 860 have been accepted for presentation at the conference. Our heartfelt congratulations to all authors! We thank all authors for submitting their work to ICLR, and hope that even if your work was not accepted at this time, you will consider joining the exciting conference ahead.

The accepted contributions will make up an exciting academic program. The word cloud shown in the figure gives you a first glimpse of the range of topics covered. All accepted submissions will be presented as virtual posters. In addition, 114 accepted submissions have been selected for Spotlight presentations, and 53 will be presented in longer Oral presentations. The full list of accepted papers can be found on Open Review, listed by presentation type: Oral, Spotlight, and Poster presentations.

A lot of work has happened behind the stage between the paper submission in October and acceptance notifications in January. None of this would have been possible without the dedication and energy of all those involved. We are deeply grateful to all authors, reviewers, area chairs, and the amazing team at open review for making it all happen. In numbers: we recruited 4086 reviewers and emergency reviewers who contributed a total of just over 11,500 reviews. The discussion and decision process was guided by 450 area chairs and emergency area chairs.

This year’s discussion and rebuttal phase proceeded in stages, starting with a broad public and author discussion, and closing with a focused discussion between area chairs and reviewers. The aim of this staged discussion was to ensure area chairs had all relevant information by the time they needed to finalize decisions and meta review. The following statistics give a sense of the scale of discussion during the review and rebuttal phase. Note that just over 300 submissions were withdrawn during this phase, and the statistics only consider the submissions that were active at the end of the rebuttal process but before acceptance notification. Authors of 84% of active submissions updated their manuscripts after reviews were released, and the authors of 93% of active submissions sent rebuttal messages that were visible to the reviewers. For the same set of active submissions, 75% had at least one review that was updated during the rebuttal phase. 46% of active submissions had a public discussion, and 61% of active submissions had a private discussion between reviewers and/or area chairs. Members of the public (other than authors, reviewers and area chairs) were able to contribute comments during the first discussion stage, and 142 submissions received a total of 186 public comments. The discussion continues, as public comments on all submissions have now been re-enabled.

A common question from authors is to what extent the rebuttal process influences reviews. We can shed some light on this based on the following data. We identified the reviews that were updated after reviews had been released to authors on November 10, and compared ratings before and after updates. We found that 32% of all reviews were updated at least once during the rebuttal phase. Of these, scores increased for 54%, decreased for 9%, and remained unchanged for 37% of all updated reviews.

The organizing team is ramping up preparations for the virtual conference on May 4–8, and we are excited about the fantastic program you’ll get to enjoy. We will share details over the coming months — one of the next announcements will be about our keynote speakers.

Stay tuned!

One major change in the submission process this year was the abstract submission. As ICLR grows rapidly every year, it is difficult to estimate the number of submissions. With an abstract deadline, we get a few more days to make sure we have enough area chairs and reviewers to handle the submissions we receive. Also, area chairs (ACs) get a head start on bidding on abstracts, and assigning ACs before reviewers allows ACs to make reviewer recommendations. That in turn allows us to assign reviewers to papers with more evidence: paper matching scores based on reviewer profiles, reviewers’ bids on papers, and AC recommendations. We hope that this new assignment process is an improvement through which we ensure that all papers are reviewed and meta-reviewed with the expertise and care that they deserve.

ICLR draws submissions from all over the world, and the unusual abstract (and full paper) submission time was in honor of the originally planned location of Vienna. This year, many authors, some more than others, were under additional stress, with the pandemic affecting many parts of the world and institutions providing different levels of support. We realize our colleagues in some Asian countries had to work with a deadline on one of their major holidays, and our colleagues in the Americas had to submit early in the morning. To all authors who faced challenges of one kind or another, we salute and appreciate your endeavors to make progress in representation learning research. And we will work toward being more supportive and inclusive of the diverse needs of the community in future ICLR conferences.

Now the important work of the reviewers and area chairs begins. This year we anticipated more submissions than last year and recruited over 400 area chairs and over 4,000 reviewers. These numbers are significantly higher than previous years, leading to a much lower reviewing load, and we are very grateful that so many have said yes to our invitation to serve. The process of their work can be seen in our diagram of the global review process, which is also available in the Reviewer Guide. New this year is the Code of Ethics, and the reviewers and area chairs are asked to flag any submissions that may violate the Code.

As you can see in the review process diagram, authors’ work does not end with submission. The upside is that you can make your paper stronger with additional work during the discussions. Reviewers will ask questions about your paper, requesting more explanation and sometimes even additional results. If you can anticipate what those comments and questions will be, you can prepare your answers and even revise your paper in advance. There may also be comments and questions from the general public (often other researchers) who are interested in your research. You can see the forums from ICLR 2020 to understand how the public discussions happen. You do not need to respond to every single question from them, but you can use the questions to clarify and highlight the important contributions of your paper. We try to make sure the process of peer-review and discussion is an overall positive experience, through which everyone learns and contributes to the advancement of the field. If you find any activity that violates our Code of Conduct including but not limited to trolling and inappropriate language, please let us know.

ICLR will feature excellent research papers contributed by the authors and curated by the reviewers and area chairs during the main session of the virtual conference from May 4th to May 7th, 2021. Following the main sessions is the Workshop Program on the 8th, for which we are soliciting your proposals by Oct 30, 2020. The workshops, featuring interesting and new research directions, draw active participation and lead to sub-communities for sharing and shaping on-going and future research. We look forward to seeing your proposals for workshops that will expand the horizons of ICLR and related research communities.

Authors, thank you for your submissions. Reviewers and Area Chairs, thank you for your service. Everyone else, thank you for your interest in ICLR 2021.

By ICLR 2021 Senior Program Chair Katja Hofmann(Microsoft Research Cambridge) and Program Co-Chairs Naila Murray (Facebook), Alice Oh (KAIST), Ivan Titov (U. Edinburgh)

Organising the 8th international conference on learning representations (ICLR 2020) was highly challenging, but ultimately, highly rewarding for our organising committees. Our initial work for hosting the conference in Ethiopia was far along, but then COVID-19 pandemic gave us the opportunity to create a new way of hosting our conferences. We look back now on the virtual conference week satisfied that we were able to show that a virtual conference is a viable way for us to share our science and build an international machine learning community, and that many meaningful avenues for connection are already possible.

We thank everyone for their support, patience and kindness in this time of change and experimentation. We thank the authors, organisers of workshops and socials, our volunteers, and everyone who registered. Together you made ICLR2020 a reality and a success.

We know that there is much more to do to create a better virtual conference experience for everyone. We hope this first virtual conference is one we can collectively build upon. Towards that aim, this post summarises some of our key reflections on the design and experience of the ICLR2020 Virtual Conference.

🎨 Design Principles

We used two key principles — to be Inclusive and Interactive — in our approach to the virtual conference. To these principles we returned whenever we found ourselves needing to make a difficult decision or tradeoff between competing approaches. What did these principles mean for us?

Inclusive

• Asynchronous-first. The conference had to be time-zone agnostic, so that everyone could participate within healthy working hours, and when they could. This meant that the conference had to be asynchronous-first, so that no one needed to be awake at odd hours, everyone could create their own flow of activities, catch up on anything missed, or skip ahead as needed.
• Extended live sessions. Live components had to be spread as evenly as possible across the world. For this reason we had 5 poster sessions each day at time zones across the world, and every paper was presented twice (in 2 separate sessions).
• Globally accessible. All parts of the conference had to be accessible everywhere. This meant we couldn’t use tools and services that only worked in certain countries, or that did not have easy ways to overcome any limitations.
• Low cost. The cost of the participation should not be a barrier. We created several opportunities for free registration, and lowered the fee to as low as we could, while still allowing income to pay for the services that were needed.

Interactive

• Chat everywhere. We needed to support as many forms of interaction as we could. Combined with the principle of inclusivity, this meant that every component of the conference had a chat channel associated with it to allow asynchronous, direct, and open conversation.
• Face-to-face. Poster sessions happened over video calls that took place at two times for every paper. We also hosted an extended live Q&A with every keynote speaker that allowed many more people than usual to directly ask questions of our speakers.
• Social. We asked for social meetups to be hosted that would allow the community to meet around shared interests and themes.
• Self-organising. We wanted there to be opportunities for direct and impromptu ways of interacting. We experimented with on the fly mentoring sessions, ad-hoc meetings, and having repeated socials.

🎒Lessons and Recommendations

We brought the entire virtual conference together within a 6 week period, so many of the issues we identified could be addressed with more deliberation and time. Overall, we believe we achieved all our goals with this approach to a virtual machine learning conference. We share some of our reflections and sites for improvement here.

Organisation and Communication

• Early communication. The virtual format is new, and requires a lot of scheduling to be done upfront. Being able to communicate early and in small chunks would have helped us smooth out confusion, build up an understanding of the virtual format, and help everyone be more prepared for this new format.
• Fees and scholarships. The virtual format opens up new avenues for inclusion, meaning that the programmes we had to support participation needed to grow proportionately. They needed to be clear, easy to find, and allow time for applications. We did support many free registrations through our volunteers, with additional registration support through our community groups. Having a clear place for this, and then multiple channels for outreach will help these programmes better reach those who need them.
• Volunteers. We had 500 volunteers who were a highlight of the conference for us. Our volunteers were energetic, spread across the world, and committed to improving the virtual experience for everyone. Our volunteers received free registrations in exchange for supporting the conference in several roles. Volunteers had early access to the conference site and helped us stress-test all aspects of the system. They were an on-call help team throughout the conference to help anyone who had technical questions or difficulties. A volunteer programme is one we highly recommend.
• Attendance and Engagement. We had a higher number of attendees than we expected to get. The physical conference in 2019 had 2700 attendees, and the virtual in 2020 had 5600 attendees. It is likely that, without the typical funding and visa restrictions for physical travel, more individuals were able to participate. This number remained highly manageable, because interaction was asynchronous across time zones. At any given time, maybe 200–300 people were engaged in an event. We also suspect engagement was skewed, with some participating in many events and others rarely present. We suspect some could not engage fully, because they did not take time off of work or had difficulties engaging from home rather than a dedicated physical space. A culture shift in engagement for virtual events is needed for better engagement.

Conference Components

• Pre-recorded videos. All the talks were pre-recorded and made available at the start of the conference. The ability to watch the invited talks and discuss questions (and upvote questions) before the Q&A made for a highly engaging and informative Q&A session. The availability of a short 5 minute video for each paper was also very useful for getting the main message of the work. The quality of all the videos was surprisingly good, and the system for recording was simple to use.
• Poster Assignments. Assignments worked overall and we struck a good balance between the needs of authors and the availability of events across the world. More time would have prevented missing and incorrect time requests from authors, and allowed us to iterate on the scheduling with the authors.
• Poster Interactivity. Posters sessions done over video need to be approached differently than what we are used to at a physical conference. There is a missing energy that is usually shared across presenters and attendees. Presenters are disconnected from their audiences and didn’t know how many people were engaging with their posters/videos. There is a reluctance from participants to join calls and be the only person, and a sense of pressure to have either read the paper or have insightful or ‘intelligent’ questions. This is where most iteration will be needed in future, by finding a way for authors to see the watch counts of their videos, the number of people on a video-call already, and in other creative virtual environments. But this also requires a change in the conception of how poster sessions work from everyone, and is something that will evolve as we all get used to these new forms of engagement.
• Speaker Q&A. The speaker session worked out as hoped: they allowed us to have more meaningful engagement with our speakers, reduced the barrier to asking questions by having multiple routes for questions to be asked, and allowed many more people than usual to ask questions. Even better, many of our speakers joined their chat session after their session and continued to answer questions over chat (for over an hour more in one case). Times for the live sessions were chosen for the convenience of the speaker, but the videos of these sessions were available immediately afterwards, so anyone who missed it live could watch it whenever they could.
• Workshops. Workshops were more difficult to design. They required a significantly larger amount of coordination between technical teams and chairs. We also required much more communication with individual workshop organisers who needed to provide details of their workshop timings, speakers, help develop an understanding of technical toolset and flow, website needs, etc. In future, this requires a slightly larger team of chairs, and other technical tools to allow components of the main conference website to be customised by workshop organisers for their own use.

Interactivity and Socials

• Geographic-spread of socials. We had a call for virtual meetups to be hosted and that would be supported by the conference, ultimately hosting 29 socials throughout the week. The set of socials we had were not as spread across global locations as we would have liked, and more targeted outreach would have allowed us to support social events spread across the world. The coverage of topics was also incomplete; again, with more time, we would likely have made calls for particular topics for socials, to facilitate interaction between attendees.
• More interactivity. We found that groups self-organised over chat for meetups on topics. A community-driven mentorship scheme also emerged where more experienced researchers would answer questions over video chat with younger researchers about research questions and career paths. Future improvements could be to pre-arrange video meet-and-greet sessions (either random or matched by interest, like the neuromatch approach) and to make such mentorship part of the conference programme, and spread across the conference week. We could also create standing virtual coffee sessions in future, and explore other ways for interactivity. Another component we experimented with was ICLRTown, which is a virtual environment that allowed participants to interact with each other in a virtual environment, which had a small user base, but received a positive reaction.

Technical Systems and Support

• Chat, Video, Pre-recorded video. We chose tools that worked everywhere that allowed us to control data and use, work everywhere across the world, were open-source projects if possible, and simple to integrate into our systems. We had no problems in using SlidesLive, Rocket.Chat, and Zoom for these needs and would use them all again.
• Single sign-on. Reducing the technical system to a single sign-on would have made the experience simpler and more convenient for everyone. In addition, it would have allowed us to implement more features for safety and privacy.
• Web development. We identified early on that a portal bringing together all the different tools we needed was not available. We explored some of the available platforms, but decided that we could best serve the culture and needs of our conference by creating this platform ourselves. This meant that a burden of the technical development load fell to members of our team. Having web developers that can be part of the central team and that take on this load will be needed in future.
• On-call support. Our volunteers were an on-call support throughout the day and across the world to support participants queries using the #helpdesk channel, which worked extremely well. There were 2 incidents of ‘zoom bombing’ during the conference because authors shared their links publically. Addressing this requires even more on-call support, to be able to create new video links and update this across the site. Other ways of preventing this, through shared responsibility across the community and other technical tools will need to be developed in future.
• Machine Learning in the design of the conference. Where it made sense, we wanted to use machine learning in the design of the conference programme and experience itself. We used a latent variable model for review score calibration, a vision model to extract thumbnails from each paper to be used on the web, natural language tools to visualise related papers, and recommender systems to create sets of balanced recommendations for papers and participants. The virtual conference creates a new environment within which we can try out and explore many of the machine learning approaches that the conference itself showcases.

🏁 Getting Ready for ICLR 2021

• Congratulations 🎉 Our best wishes and congratulations to the ICLR 2021 programme chairs: Katja Hofmann (senior PC), Alice Oh, Ivan Titov, and Naila Murray.
• Be an ICLR2021 Reviewer: If you have not reviewed for ICLR before and want to be considered as a reviewer or an area chair for the next round, please complete this short form: https://surveymonkey.co.uk/r/QFBT67V
• Next Deadline: Next deadline will be the beginning of October 2020, with the conference in the first week of May 2021. To be or not to be virtual, to be decided.

Signing off, Your ICLR2020 Organising Committees

Sasha Rush and Shakir Mohamed (General Chair and Senior Programme Chair); Dawn Song, Kyunghyun Cho, Martha White (Programme Chairs); Asja Fischer and Gabriel Synnaeve (Workshop Chairs); Adam White (Social Chair), Hendrik Strobelt (Virtual Chair); Anima Anandkumar and Kevin Swersky (Diversity Chairs); Timnit Gebru and Esube Bekele (Logistics Chairs); Andrea Brown and Lee Campbell (ICLR Secretariat), and the ICLR Board.

We’ll now be taking a much needed break from conferencing. We thank you for your support during these uncertain times. We wrote this, collecting our final notes for the next committees, and listening to Started from the Bottom by Drake 🎧.

To make the conference fit the culture and needs of the ICLR community, we built our own virtual portal that gives you all the elements of the conference that you expect. Access is restricted to registered participants. You can still register at iclr.cc if you want to join us.

Registering allows you participate in all the live components of the conference:

• Meet all the authors of some of your favourite papers and speak to them live and over chat.
• Speak to our keynote speakers like Ruha Benjamin, Leslie Kaelbling and Michael Jordan.
• Hang out at the social for Women in Machine Learning, wine and cheese for lapsed-physicists, or with Rich Sutton to talk about bitter lessons.
• Speak to our sponsors about full-time and internship opportunities and their work.
• Most importantly, connect with other participants and meet new people from across our field and around the world.

We know the virtual format will take some getting used to, but you will create the virtual experience along with 3500 other people from 63 countries across the world. This is a quick visual tour of what you’ll find on the conference portal.

🗺️ A Visual Tour of the the Virtual Conference

You will find everything you need to navigate the virtual conference from the homepage and the menu-bar. All content can only be accessed by registered participants and you will need your registration to access the site. Here’s some of what you‘ll find.

The conference comprises the following elements:

• Keynote talks. Invited talks are pre-recorded and will be released each day. Each talk has a live Q&A session so we can interact in real-time with the speakers.
• Papers: 5min talks take the place of posters for all papers and 15mins for longer talks. All are pre-recorded and are also released each day.
• Poster sessions. You interact with posters by joining a video meeting with the authors. These occur at timezones across the world and several times per day.
• Socials. You can also meet new people and other participants by using the virtual rooms and socials that are organised throughout the week.
• Schedule. You can find the times for all the live sessions using the Schedule, where you can choose your timezone and find live sessions that work for you.
• Sponsors. Meet our sponsors by visiting the virtual sponsor hall.
• Help. If you have any questions about planning your time or finding information, look under Help for answers to frequently asked questions, and technical support where you can chat directly with some of our on call volunteers.

🧠 Other Considerations

We have taken many steps to support a safe and respectful online conference:

• Participants must abide by our code of conduct, which has been updated for the virtual conference. We have several reporting mechanisms for unacceptable behaviour.
• We have restricted settings to prevent phenomena like ‘zoom bombing’.
• We have done our best to test the site to work in all countries.
• The structure of the conference is such that much of it can work in any timezone because of its asynchronous nature.
• Ultimately, we hope that the conference still provides new opportunities to meet with new people and have fun.
• About two weeks after the conference all the videos will be made available for everyone to watch.

If you want a quick overview, you can watch a short welcome video from the programme committee here. We welcome feedback about the virtual conference and format that will help us improve it in future, so send thoughts to the #virtual-feedback channel in the portal.

Creating and getting the virtual conference to this stage is due to the effort of a large number of people behind the scenes — a huge thanks to them all 💙

🟢 Conference Status and Updates

• Registration: Still open, go to iclr.cc.
• Virtual Socials/Booths: You can still help shape one of the social-elements of Virtual ICLR. It takes 5 mins to submit a short request
• Dates: Workshops are on Sunday 26 April. Main conference Monday-Thursday, 27–30 April.

ICLR2020 Organising Committees

Sasha Rush and Shakir Mohamed (General Chair and Senior Programme Chair); Dawn Song, Kyunghyun Cho, Martha White (Programme Chairs); Asja Fischer and Gabriel Synnaeve (Workshop Chairs); Anima Anandkumar and Kevin Swersky (Diversity Chairs); Andrea Brown and Lee Campbell (ICLR Secretariat), and the ICLR Board.

We hope you are safe and in good spirits; and that the virtual conference will be a welcome experience. We wrote this post skittishly testing zoom links and Rocket invites and listening to the Classica Orchestra Afrobeat rendition of Fela Kuti’s No Agreement. 🎧 .

👷 The 2020 Workshops

We’re confident that we will be able to provide a useful, meaningful, and intellectually stimulating workshop experience, even though we will not be physically co-located. We are looking forward to our virtual workshops which will be a mix of pre-recorded talks, asynchronous engagement, and live engagement through Q&A and in-person video calls.

Visit the websites of all 15 of our virtual workshops:

1. AfricaNLP — Unlocking Local Languages
   Showcase work on NLP for African languages, which are typically low resource languages, and bring it to a global audience.
2. AI for Affordable Healthcare
   Highlight recent advances in AI for enabling, democratising, and upholding high standards of healthcare worldwide.
3. AI for Earth Sciences
   Bring cutting edge geoscientific and planetary challenges to the fore for the machine learning and deep learning communities.
4. AI for Overcoming Global Disparities in Cancer Care
   Identify the potential of AI to overcome global disparities in access, diagnosis, and treatment in cancer care.
5. Beyond “tabula rasa” in reinforcement learning: agents that remember, adapt, and generalize
   Bring together researchers from different backgrounds working on how to extend current RL algorithms to operate in changing environments and tasks.
6. Bridging AI and Cognitive Science (BAICS)
   Inspire connections between AI and cognitive science across a broad set of topics.
7. Computer Vision for Agriculture (CV4A)
   Expose the progress and unsolved problems of computational agriculture to the AI research community.
8. Causal learning for decision making
   Investigate how much progress is possible by framing the learning problem beyond learning correlations, that is, by uncovering and leveraging causal relations.
9. Integration of Deep Neural Models and Differential Equations
   Where theoretical and experimental researchers can come together to join forces towards the goal of integration of deep neural networks and differential equations.
10. Fundamental Science in the era of AI
    How can AI help deal rigorously with the data deluge of “big science”? How can AI help amplify the social impact of fundamental science?
11. ML-IRL: Machine Learning in Real Life
    Examine how we develop, evaluate, and deploy ML in real life applications, and discover problematic implications.
12. Neural Architecture Search
    Build a strong, open, inclusive, and welcoming community of colleagues working on neural architecture search.
13. Practical ML for Developing Countries: learning under limited/low resource scenarios
    Bring together researchers, experts, policy makers, and related stakeholders under the umbrella of practical ML for developing countries.
14. Towards Trustworthy ML: Rethinking Security and Privacy for ML
    Bring together experts from a variety of communities (ML, computer security, data privacy, fairness, & ethics) to work on promising ideas and research directions.
15. Tackling Climate Change with Machine Learning
    Show that ML can be an invaluable tool both in reducing greenhouse gas emissions and in helping society adapt to the effects of climate change.

A word about how we picked the workshops. We had reviewers score workshops according to a set of criteria, that would allows us to select workshops that would be most topical, exciting, diverse, and inclusive, and which left room for discussion. When multiple candidate workshops were almost exactly on the same topic, we decided to pick only one not to have topic overlap. We were impressed by the quality of the workshop submissions, and we hope and think that you will find some intellectually stimulating and welcoming workshops in this selection.

🏕 A Virtual Format for Workshops

The virtual format of the workshop means that asynchronous engagement becomes a basis of the workshops so that it works across time zones and access needs. The general structure of the workshops will follow that of the main conference, while allowing workshops to continue to explore new ways of building research communities and sharing knowledge.

• Pre-recorded videos will be used for all keynote talks, contributed talks, and posters. Posters are also presented as short videos.
• A Q&A tool will be used to gather questions for the keynotes throughout the day
• A moderator will be available to read out submitted questions, as well as to coordinate and take any live questions.
• There will be dedicated blocks of time for live engagement (in a chosen timezone), where live content, such as panel discussions, live poster spotlights, and Q&A sessions are hosted.
• All content will be accessed through the individual webpages of the workshops.

Accessibility across time zones is one of the major challenges of the virtual format. By mixing the asynchronous and synchronous formats, workshops will still be able to form a sense of community and engagement around their shared topic. Other challenges facing workshop organisers include managing web presence, sequencing live content, and supporting their sponsorship needs and engagement, but plans are in place to address all of these and to host as successful workshops as are possible.

🟢 Conference Status and Updates

• Virtual Socials/Booths: Takes 5 mins. Submit a proposal to help shape one of the social-elements of Virtual ICLR. Submit by 13 April.
• Registration: Already open, go to iclr.cc for details.
• Workshops: Organiser’s discuss on your Slack group; general workshop instructions.
• Questions: Check the FAQ, which we update regularly.

If you need more, see our last update on the Voices of ICLR2020: Announcing our Keynote Speakers, or on the Format for the ICLR2020 Virtual Conference.

Your Workshop Chairs and ICLR Organising Committees

Asja Fischer and Gabriel Synnaeve (Workshop Chairs); Sasha Rush and Shakir Mohamed (General Chair and Senior Programme Chair); Dawn Song, Kyunghyun Cho, Martha White (Programme Chairs); Anima Anandkumar and Kevin Swersky (Diversity Chairs); Andrea Brown and Lee Campbell (ICLR Organisation team), and the ICLR Board.

We wrote this in solidarity with all the people across the world working every day to ensure we can safely isolate and distance ourselves indoors. And while we have been re-re-reading workshop abstracts, we have been keeping our concentration by listening to some Touareg desert blues — on repeat 🎧 .

The collection of accepted papers formed #OurHatata: Our inquiry into the principles and practice of machine learning and artificial intelligence. All accepted papers will be presented as videos with live video poster sessions. These presenters are joined by a set of incredible keynote speakers, who will amplify our conversations during the conference. This blog introduces you to our speakers and what they will be talking about.

📚 Keynote Topics and Speakers

We are proud this year to have 9 keynote speakers who will share their work and insights with us. The themes and speakers (in order of expected live Q&A sessions) for 2020 are:

• AI + Africa = Global Innovation. Dr Aisha Walcott-Bryant, IBM Research Africa, Nairobi
• Doing for Our Robots What Nature Did For Us. Prof. Leslie Kaelbling, MIT
• 2020 Vision: Reimagining the Default Settings of Technology & Society. Prof. Ruha Benjamin, Princeton University
• Invertible Models and Normalizing Flows. Dr Laurent Dinh, Google AI
• Machine Learning: Changing the Future of Healthcare. Prof. Mihaela van der Schaar, University of Cambridge, The Alan Turing Institute, University of California Los Angeles
• AI Systems That Can See And Talk. Prof. Devi Parikh, Georgia Tech and Facebook AI Research
• Reflections from the Turing Award Winners. Prof. Yann LeCun, Facebook AI Research and New York University, and Prof. Yoshua Bengio, Montreal Institute for Learning Algorithms
• The Decision-Making Side of Machine Learning: Dynamical, Statistical and Economic Perspectives. Prof. Michael I Jordan, University of California, Berkeley.

🎙 Live Q&A Sessions

Each speaker’s talk will be pre-recorded and released for you to watch. As you are watching you can leave questions for the speakers on our Q&A tool. Live sessions with each speaker are scheduled to take place each day, and will let us all come together around the insightful ideas raised by the speakers, and let us dive deeper into their talks. We will discuss the questions submitted to the Q&A tool, and also take live questions.

🟢 Registration Open and Conference Updates

• Registration: Already open, go to iclr.cc for details and register as soon as you can.
• Questions: Check the FAQ, which we update regularly.
• Volunteers: We request volunteers to help us with testing and quality control. Apply here.
• Authors of accepted papers: Please complete your videos and sign up for poster slots before 8 April. Reminder of instructions online.
• Next steps: We’ll continue to share updates on our progress, with more details on workshops, code of conduct updates, and how to prepare for the conference.

See our previous updates and get a sneak peek at what is happening behind the scenes like #OurHatata: The Reviewing Process and Research Shaping ICLR in 2020, or on making the big switch to ICLR as a Fully Virtual Conference, and more details on the Format for the ICLR2020 Virtual Conference.

Your ICLR2020 Organising Committees

Sasha Rush and Shakir Mohamed (General Chair and Senior Programme Chair); Dawn Song, Kyunghyun Cho, Martha White (Programme Chairs); Asja Fischer and Gabriel Synnaeve (Workshop Chairs); Anima Anandkumar and Kevin Swersky (Diversity Chairs); Andrea Brown and Lee Campbell (ICLR Organisation team), and the ICLR Board.

We hope you are all safe during this challenging period of social distancing and lockdown. We wrote this post safely indoors, in constant discussion about virtual conference organising, and listening to DJ D-Nice’s Homeschool 🎧 .