It’s been a curious year for AI — with important breakthroughs in NLP and drug discovery tempered by AI’s to-date modest contributions in the fight against Covid-19.

Compiled with my friend Ian Hogarth during the pandemic, this year’s State of AI Report (https://www.stateof.ai/) is a remedy to fears of a new AI winter, and we hope a much needed reminder of the extent of innovation and progress carrying on despite the present circumstances.

However, our report also demonstrates just how much of this innovation is enabled by massive computing infrastructure that is increasingly in the hands of big tech companies.

We need to think carefully about what this means for the future of AI innovation, but also acknowledge that we are seeing more and more AI-first startups implementing the core ideas that emerge from this research.

Elsewhere in the report, look out for the wealth of biotech progress. I’m particularly excited about the impact that AI is having in the life sciences as biology and medicine become large data domains ideal for AI applications. So much has changed since I finished my PhD in cancer research in 2013…we’re now on the brink of being able to decode a lot more about our health and revolutionise how we treat disease.

We write this report to compile the most interesting things we’ve seen, with the aim of provoking an informed conversation about the state of AI, and unpacking what developments in the field mean for our future. We’d love your view on the report and your take on our predictions. Please do share any observations you might have!

New to the 2020 edition are invited content contributions from two dozen well-known and up-and-coming companies and research groups. We’re incredibly grateful for their contributions that dive deeper into key themes of 2020: NLP, biology, and autonomous systems. Thank you to our reviewers for volunteering their time and valuable insights that keep us honest.

What are the key takeaways, you ask? There are many, but here are four key findings:

1. AI R&D costs are soaring: Only a few labs that are led or funded by the likes of Google, Microsoft, and Facebook can afford the high cost of computing power needed to pursue the most exciting AI research. It most likely cost Microsoft-funded OpenAI $10 million to train GPT-3. Practitioners fear innovation could stagnate.
2. 85% of AI research papers do not publish code: Open sourcing AI research is important for accountability, reproducibility and driving progress in AI but, in reality, the vast majority of AI research papers are behind corporate lock and key. The industry has not evolved much since 2016 when 90% of research did not publish code along with their papers.
3. Biology experiences its “AI moment”: This year signaled a new era for AI-first drug discovery startups with mega-rounds, the first-ever AI-created drug in clinical trials and the first medical imaging product to be acknowledged by the US government (Medicare/Medicaid). In spite of pharma doubts, AI-powered medical companies started rapidly industrialising and scaling
4. Explosion of AI-powered military: As the military applications grow, so do calls for new AI safety measures. With another wave of countries declaring national AI strategies, more governments doubled down on the military adoption of AI technology.

With thanks to: Babylon Berkshire Grey CloudNC ComplyAdvantage Disperse Faculty Graphcore Hugging Face InVivo AI LabGenius James Field Lyft Level 5 Niantic Onfido ONI OpenMined PolyAI PostEra Recursion Secondmind Signal Tessian tinyclues Tractable ZOE

And our reviewers:

Jack Clark, Jeff Ding, Chip Huyen, Rebecca KaganKagan, Andrej Karpathy, Moritz Mueller-Freitag, Torsten Reil, Charlotte Stix, and Nu (Claire) Wang.

In this post, we share key takeaways from our Air Street Summit 2019 on AI-first startup investing, product building, selling companies, and investing in VC funds. If you’re a GP, angel, or LP and you’d like to join us next year, drop us a line!

The AI-first startup investing playbook

AI is a hot topic amongst investors across the capital spectrum. By some counts, over 3.6k “AI startups” in 70+ countries have raised $66B since 2013. The pace is picking up too: $7.4B was raised in Q2 2019 alone. In an environment like this one, it’s important for investors to carefully select signal from noise using an experience playbook (e.g. metrics, frameworks, benchmarks). Just like today’s well-established technology themes such as SaaS, we believe that the AI-first theme will follow a similar trajectory where investors experiment and acquire experience over the years. Through (more) success and (less) failure, this ultimately culminates in a best practice “AI investing playbook”.

Air Street Capital is a venture firm focused on AI-first technology and life science companies. By virtue of our industry and academic roots, Operating Partnership, and existing community efforts (e.g. RAAIS, London.AI), we consider ourselves as natives to the AI community. We believe that the best investors in AI-first technology companies will be those who are “close to the metal” when it comes to understanding technology-problem fit and the particularities around how to build, market, sell, and scale AI-first products in various markets.

As a result of our position in the market, we decided to run an experiment by hosting a half-day knowledge-sharing forum for VC General Partners (GPs), Limited Partners (LPs), and the angels who are investing in AI-first technology and life science companies. Our goal? To share knowledge around the emerging AI-first startup investing playbook and strengthen a collegial investing community around this theme.

To help us achieve this goal, we brought together friends across the industry to discuss:

• Investing: Best practices for AI-first startup investing, a conversation between Zavain Dar (Lux Capital) and Nathan Benaich (Air Street Capital).
• Building: Learnings from the front lines of creating and selling AI-first products, a talk by Julien Cornebise (Air Street Capital, ex-DeepMind).
• Exiting: Current dynamics in the M&A market, a talk by Peter Spofforth (Qatalyst Partners).
• LPs: Market insights into VC fund investing and liquidity, a talk by Nina Kraus (Hamilton Lane).

Air Street Summit attendees

We hosted 45 of the industry’s best GPs, LPs, and angels from the following firms:

Key takeaways

We covered a lot of ground, from investing through the building and selling of products and companies to investing in VC funds. Here is a snapshot of the key takeaways:

GP investing

• ML expertise on the (founding) team is important for creating the initial edge of the business. This confers an advantage over competitors while the technology potentially commoditizes.
• Deepening the technical talent bench is equally important to keep the “innovation wheel” spinning so this advantage can compound over time as the business grows its brand, its understanding of the problem, and high-quality data.
• We’re looking to invest in the dogged commercial entrepreneur and technologist/scientist from whom technical edge arises. This can be in one or more people on the founding team. If you have one or the other, tread at your own risk.
• When investing ahead of the curve (e.g. further on the deep tech spectrum), it is normal that companies might not have a defined customer persona or buyer budget yet. However, companies should map to “traditional” persona/budgets as they enter Series A/B land and need to interface with less technical-driven investors.
• Bio + AI investing is hot and there is a lot of value to be created. More on this here ;-)

M&A deals

• Although the return on investment proof points for AI-first software products might take longer than traditional SaaS products, when they reach scale, their ACVs can be larger than incumbents. 2017 seems like a breakpoint for vertical AI-first companies vs. tools/generic platform providers in terms of their momentum.
• We see many more $50M growth cheques written into AI-first companies in 2018/2019. This is a sign of company maturity because growth investors invest when they can see dollars in = more dollars out. AI-first companies are real businesses. Customers and buyers see beyond the AI technology and into the business ROI.
• Buyers need to understand what they’re buying.
• We see a maturation in the acquisition valuation metrics from $/engineer (close to peak hype) towards traditional business metrics (revenue multiples, profitability).
• There are fewer smaller acquisitions in AI now due to the prior cohort of venture-funded companies closing at high private valuations. Buyers are, however, still open to tag on acquisitions of talented teams.

LP investing

• Measured by pooled IRR by vintage year, the VC asset class has topped the charts from 2010 through 2015.
• Top quartile venture returns beat private equity pooled IRR from 2006 through to 2018.
• Strong returns have led to an increase in global VC fundraising. Almost $200B was raised globally in each of 2017 and 2018 (including SoftBank). Without SoftBank, the numbers become ~$50M and ~$100M, respectively.
• While North America, particularly Silicon Valley, has historically been the primary hub for venture capital managers and deal flow, globally diversified VCs have gained market share in recent years as regional start-up ecosystems continue to grow. Europe’s share of VC and growth fundraising (including SoftBank), grew from 5% in 2014 to 13% in 2018. In the same time period, North America’s share decreased from 49% to 39%.
• Distributions have fallen to historic lows over the last 10 years as the age of companies at IPO and the broader availability of capital for venture-backed businesses increases. On average, the annual distribution rate of VC and growth companies sits at 23.2%.

Thanks for reading! Drop us a line if you’d like to attend next year’s Air Street Summit

Huge thanks to Zavain Dar, Julien Cornebise, Nina Kraus, and Peter Spofforth for sharing their experience. Thanks to Cooley LLP for hosting us.

— —

For more, follow me on @nathanbenaich or sign up to my newsletter.

Learn more about Air Street Capital here.

Air Street Summit: Investing in AI — key takeaways was originally published in Stories from Air Street on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

As machine learning (ML) grows in popularity amongst large technology companies and startups, businesses of all kinds are taking notice and asking themselves how they can follow suit. There are many resources that help codify the discovery of real-world use cases where it makes technical and commercial sense to use an ML-based solution. For example, inspired by the popular startup business model canvas, the ML Canvas does a great job at structuring this discovery process.

However, less has been written about the common challenges faced by specific business personas as a function of their level of expertise in ML. From my vantage point as an investor in AI-first technology and life science companies with Air Street Capital, I will describe three business personas that I have observed as well as their common challenges, technology stacks, talent requirements and organisational structure. The goal here is to help you define your starting position such that you can make informed decisions as the steps in your ML journey.

Three business personas

Broadly speaking, I have observed three different business personas that can be distinguished by the extent to which ML and software is a core value creator today.

1. The ML-first company

Description: This business persona is creating a product uniquely enabled by ML. For example, this could be an enterprise cybersecurity product that monitors, detects and protects against malicious network activity. The management/senior team are ML natives and have extensive experience with state-of-the-art tools (and often research code) and processes.

Technology choices: Invest significantly into proprietary tooling and adopts open source as much as possible. They will depend on cloud vendors for commodity infrastructure, but will shy away from purchasing third party services that play to the lowest common denominator ML user for problems that they can solve themselves. Examples: Ravelin.

Common challenges: Creating a resilient, reliable, scalable, fast, secure, and cost-effective technology stack. Read more on this topic in an excellent post on data science best practice here.

Talent requirements: Examples include: hiring ML research engineers, DevOps for ML, data engineering. I.e. roles focused on either a) advancing state of the art in production, b) reliable, large scale ML production system engineering.

Setup of ML organisation: A core ML platform team that is responsible for building a platform to make model building, training, and deployment simple and as automated as possible. Then, several product-focused ML engineers or data scientists who are embedded in product teams. These individuals use the core ML platform to train problem-specific models, evaluate them and serve them in production.

2. The software-native company

Description: A software company that is strategically evolving its offerings to be ML-first. This persona might be a workflow SaaS company or enterprise software business whose product essentially consists of a client-side application, server-side application infrastructure, and databases. There is no real use of ML baked into the product when it was built.

Technology choices: This type of company will usually have the necessary business intelligence infrastructure in place (i.e. a data-warehouse) and is looking to deploy internal or hired data scientists in order to leverage their data assets. However, since the existing software architecture was typically built for a different purpose, it might be challenging to deploy models in production. They will often look to simple to use open source tools, services offered by cloud vendors, or other SaaS companies that provide ML tooling. A focus is placed on customizability, cost, and support. For example, see OpenDoor.

Common challenges: The challenges that the data team will face are around convincing business stakeholders of the value of ML. This persona tends to struggle knowing where and how to begin. If no one in the business has experience with ML, the challenges include defining a low-hanging and addressable problem, deciding to build or buy technology, and determining what success looks like.

Talent requirements: While these businesses have software engineering expertise, they often do not have particular skills in ML. They might have individuals who can prototype a model using a data science notebook or who can integrate cloud available prediction APIs into a functioning demo. Should they hire new people and, if so, should these be data engineers, data scientists, or ML engineers? In many cases, one could argue that these businesses should first hire a data-focussed business person to project manage rather than going directly to hire technical talent. For more, read this piece on setting up analytics competencies from scratch at different stages of company growth.

Setup of ML organisation: For this persona, it often makes most sense to start by pairing a group of software engineers and data scientists with a particular product organisation. The goal here is to scope, build and test new ML prototype. If this is successful, weigh up the merits of building out an organisation as in the first persona.

3. The incumbent company

Description: This business offers software-based products or uses software to power their business but it really treats software as “old-school IT”. This persona does not see software as a core competence or a vector of differentiation against their competitors. Software is not really a value creator, but more of a means to an end.

Technology choices: This persona will most likely be using many legacy IT systems, will generally be early in their migration to the cloud and does not tend to procure using SaaS or similar low-friction approaches. They tend not to use much open source and instead by software from large IT vendors or have custom solutions written for them.

Common challenges: This persona is likely to want an end-to-end platform of some kind that offers lots of hand holding, starting with business intelligence, analytics and data warehousing. It is probably too early to be worrying about ML :-)

Talent requirements: Little provided internally. Instead, they will depend on external development agencies or large technology consultancies.

Setup of ML organisation: Pretty much nonexistent.

Bonus section: A list of ML tools

The ecosystem of ML tooling is evolving quickly. To help you bootstrap your search,here is a selected list of solution providers and open source offerings tackling specific parts of the ML value chain:

• End-to-end platforms: DataRobot, Polyaxon, MLflow (open source), Peltarion, Petuum, Domino Data Lab, BigHead (open source), Google’s AI Platform, Spell, MSFT Azure, AWS Sagemaker, Flyodhub.
• Model deployment platforms: Kubeflow, Seldon, Algorithmia, Datmo, Atalaya.
• Data science collaboration: Dataiku.
• Notebooks: Deepnote, Neptune, Jupyter, Google’s Colab.
• Data pipeline building: Tecton, Pachyderm, Airflow, Google Dataflow, Amazon step functions.
• Workflow Orchestration: Airflow (open source), Luigi.
• Distributed programming frameworks (batch): Hadoop, Spark
• Distributed programming frameworks (real time): Spark Streaming, Flink, Kafka streams
• Version controlling for models and data: DVC, MissingLink, Verta
• Experiment tracking and analysis: Weights and Biases, Comet, MissingLink
• Data labelling: Scale, Mechanical Turk, Heartex, Deepen, LabelBox, Superannotate, Lionbridge.
• ML task specific end-to-end platforms: platform.ai (computer vision).
• Trained model marketplaces: Runway.
• Distributed training acceleration: EngineML, Horovod (open source).
• Model auditing and explainability: Untangle, Fiddler.

For even more depth, check out this compendium on GitHub.

Special thanks to Hendrik Brackmann and Moritz Müller-Freitag for critical review and input on this piece.

3 business personas in ML, their challenges and opportunities was originally published in Stories from Air Street on Medium, where people are continuing the conversation by highlighting and responding to this story.

Just like our first report, we set out to capture a snapshot of the exponential progress in AI with a focus on developments in the past 12 months. You can consider this report as a compilation of the most interesting things we’ve seen that seeks to trigger an informed conversation about the state of AI and its implication for the future.

Please do drop us a line by hitting reply or tweeting us on @nathanbenaich and @soundboy with comments, feedback and suggestions.

If you enjoy the report, please do share www.stateof.ai with your friends and collaborators.

Here are 2 snapshots per section from our Report to whet your appetite:

🔬 Research

🎓 Talent

👉 Read the State of AI Report 2019 here 👈

🏭 Industry

📖 Politics

👉 Read the State of AI Report 2019 here 👈

🇨🇳 China

🔮 Predictions for 2019 and a review 2018's

You’ll have to read the Report for our 2019 predictions :-)

👉 Read the State of AI Report 2019 here 👈

🔖 Your State of AI Report 2019 was originally published in Stories from Air Street on Medium, where people are continuing the conversation by highlighting and responding to this story.

The Research and Applied AI Summit (RAAIS) is a community for entrepreneurs and researchers who accelerate the science and applications of AI technology. In the lead up to our 5th annual event on June 28th 2019 in London, we’re running a series of speaker profiles to shed more light on what you can expect to learn on the day!

A driving force behind the blossoming of machine intelligence is next-generation hardware that accelerates training and inference at scale. In 2017, we hosted Graphcore Co-Founder and CTO Simon Knowles. In his talk, Simon motivated the need for novel hardware and outlined the design principles behind the Company’s Intelligence Processing Unit (IPU). Fast forward two years, Graphcore has established itself as a World-leader in machine intelligence hardware and has raised a further $280M from the likes of Sequoia, Atomico, Sofina, Microsoft and BMW.

It’s in this context that we’re extremely excited to welcome Carlo Luschi, Director of Research at Graphcore, to RAAIS 2019! Carlo is responsible for the study and development of algorithms for machine intelligence. Prior to Graphcore, Carlo was a Member of Technical Staff at Bell Labs Research, Lucent Technologies, and more recently Director of Algorithms and Standards at Icera Inc., which was acquired by NVIDIA in 2011. Carlo subsequently served as Director of Algorithms and Standards at NVIDIA until joining Graphcore in 2016. Carlo is also a prolific inventor, having authored 55 patents granted or pending. He holds a PhD in Electrical Engineering from the University of Edinburgh.

In his talk at RAAIS 2019, Carlo will focus on the ground-breaking architecture of Graphcore’s IPU. The processor is based on a revolutionary approach for the efficient use of massively parallel computing resources for Machine Intelligence applications, under the current physical constraints. Carlo will address some of the fundamental work carried out at Graphcore Research to support the realization of the above goal. This work is based on the adoption of an efficient numerical representation and through algorithmic innovation, including optimization approaches based on evolutionary computing and new solutions for the training of large-scale distributed machines.

You can read about Carlo’s views on the directions of AI research on Graphcore’s blog here.

Welcome, Carlo!

RAAIS 2019 — Carlo Luschi, Graphcore was originally published in The RAAIS Journal on Medium, where people are continuing the conversation by highlighting and responding to this story.

The Research and Applied AI Summit (RAAIS) is a community for entrepreneurs and researchers who accelerate the science and applications of AI technology. In the lead up to our 5th annual event on June 28th 2019 in London, we’re running a series of speaker profiles to shed more light on what you can expect to learn on the day!

This year, we’re organising a deep-dive session on how AI methods are transforming the life sciences. We’re thrilled to welcome Jonathan Bloom from the Broad Institute of MIT and Harvard! Jon is a mathematician, engineer, and Institute Scientist at the Broad Institute. Launched in 2004 on the back of the Human Genome Project, the Broad Institute exists to accelerate the pace at which the world cures disease by combining industry-scale data generation with the openness and creativity of academic research.

At Broad, Jon co-founded the Hail team to build distributed systems now used throughout academia and industry to harness the flood of sequenced genomes to uncover the biology of disease. He also co-founded and directs the Models, Inference, and Algorithms Initiative (MIA) to foster education and collaboration at the interface of biology and computation.

In his talk, Jon will share the story behind the Broad, Hail, and MIA from the perspective of a theoretical mathematician who began to stare at other people’s shoes. For a primer on the journey, check out this week’s feature The Kendall Square Codebreakers in The Harvard Crimson. Jon will also argue that machine learning needs biomedicine as much as biomedicine needs machine learning, drawing on real-world examples from his own interdisciplinary collaborations since joining Broad in July 2015.

These include building software and algorithms to run the largest ever genome-wide association study in hours; autoencoding single-cell gene expression, leading to new algorithms for eigendecomposition that suggest how the brain might learn like an artificial neural network; delivering therapeutics past the blood-brain barrier; creating the most unbiased genome sequencing benchmark to date; computing secure multiparty linear regression across biobanks or companies at plaintext speed; and aligning the skull for orthodontic treatment. You can catch him lecturing on the fusion of ML and bio here, on scalable infrastructure for genomic data science here, and on the topology of PCA itself here.

Prior to joining Broad in 2015, Bloom did theoretical research in algebraic topology and geometry as a Moore Instructor and NSF Postdoctoral Fellow in Mathematics at the Massachusetts Institute of Technology. While there, he re-architected 18.05: Introduction to Probability and Statistics, now among the most popular offerings on MIT OpenCourseWare. Bloom received his PhD from Columbia University and B.A. from Harvard University in Mathematics.

You can find him on Twitter here. Welcome to #RAAIS2019, Jon!

RAAIS 2019 — Jonathan Bloom, Broad Institute was originally published in The RAAIS Journal on Medium, where people are continuing the conversation by highlighting and responding to this story.

🔬 Setting the scene

In 2013, the machine learning (ML) research community demonstrated the uncanny ability for deep neural networks trained with backpropagation on graphics processing units to solve complex computer vision tasks. The same year, I wrapped up my PhD in cancer research that investigated the genetic regulatory circuitry of cancer metastasis. Over the 6 years that followed, I’ve noticed more and more computer scientists (we call them bioinformaticians :) and software engineers move into the life sciences. This influx is both natural and extremely welcome. Why? The life sciences have become increasingly quantitative disciplines thanks to high-throughput omics assays such as sequencing and high-content screening assays such as multi-spectral, time-series microscopy. If we are to achieve a step-change in experimental productivity and discovery in life sciences, I think it’s uncontroversial to posit that we desperately need software-augmented workflows. This is the era of empirical computation (more on that here). But what life science problems should we tackle and what software approaches should we develop?

AI methods in life sciences

I’ve maintained a subscription to Nature Methods ever since my senior year in undergrad. Looking at the cover of each issue gives me a temperature check for what scientists think is new and exciting today. Over the last 12 months, there’s seldom been an issue that runs without a feature story about a new ML solution to a life science problem. These tasks tend to have previously been “solved” by hand or with a brittle, rules-based system from the 90s. Below you’ll find a snapshot of the 2018 Nature Methods covers. Can you see a trend?

The papers making the headlines include 3D super-resolution microscopy, DeepSequence for variant calling, 3D cell segmentation and tracking animal behaviour through video analysis. These tasks involve high-dimensional input data, an agreed taxonomy of labels, and large volumes of data that is becoming cheaper to produce at scale. As such, ML methods can likely help classify, cluster and generate novel data points for these tasks. If we sample Nature, Nature Methods, Nature Biotechnology and Nature Medicine papers over the last 5 years, we can see that in fact over 80% of papers mentioning “artificial intelligence” or “deep learning” were published in the last 2 years alone!

6 impactful applications of AI to the life sciences

Two years ago I wrote a piece describing 6 areas of machine learning to watch closely. In this post, I’ll describe 6 areas of life science research where AI methods are making an impact. I describe what they are, why they are important, and how they might be applied in the real world (i.e. outside of academia or industrial research groups). I’ll start from the small scale (molecules) and work up to a larger scale (cells). I look forward to hearing your comments and critique 👉@nathanbenaich.

1. Molecular property prediction

Drug discovery can be viewed as a multi-parameter optimisation problem that stretches over vast length scales. Successful drugs are those that exhibit desirable molecular, pharmacokinetic and target binding properties. These pharmacokinetic and pharmacology properties are expressed as absorption, distribution, metabolism, and excretion (ADME), as well as toxicity in humans and protein-ligand (i.e. drug to protein target) binding affinity. Traditionally, these features are examined empirically in vitro using chemical assays and in vivo using animal models. To do so, most academic labs will rely on lab scientists endlessly pipetting and transferring small amounts of liquids between plastic vials, tissue culture and various pieces of analytical equipment. Pharmaceutical companies will do much of the same, but they might have budget for purchasing robotic automation equipment for tissue culture and liquid handling.

There are also computational methods that help elucidate molecular properties in silico. However, most computational efforts have been based in some way on physics, i.e., calculating energies from quantum mechanics or doing thermodynamics-based simulations, or simple empiricism. In contrast to ML-based approaches that will learn to approximate the underlying mechanisms of a target process from raw data, these physics-based computational tools will only capture the underlying mechanisms that developers have explicitly encoded during the development process. Thus, from an ML point of view, physics-based computational tools are essentially feature-engineered and thus rather brittle and do not generalise well to out of sample distributions.

Over the last two years or so, the ML research community has produced a flurry of papers describing novel deep learning methods to predict the properties of small molecule drugs and proteins therapeutics. A driver of success or failure for these ML approaches rests in the computational representations that these models use to learn the intrinsic rules of chemistry that are encoded in raw experimental data. Broadly speaking, there are three representation types for chemical molecules that are used today: Extended Connectivity Fingerprint (ECFP, a fixed length bit representation), Simplified Molecular Input Line Entry System (SMILES, where molecules are essentially represented as a linear string of letters), and graph-based representations (where nodes are atoms and edges are the chemical bonds linking atoms). Each representation imposes different constraints on model performance and a machine’s ability to understand the data. Here, ECFP is extracted from SMILES, and SMILES is extracted from a 3D conformations of the molecule graph. Each transformation process destroys a certain amount of data as a trade for machine readability and computability. The following figure illustrates this concept. The ideal scenario is to use the most comprehensive molecular representation that a machine can fully utilise.

The appeal of these deep learning systems is that they can automatically learn the underlying rules that relate molecular composition and structure to chemical properties from raw experimental data. Indeed, the ML models mentioned here can predict >10 chemical properties accurately enough to be useful to chemists. Most importantly, these ML approaches can generate property predictions up to 300,000 times faster than it would take to simulate them using computationally-intensive numerical approximations to Schrödinger’s equation. This means that a chemist might no longer have to run a flurry of chemical assays to empirically determine a molecule’s chemical properties. She can instead draw a molecule in ChemDraw and an ML model will immediately return its chemical properties! One could imagine using molecular property predictions as a means of scoring large-scale virtual drug library screens. Here, a chemist would explicitly stipulate her desired properties and a screening system would score a large library of molecules against these properties to return the best fitting candidates. Going further, we could even consider exploring the search space of yet-to-be-synthesised compounds to more rationally direct development efforts. This is all made possible because ML approaches to computational chemistry learn to approximate the underlying mechanisms of chemistry after training on datasets of molecules and property pairings.

2. Generating novel chemical structures

There are an estimated 1,453 drugs that have been approved by the US Food and Drug Administration (FDA) in its entire history. If we take small molecules as a case study (as opposed to protein therapeutics), most drug libraries that can be purchased from suppliers come in batches of 1,000 different molecules. This is literally a drop in the ocean because computational estimates suggest that the actual number of small molecules could be 10⁶⁰ (i.e. 1 million billion billion billion billion billion billion). This is 10–20 orders of magnitude less some estimates of the number of atoms in the universe (for readers who are familiar with the AlphaGo story, this comparison may sound familiar). The American Chemical Society Chemical Abstracts Service database contains almost 146 million organic and inorganic substances, which suggests that almost all small molecules (>99.9%) have never been synthesized, studied and tested.

Taken together, these statistics mean that research labs can only test a few thousand drug candidates in the real world. Once they’ve found a few candidates that look functionally promising, they often iteratively explore structural variants to determine if there are better versions out there. This again requires synthesis experiments and chemical assays. As such, to even make a dent into the 10⁶⁰ drug-like small molecules in existence, we need to use software as a means to rationally and automatically explore the chemical design space for candidates that have desired properties for a specific indication.

To solve this problem, ML researches have turned to sequence-based generative models trained on large databases of chemical structures (often using SMILES representations). Here, chemistry is effectively treated as a language with its own grammar and rule set. Generative models are capable of learning the probability distribution over molecule space such that they implicitly internalise the rules of chemistry. This compares somewhat to how generative models trained over a corpus of English text can learn English grammar. To this end, a 2017 paper from the University of Münster and AstraZeneca presented a recurrent neural network that was trained as a generative model for molecular structures. While the performance wasn’t state-of-the-art, the system could reproduce a small percentage of molecules known to have antibacterial effects. Related work from AstraZeneca expanded on the RNN-based molecule generator by using a policy-based reinforcement learning approach to tune the RNN for the task of generating molecules with predetermined desirable properties. The authors show that the model can be trained to generate analogues to the anti-inflammatory drug Celecoxib, implying that this system could be used for scaffold hopping or library expansion starting from a single molecule.

Other ML methods have been used to build molecule generators. The University of Tokyo and RIKEN borrowed from the AlphaGo architecture to build a system called ChemTS. Here, a SMILES-based molecule generator system is built using Monte Carlo Tree Search for shallow search and an RNN for the rollout of each downstream path. The same group also published an evolutionary approach called ChemGE that relies on genetic algorithms to improve the diversity of molecules that are ultimately generated. Finally, researchers at the University of Cambridge applied a pair of deep networks trained as an autoencoder to convert molecules represented as SMILES strings into a continuous vector representation. The autoencoder was trained jointly on a property prediction task. Finally, even generative adversarial networks (GANs) have been adapted to operate directly on graph-structure molecule data. The MolGAN system from the University of Amsterdam combines GANs with a reinforcement learning objective to encourage the generation of molecules with specific desired chemical properties. Taken together, these papers illustrate the promise for accelerating chemical search space exploration using virtual screening.

3. Chemical synthesis planning

Now that we’ve discussed AI methods for exploring the chemical design space and predicting pharmaceutically-relevant chemical properties of these molecules, the next problem for AI to tackle would be chemical synthesis planning. Chemists use retrosynthesis to figure out the most efficient stepwise chemical reaction process to produce a small molecule of interest. This task is similar (albeit admittedly more complex) to being given a cake and being asked to design the optimal ingredient list and recipe to recapitulate the same cake by working backwards. Given that we’re talking about chemical reactions, the transformations from one molecule state to another are derived from a database of successfully conducted similar reactions with analogous starting materials. These reactions often carry the name of their scientific discoverers. Chemists intuitively select from these reactions, mix and match, and empirically assess the efficiency with which each reaction generates sufficient yield of the desired product. If a molecule doesn’t react as predicted it is “out of scope”. If the molecule reacts as predicted it is “in scope”. Predicting which will occur is challenging and uncertain.

To address this task, researchers at the University of Münster and Shanghai University developed a system for rapid chemical synthesis planning. The system uses three neural networks and Monte Carlo Tree Search (together called 3N-MCTS). It was trained on 12.4 million single-step reactions, which is essentially all reactions ever published in the history of organic chemistry. Paraphrasing from their introduction, “The first neural network (the expansion policy) guides the search in promising directions by proposing a restricted number of automatically extracted transformations. A second neural network then predicts whether the proposed reactions are actually feasible (in scope). Finally, to estimate the position value, transformations are sampled from a third neural network during the rollout phase.”

What’s interesting is that their method is far faster than the state-of-the-art computer-assisted synthesis planning. In fact, 3N-MCTS solves more than 80% of a molecule test set with a time limit of 5 seconds per target molecule. By contrast, an approach called best first search in which functions are learned through a neural network can solve 40% of the test set. Best first search designed with hand-coded heuristic functions performs the worst: it solves 0% in 5s. If provided with infinite runtime, however, the three approaches will converge to the same performance. One could imagine that automated chemical synthesis planning methods like 3N-MCTS should tie in with virtual screening approaches to remove some heavy lifting from chemists.

4. Protein structure prediction

When we think about health and disease, we often focus on genetics. However, the manifestation of phenotypes is actually exerted by proteins, which are encoded by our genetics. For example, mutated genes might encode erroneous proteins that behave the wrong way in pathways responsible for cell division, which could drive the onset of cancer. Moreover, the drugs we take to treat health conditions act by binding to other proteins or DNA in order to modulate the cellular processes that underpin that condition. What’s more, we live in a three-dimensional world (with time as a fourth dimension). This means that understanding how proteins work and how they contribute to health and disease is made a lot easier by having their 3D structure, not just their amino acid sequence.

The problem, however, is that generating a 3D structure for a protein is expensive, complicated and time-consuming. The process requires a protein of interest to be isolated, purified, crystallised and X-Rays blasted at the protein to measure how the beams diffract off the crystal surface. As a result, there are orders of magnitude more proteins in the human body than crystal structures available in public databases. The number of complete amino acid sequences also outnumbers the number of crystal structures available in public databases by orders of magnitude. Researchers have therefore been working on computationally predicting the 3D structure of a protein from its amino acid sequence for years. This is the “protein folding problem”.

Recently, DeepMind published a system called AlphaFold that uses two deep residual convolutional neural networks (CNNs) to reach a new state-of-the-art for protein folding prediction. Specifically, the system predicts (a) the distances between pairs of amino acids and (b) the angles between chemical bonds that connect those amino acids. By predicting how close pairs of amino acids are to one another, the system creates a distance map of the protein. This map can essentially be extrapolated to generate a 3D structure or match one from an existing database. For a longer critical review of the work that is outside the scope of this piece, please check out this post. In theory, AlphaFold could enable scientists to generate de novo 3D protein structures for novel proteins that they discover or for known proteins that have yet to be crystallised.

Furthermore, one can imagine AlphaFold being integrated into drug development pipelines as an additional signal to predict whether a drug of interest will indeed bind its target and, if so, with what affinity.

More work is needed to expand AlphaFold to predict the 3D structure of protein complexes (i.e. several individual proteins working together) and to make the system’s predictions more dependable. On the former point, recent research from Stanford presented the Siamese Atomic Surfacelet Network could help. Their system is an end-to-end learning method for protein interface prediction that uses representations learned from raw spatial coordinates and identities of atoms as inputs. This network predicts which surfaces of one protein bind to which surfaces of another protein. It is worth noting that the 3D conformation of proteins before, during and after binding can change significantly and are also affected by their environment. Models like this one will need to be hugely flexible to accommodate for such real-world nuances.

5. Protein identification using mass spectrometry

In order to elucidate the drivers of health and disease, we must discern which genes and proteins are functionally connected to a phenotype of interest. To do so, we might design experiments where we induce a disease phenotype and compare the genomic and proteomic profiles of diseased cells with healthy cells. If we focus on the proteome, a key method for identifying and quantifying the proteins, their complexes, subunits and functional interactions that are present in a cell mixture is called protein mass spectrometry. Briefly, this protocol works by taking a protein mixture sample, separating the constituent peptides (short sequences of amino acids), ionising them and separating them on a spectrum according to their abundance and mass/charge ratios. The result is a histogram (called a chromatogram) of the different ways that a peptide can fragment where each fragment appears as a peak on the plot. A scientist must interpret this pattern of peaks to find fingerprints that are unique to specific peptides that make up a whole protein. Only by doing so can they reconstruct the protein population in the sample mixture.

While there are a variety of open source and proprietary software tools to analyse mass spectrometry data, a central challenge to data interpretation is that there are few ground truth pairings between peptides and spectra. This means that ML methods in this domain must be able to use existing noisy labels to make progress. Recent work from GoogleAI, Calico Life Sciences and InSitro present a deep learning system called DeepMatch (schematic below) that can identify peptides corresponding to a given mass spectrum based on noisy labels from classical algorithms. The model is also trained to distinguish between genuine true spectra and dummy synthetic negative spectra to help with overcoming overfitting. DeepMatch achieves a 43% improvement over standard matching methods and a 10% improvement over a combination of the matching method and an industry standard cross-spectra reranking tool.

One can imagine that techniques like DeepMatch form part of the early phases of drug discovery where scientists need to rapidly, scalably and more reliably quantify protein candidates that have a potentially causative relationship with a disease phenotype of interest. Interestingly, ML methods applied to mass spectrometry can also be used to for chemicals where the task is to predict the presence of substructures given the mass spectrum, and matching these substructures to the candidate structures.

6. Automated analysis of experimental image data

While virtual screening is very appealing, the power of these computational models depends on three key factors. These include the model architecture, the data representation it uses, and the availability of high-quality ground truth data. For empirical computation to deliver on its promise, we must close the computational-to-real-world feedback loop with rationally-guided wet lab experiments such as omics sequencing and high-content microscopy. These results serve as our ground truth. However, as the volumes, dimensions and resolution of ground truth and experimental datasets grow, the methods we use for data analysis are also due for a refresh with more flexible and reliable ML-driven replacements.

Let’s consider high-throughput drug screens using human cancer cells cultured in vitro as our model system. Assume that we are running a hypothesis-generating experiment whereby many single drug candidates and combinations thereof are tested for their ability to halt the division of human breast cancer cells. In order to detect cancer cells that no longer divide, we must detect and quantify markers of this cell state. We might be interested in localising the cell’s nucleus, its cytoskeleton, as well as proteins that regulate cell division and mark the stage of its cell cycle. The workhorse method we use involves binding a fluorescent antibody probe to the protein marker of choice such that we can localise and quantify the extent of its presence or absence within a cell by fluorescent microscopy. This is called immunofluorescence. There are several technical and workflow challenges to this protocol. For example, sample preparation and staining are quite laborious, results are sensitive to concentrations of target and probe, the protocol is rather inconsistent and is terminal in the sense that the sample cannot be reused afterwards. We’re also limited in the number of proteins we can visualise because of overlap on the fluorescence spectrum. Finally, we’re only “lighting up” what we believe to be important for analytical purposes. Cells are incredibly complex organisms and we might very well be missing key features that help us understand their biology.

Tying back to the opening of this piece, computer vision has come leaps and bounds for complex feature learning in 2D and 3D images thanks to deep learning. This means that we should be able to apply deep learning to automate the analysis of high-content microscopy images. Last year, a collaboration led by UCSF and Google resulted in a deep learning system capable of reliably predicting several fluorescent labels (nuclei, cell type, cell state) from unlabeled transmitted light microscopy images of biological samples. The method is consistent because it is computational and is not limited by spectral overlap. More importantly, in silico predictions mean that we no longer need to “waste” the sample on immunofluorescence. The trained network exhibits transfer learning too, which means that once it has learned to predict a set of labels, it can learn new labels with a small number of additional data.

While this study predicted immunofluorescence in 2 dimensions, a study from the Allen Institute in Seattle demonstrated a system to do the same but in 3 dimensions. This method can, therefore, generate multi-structure, integrated images and can also be extended to predict immunofluorescence from electron micrographs (more detailed, higher magnification images of smaller cellular structures). The Allen Institute paper used a U-Net CNN architecture developed by Olaf Ronnenberger. Indeed, Olaf’s group recently applied the U-Net for automated cell counting, detection and morphometry that comes as a simple plugin to the popular (but very basic) image analysis software tool called ImageJ. Side joke: I always had the feeling ImageJ was like Microsoft Paint for biologists 😂. The U-Net was also used by DeepMind and Moorfield’s Eye Hospital in their Nature Medicine paper for segmenting optical computed tomography scans of the eye. Finally, GANs have proven useful for transforming diffraction-limited input images into super-resolved ones, which could one day obviate the need for using expensive super-resolution microscopes. Indeed, a study from UCLA developed a framework to improve the resolution of wide-field images acquired with low-numerical-aperture objectives, matching the resolution that is acquired using high-numerical-aperture objectives. Neat! Taken together, it is clear that computer vision methods applied to microscopy will enable much more reliable and automated visual data interpretation. Doing so could potentially open new questions and offer new takes on our current understanding of biology.

Conclusion

I hope that this essay has shined a light onto current progress around AI applications to life sciences and pointed to the immense potential in front of us. This piece is by no means comprehensive and somewhat reflects my personal bias of what I find impactful and scientifically interesting. If you’d like me to include or feature your work, don’t hesitate to get in touch in the comments or drop me a note 👉@nathanbenaich.

🔬6 impactful applications of AI to the life sciences [new essay] was originally published in Stories from Air Street on Medium, where people are continuing the conversation by highlighting and responding to this story.

London.AI is a community of AI practitioners from technology companies, startups and academia.

Yesterday evening, we hosted our 14th edition of London.AI at the QuantumBlack offices in central London. Drawing from 412 applicants, we were able to host approx. 100 engineers, researchers, founders, students and operators from organisations such as:

👉 Check our newsletter to hear about our next event!

Speakers

Scenes from the evening

Introduction from Pippa Kennedy at QuantumBlack.

Mostafa, CEO of Automata, taking us through the founding journey for the Company. He began life as an architect in the computation and design group at the World-acclaimed Zaha Hadid.

Philip Torr (CSO) and Stan Boland (CEO, Founder) from FiveAI running through their presentation entitled The World is Not Enough. HT to James Bond :)

Alex Caccia from Animal Dynamics showing us a promo reel for their long-distance aircraft.

A customary tweet-storm to recap the evening’s learnings!

Glenn Wilkinson 🇿🇼 on Twitter

Some things I learned about #robotics at @LDN_AI from the three speakers last night: (1) @FiveAI (autonomous cars): Twinning is hard (training autos in simulated environments) but they're building an incredible replica of London to train their vehicles in. [1/6]

Glenn Wilkinson 🇿🇼 on Twitter

European roads are very different to US roads for autos (medieval, many GPS shadows). CCTV footage is great for learning driver behaviour, but red tape is tricky. [2/6]

Glenn Wilkinson 🇿🇼 on Twitter

(2) @AnimalDynamics_ : Mimicking nature for mimics sake alone is a bad idea, but taking inspiration and nature's rapid iterative approach is good. Dragon fly wings are the most incredible piece of engineering ever. [3/6]

Glenn Wilkinson 🇿🇼 on Twitter

Using £40m helicopters to deliver rice & water is inefficient, but using a 15kg autonomous microlight to deliver 50kgs is amazing. Improving batteries is good, but vehicle efficiency is more interesting. We don't really understand how fish swim, and it's actually amazing. [4/6]

Glenn Wilkinson 🇿🇼 on Twitter

(3) @automata_tech (building robot arms): Career changes are possible (architects to roboteers)! Robot arms are either <£1k and toys or >£15k and clunky industrial things. 93% of world wide manufacturing is still manual. [5/6]

Glenn Wilkinson 🇿🇼 on Twitter

Two (but really one) companies supply gearboxes for all robot arms in the world (so these dudes built their own!). I really, really want an Eva robot arm. [6/6]

— by Nathan Benaich and Alexandre Flamant

London.AI 14 — robotics with FiveAI, Animal Dynamics and Automata was originally published in London.AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Investments

Q4 2018 has been an active one! Here’s a highlight of the most intriguing financing rounds:

Zymergen, a Bay Area company with a mission to search beyond the bounds of human intuition to deliver novel products and materials, closed a $400M Series C investment led by SoftBank’s Vision Fund (SoftBank proper led their Series B). This business is often held up as a posterchild for how the software industry can intersect with life sciences to accelerate what is an otherwise slow discovery and development process. The key idea is to use machine learning (and other computational methods with lab automation) with real-world and simulated data to navigate a vast search space of potential (in)organic molecules to make discoveries in a much more directed fashion. Zymergen now has over 500 employees, 42% of whom are in research and 21% are in engineering.

Horizon Robotics, a Chinese semiconductor company focused on embedded computer vision processors for large-scale facial recognition in security and mobility, is on the market to raise $1B at $3B-$4B valuation. Horizon is yet another Chinese AI startup founded by former members of Baidu’s self-driving car project (another example being PonyAI).

Automation Anywhere, one of the big 3 robotic process automation vendors, raised a $300M extension to their $250M Series A that was announced only in July. This round is led by SoftBank Vision Fund at $2.6B post-money while the prior one was led by Goldman Sachs and NEA at $1.8B post-money.

DataRobot, vendor of a predictive analytics platform for enterprises to build and deploy predictive models the cloud or on-premise, continues its rapid growth by raising a $100M Series D led by Sapphire Ventures and Meritech.

A few semiconductor related financings:

Esperanto Technologies, a maker of energy-efficient RISC-V computing solutions, raised a $63M Series B;

Habana Labs, the Israeli maker of a separate training and inference AI processors, raised a $75M Series B led by Intel along with Bessemer and Battery Ventures.

Wave Computing, which provides dataflow-based systems to “eliminate the need for a host and co-processor in the processing of a neural network”, raised $86M in Series E financing.

Two businesses raised significant capital to apply AI techniques in the sales arena. First, Afiniti raised $130M at a $1.6B valuation. On the one hand their product analyses sales people’s performance with specific types of calls and situations, and on the other hand, it analyses customers’ prior interactions with a company. It then matches up customer service reps who it believes will be most compatible with specific customers. Second, Chorus.ai raised a $33M Series B from Georgian Partners to help sales teams have “higher quality conversations that result in higher quota attainment, higher rep productivity, and shorter new hire ramp time”.

Apex.ai, led by Jan Becker (a longtime Bosch AV engineer and member of Sebastian Thrun’s Darpa robotics challenge team), raised a $15.5M Series A led by Canaan and Lightspeed to build an operating system for self-driving cars.

AEye, whose iDAR sensor couples a solid state LiDAR and high-resolution camera in a single device to detect and track moving vehicles up to a kilometer away (!), raised a $40M Series B led by Taiwan’s government-backed venture fund.

Standard Cognition, an Amazon Go-style autonomous checkout system (yeah, it’s becoming Uber for X at this point), raised a $40M Series A led by Initialized Capital.

Primer, which automates the understanding of large text corpuses in the enterprise, raised $40M Series B led by Lux Capital to expand into new verticals outside of fintech and government.

Appzen, an AI-based solution for auditing employee expenses in the enterprise back office, close a $35M Series B led by Lightspeed.

M&A deals

Cylance, a Bay Area cybersecurity company focused on predicting and preventing the execution of advanced threats against endpoints, was acquired for $1.4B by Blackberry in the latter’s bid to evolved into a software-only vendor. Cylance’s technology is deployed over 14.5 million endpoints in including many Fortune 100 organizations and governments. The company was founded in 2012 by Ryan Permeh (formerly Chief Scientist at McAfee) and Stuart McClure (formerly EVP and Worldwide CTO at McAfee). The business has almost 1,000 employees and raised $297M.

Blue Vision Labs, a London-based post-Series A stage company building city-scale 3D maps for shared AR experiences, was acquired by Lyft’s Level 5 (self-driving) team. The deal is worth $72M with an extra $30M on the basis of milestones. BVL built a team of 40 people, many of whom have significant computer vision, robotics and machine learning experience from Oxford and Imperial, as well as large tech companies. The company had opened their developer SDK in Q1 2018 ahead of Niantic’s Real World Platform that was announced in June 2018.

Silk Labs, a US startup focused on on-device AI to empower businesses to build the next generation of intelligent connected devices, was acquired by Apple for an undisclosed amount. The company was founded by two former Mozilla alums and a Qualcomm alum in 2015. The team ran a Kickstarter campaign where they raised $150k (after $4M in VC capital) for an AI-enabled smart camera and hub, called Sense. The product’s USP was that all the processing was done locally on a Qualcomm chip. The product never shipped and money was returned to backers.

Research Papers

Optimizing Agent Behavior over Long Time Scales by Transporting Value, DeepMind.

How often have you reflected on the outcome a decision that you made a week or a month ago? Learning from our successes and mistakes requires linking actions and consequences over long spans of time (the “credit assignment problem”). Doing so is key to our ability to learn efficiently. In this paper, the authors explore this feature of human nature that is otherwise absent in AI models today that can only reason over short timescales. The authors introduce a new paradigm for reinforcement learning that is based on the following three principles where agents must: 1) Encode and store perceptual and event memories; 2) Predict future rewards by identifying and accessing memories of those past events; 3) Re-evaluate these past events based on their contribution to future reward. Their system, Temporal Value Transport (TVT), integrates these requirements by using neural network attentional memory mechanisms to credit distant past actions for future rewards. According to the authors, “the algorithm is not without heuristic elements, but we prove its effectiveness for a set of tasks requiring long-term temporal credit assignment over delay periods that pose enormous difficulties to conventional deep RL.”

Visualizing and understanding generative adversarial networks, ICLR 2019 submission.

The authors sought to identify an interpretable structure to GANs that could provide a window into their internal representations. Interestingly, they found many parts of GAN representations that can be interpreted to have causal effects on the synthesis of objects in the output image. Of note, these interpretable effects can be used to compare, debug, modify, and reason about a GAN model. This could make its way into photo editing software, for example.

Large-scale GAN training for high fidelity natural image synthesis, ICLR 2019 submission.

Some seriously pretty class-conditional samples generated synthetically by their BigGAN. And some more here from NVIDIA Research.

Deep Imitative Models for Flexible Inference, Planning, and Control, Berkeley.

The authors present Imitative Models, a class of probabilistic predictive models that can plan expert-like trajectories to achieve arbitrary goals. They combine the flexibility benefits from model-based RL (MBRL) and the ability of imitation learning to learn from human demonstrations. They show that deep imitative models substantially outperform both direct imitation and classical MBRL in simulated driving tasks and can be learned efficiently from a fixed set of expert demonstrations.

ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst, Waymo Research

In this study, the authors seek to build a system to drive by imitating an expert. They created a deep RNN, ChauffeurNet, that is trained to generate a driving trajectory by observing a “mid-level representation of the scene” as an input. Interestingly, this representation is not built directly from raw sensor data. As such, it factors out the perception task and allows Waymo to combine real and simulated data for easier transfer learning. Blog post here.

Measuring the Effects of Data Parallelism on Neural Network Training, Google Brain.

Continuing my search for empirical evidence around data requirements to build high-performance ML models, I bring you this paper. Here, the authors explore the exact relationship between the batch size (e.g. how many images to use for each pass through a neural network) and how many training steps are necessary. They explore how this relationship varies with the training algorithm, model, and data set. The study finds that for idealized data parallel hardware, there is a universal relationship between training time and batch size, but there is dramatic variation in how well different workloads can make use of larger batch sizes. On the similar topic, OpenAI publish results on how gradient noise predicts the parallelizability of neural network training on a wide range of tasks.

Beyond imitation: Zero-shot task transfer on robots by learning concepts as cognitive programs, Vicarious AI.

This paper learns concepts as programs on a ‘visual cognitive computer’, showing zero-shot task-transfer on robots, and strong generalization. Vicarious bring ideas from cognitive science about perceptual symbol systems and image schemas into the realm of machine learning.

Episodic Curiosity through Reachability, DeepMind, GoogleAI, ETH Zurich.

In this paper, the authors propose a novel episodic memory-based model of granting rewards to reinforcement learning agents when they are solving tasks that otherwise have sparse rewards. The rewards in this model are akin to curiosity, i.e. encouraging the RL agent to explore the environment but also solve the original task. They do this by enabling the RL agent to store observations from its interactions with the environment in memory. The reward is then calculated as a function of how far the current observation is from the most similar observation in memory. Specifically, a neural network is trained to predict whether two observations were experienced close together in time or far apart. If the agent makes observations not already stored in memory, it gets more reward.

Exploration by Random Network Distillation, OpenAI and Edinburgh.

Inspired by related challenges than the Episodic Curiosity paper above, this study introduces an “exploration bonus”, where we predict the output of a fixed randomly initialised neural network based on the current observations of a deep RL agent in a game environment. This bonus is based on the notion that neural networks tend to have significantly lower prediction errors on examples similar to those on which they have been trained. This motivates the use of prediction errors of networks trained on the agent’s past experience to quantify the novelty of the new experience. The authors reach state of the art performance on Montezuma’s Revenue:

Model-Based Active Exploration, NNAISENSE.

A third paper released in the same week on RL methods that encourage exploration to improve agent learning!

Reward learning from human preferences and demonstrations in Atari, OpenAI and DeepMind.

This paper seeks to address how agents can solve tasks where the subgoals are poorly defined or hard to specify as a hard-coded reward function. This is important because most complex tasks in life don’t have an obvious reward function that can be written down. They solve this by initializing the agent’s policy with imitation learning from the expert demonstrations using the pretraining part of the DQfD algorithm (Hester et al., 2018). Second, they use trajectory preferences and expert demonstrations to train a reward model that lets them improve on the policy learned from imitation.

Three reviews of EMNLP 2018, a leading natural language processing conference: here (Patrick Lewis, UCL), here (Sebastian Ruder, Aylien) and here (Claudia Hauff, TU Delft).

Here’s a list of the best paper awards at NeurIPS 2018.

What can a machine infer from a single image? First, it was depth estimation (the team now works at Niantic Labs). Now, it’s 3D object detection (this team works with Wayve.ai).

Visual Model-Based Reinforcement Learning as a Path towards Generalist Robots:

This paper presents an algorithm that can learn a general-purpose predictive model using unlabeled sensory experiences. They show that this single model can enable a robot to perform a wide range of tasks. Sergey Levine’s group also released their SOTA off-policy model-free reinforcement learning algorithm, soft actor-critic (SAC). It is arguably one of the most efficient model-free algorithms available today to enable robotic locomotion, object manipulation and more.

📑 Resources

Blog posts

Bringing machine learning research to product commercialisation: a worthwhile resource from Rasmus Rothe at Merantix. The playbook on this topic continues to be written!

Owkin published a neat primer for physicians to understand the fundamentals of ML and an example use case of predicting the age of a patient’s brain from MRI scans.

Libby Kinsey published a neat overview of the challenges of infrastructure plumbing for AI-first applications and current solution providers that are on the market. While we’re still in early days of seeing AI-first products in the wild, current deployments are already pointing to key issues with existing infrastructure that was developed for the traditional web SaaS/consumer/marketplaces application environment. In particular, this excerpt from a Google paper illustrates some differences to how we should think about testing and monitoring:

Lyft’s Level 5 team authored a nice introduction to the constituents of HD maps and what they’re used for.

What are the challenges with analysing biological and artificial neural networks? Read more here!

NYU’s AI Now Institute published their latest 2018 report. The study focuses on the “accountability gap” in AI between creators and users, AI for surveillance and government’s use of automated decision systems, fairness, bias and discrimination, as well as unregulated AI experimental on human populations. The authors identify emerging challenges within these topics and provide practical solution pathways informed by research so that policymakers, the public, and technologists can better understand and mitigate risks.

A new Pew survey of nearly 1,000 tech experts found that fewer than two-thirds expect AI technology to make most people better off in 2030 than today. Many express a fundamental concern that AI will specifically be harmful.

Robot Economy: Ready or Not, Here It Comes: A technological framework describing a robot economy is outlined and the challenges it might represent in the current socioeconomic scenario are discussed.

Videos/lectures

DeepMind’s lectures at UCL in London on “Advanced Deep Learning and Reinforcement Learning” are live on YouTube.

Datasets

VoxCeleb is the largest audio-visual dataset of human interviews from YouTube. It includes over 7,000 identities, over 1 million utterances and over 2,000 hours of video with audio files, face detections, tracks and speaker meta-data available. It’ll be useful for speaker recognition in video, speech separation, face synthesis and emotion recognition.

Researchers at the Allen Institute for AI in Washington have contributed a dataset to further visual common sense reasoning. The dataset includes 290k multiple choice QA problems derived from 110k movie scenes. In particular, the authors present an approach for transforming rich annotations into multiple choice questions with minimal bias.

A practical approach to learning machine learning, all in notebooks!

Open source tools

A series of Jupyter Notebooks providing a step-by-step introduction to data science and machine learning.

While the code behind the majority of DeepMind’s key programs continue to be closed source, the company’s research engineering team released a new TensorFlow-based library of building blocks for writing RL agents.

Two new libraries for private machine learning in TensorFlow (paper) or PyTorch (link).

Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models (link). This is a new effort for AI-driven drug discovery led by Insilico Medicine.

OpenAI released CoinRun, a new RL training environment that provides a metric for an agent’s ability to transfer its experience to novel situations.