The Research Excellence award is given to a scientist who has carried out a program of research of consistently high quality throughout an entire career yielding several substantial results. Past recipients of this honor are the most illustrious group of scientists from the field of Artificial Intelligence.
They are: John McCarthy (1985), Allen Newell (1989), Marvin Minsky (1991), Raymond Reiter (1993), Herbert Simon (1995), Aravind Joshi (1997), Judea Pearl (1999), Donald Michie (2001), Nils Nilsson (2003), Geoffrey E. Hinton (2005), Alan Bundy (2007), Victor Lesser (2009), Robert Anthony Kowalski (2011), Hector Levesque (2013), Barbara Grosz (2015), Michael I. Jordan (2016), Andrew Barto (2017), Jitendra Malik (2018), Yoav Shoham (2019) and Eugene Freuder(2020).
They are: John McCarthy (1985), Allen Newell (1989), Marvin Minsky (1991), Raymond Reiter (1993), Herbert Simon (1995), Aravind Joshi (1997), Judea Pearl (1999), Donald Michie (2001), Nils Nilsson (2003), Geoffrey E. Hinton (2005), Alan Bundy (2007), Victor Lesser (2009), Robert Anthony Kowalski (2011), Hector Levesque (2013), Barbara Grosz (2015), Michael I. Jordan (2016), Andrew Barto (2017), Jitendra Malik (2018), Yoav Shoham (2019) and Eugene Freuder(2020).
The winner of the 2021 Award for Research Excellence is Richard Sutton, Professor Department of Computing Science, University of Alberta. Professor Sutton is recognized for his fundamental contributions to the foundation of reinforcement learning.
The winner of this award will give a talk on Aug 24 at 00:00 Montreal time (EDT).
Title of the talk: The Increasing Role of Experience in AI
Abstract
We receive information about the world through our sensors and influence the world through our effectors. Such low-level experience has gradually come to play a greater role in AI over its 70-year history. I see this occurring in four steps, two of which are mostly past and two of which are in progress or yet to come. The first step was to view AI as the design of agents which interact with the world and thereby have experience; this viewpoint became prominent in the 1980s and 1990s (e.g., Russell & Norvig 1995). The second step was to view the goal of intelligence in terms of experience, as in the reward signal of optimal control and reinforcement learning. The reward formulation of goals is now widely used but rarely loved. Many would prefer to express goals in non-experiential terms, such as reaching a destination or benefiting humanity, but settle for reward because, as an experiential signal, reward is directly available to the agent without human assistance or interpretation. This is the pattern that we see in all four steps. Initially a non-experiential approach seems more intuitive, is preferred and tried, but ultimately proves a limitation on scaling; the experiential approach is more suited to learning and scaling with computational resources. The third step in the increasing role of experience in AI concerns the agent’s representation of the world’s state. Classically, the state of the world is represented in objective terms external to the agent, such as “the grass is wet” and “the car ten meters in front of me is slowing down”, or with probability distributions over world states such as in POMDPs and other Bayesian approaches. Alternatively, the state of the world can be represented experientially in terms of summaries of past experience (e.g., the last four Atari video frames input to DQN) or predictions of future experience (e.g., Predictive State Representations). The fourth step is potentially the biggest: world knowledge. Classically, world knowledge has always been expressed in terms far from experience, and this has limited its ability to be learned and maintained. Today we are seeing more calls for knowledge to be predictive and grounded in experience. After reviewing the history and prospects of the four steps, I propose a minimal architecture for an intelligent agent that is entirely grounded in experience.
