Abstract
Learned world models summarize an agent’s experience to facilitate learning
complex behaviors. While learning world models from high-dimensional sensory
inputs is becoming feasible through deep learning, there are many potential
ways for deriving behaviors from them. We present Dreamer, a reinforcement
learning agent that solves long-horizon tasks from images purely by latent
imagination. We efficiently learn behaviors by propagating analytic gradients
of learned state values back through trajectories imagined in the compact state
space of a learned world model. On 20 challenging visual control tasks, Dreamer
exceeds existing approaches in data-efficiency, computation time, and final
performance.
Behaviors Learned by Dreamer
![](/assets/behavior/dmc-cheetah-run.gif)
![](/assets/behavior/dmc-hopper-hop.gif)
![](/assets/behavior/dmc-walker-run.gif)
![](/assets/behavior/dmc-quadruped-run.gif)
![](/assets/behavior/dmc-cup-catch.gif)
![](/assets/behavior/dmc-cartpole-swingup-sparse.gif)
![](/assets/behavior/dmc-pendulum-swingup.gif)
![](/assets/behavior/dmc-acrobot-swingup.gif)
![](/assets/behavior/dmc-reacher-easy.gif)
![](/assets/behavior/dmc-reacher-hard.gif)
![](/assets/behavior/dmc-finger-spin.gif)
![](/assets/behavior/dmc-finger-turn-hard.gif)
Atari and DMLab with Discrete Actions
![](/assets/behavior/atari-fishing-derby.gif)
![](/assets/behavior/atari-ice-hockey.gif)
![](/assets/behavior/atari-kungfu-master.gif)
![](/assets/behavior/atari-assault.gif)
![](/assets/behavior/atari-boxing.gif)
![](/assets/behavior/atari-hero.gif)
![](/assets/behavior/atari-freeway.gif)
![](/assets/behavior/atari-frostbite.gif)
![](/assets/behavior/atari-montezuma.gif)
![](/assets/behavior/atari-pong.gif)
![](/assets/behavior/atari-tennis.gif)
![](/assets/behavior/dmlab-collect.gif)
Multi-Step Video Predictions
![](/assets/pred/quadruped.gif)
![](/assets/pred/fishing-derby.gif)
![](/assets/pred/collect.gif)
Read the Paper for Details [PDF]
![](/assets/pages/1.png)
![](/assets/pages/2.png)
![](/assets/pages/3.png)
![](/assets/pages/4.png)
![](/assets/pages/5.png)
![](/assets/pages/6.png)
![](/assets/pages/7.png)
![](/assets/pages/8.png)
![](/assets/pages/9.png)