An artificial intelligence that is rewarded for exploring unknown situations has played the Atari game Montezuma’s Revenge – learning from mistakes and identifying sub-goals 10 times faster than Google DeepMind, according to RMIT University in Australia, where the algorithm was developed.
“A 2015 study showed Google DeepMind AI learnt to play Atari video games like Video Pinball to human level, but notoriously failed to learn a path to the first key in 1980s video game Montezuma’s Revenge due to the game’s complexity,” said the university.”
As a way to driving learning where rewards are not otherwise obvious, the method combines carrot-and-stick reinforcement learning with an intrinsic motivation that rewards the AI for being curious and exploring its environment.
“Truly intelligent AI needs to be able to learn to complete tasks autonomously in ambiguous environments,” said researcher Fabio Zambetta. “We’ve shown that the right kind of algorithms can improve results using a smarter approach rather than purely brute-forcing a problem on very powerful computers.”
In the game, for example, a player needs to identify sub-tasks such as climbing ladders, jumping over an enemy and picking up a key before they can get to the second screen.
The AI is set up to get reward from autonomously exploring such sub-goals, which may not be obvious routes to completing a larger mission.
Other state-of-the-art systems, according to the university, have required human input to identify these sub-goals or else decided what to do next randomly.
“Not only did our algorithms autonomously identify relevant tasks roughly 10 times faster than Google DeepMind while playing Montezuma’s Revenge, they also exhibited relatively human-like behaviour while doing so,” said Zambetta. “This would eventually happen randomly after a huge amount of time, but to happen so naturally in our testing shows some sort of intent. This makes ours the first fully autonomous sub-goal-oriented agent to be truly competitive with state-of-the-art agents on these games.”
He went on to say that the technique is applicable to tasks outside video games.
“Creating an algorithm that can complete video games may sound trivial, but the fact we’ve designed one that can cope with ambiguity while choosing from an arbitrary number of possible actions is a critical advance,” said Zambetta. “It means that, with time, this technology will be valuable to achieve goals in the real world.”
The work will be presented at the AAAI Conference on Artificial Intelligence in Hawaii tomorrow in th epaper ‘Deriving subgoals autonomously to accelerate learning in sparser reward domains’.
SOURCE/LINK:
https://www.electronicsweekly.com/news/research-news/ai-algorithm-learns-play-atari-games-10x-quicker-2019-01/