The world's largest robot maker, Fanuc, is developing robots that use reinforcement learning to figure out how to do things.
Inside a modest-looking office building in Tokyo lives an unusually clever industrial robot made by the Japanese company Fanuc. Give the robot a task, like picking widgets out of one box and putting them into another container, and it will spend the night figuring out how to do it. Come morning, the machine should have mastered the job as well as if it had been programmed by an expert.
Fanuc demonstrates a robot trained through reinforcement learning at the International Robot Exhibition in Tokyo in December.
Industrial robots are capable of extreme precision and speed, but they normally need to be programmed very carefully in order to do something like grasp an object. This is difficult and time-consuming, and it means that such robots can usually work only in tightly controlled environments.
Industrial robots are capable of extreme precision and speed, but they normally need to be programmed very carefully in order to do something like grasp an object. This is difficult and time-consuming, and it means that such robots can usually work only in tightly controlled environments.
Fanuc’s robot uses a technique known as deep reinforcement learning to train itself, over time, how to learn a new task. It tries picking up objects while capturing video footage of the process. Each time it succeeds or fails, it remembers how the object looked, knowledge that is used to refine a deep learning model, or a large neural network, that controls its action. Deep learning has proved to be a powerful approach in pattern recognition over the past few years.
“After eight hours or so it gets to 90 percent accuracy or above, which is almost the same as if an expert were to program it,” explains Shohei Hido, chief research officer at Preferred Networks, a Tokyo-based company specializing in machine learning. “It works overnight; the next morning it is tuned.”
Robotics researchers are testing reinforcement learning as a way to simplify and speed up the programming of robots that do factory work. Earlier this month, Google published details of its own research on using reinforcement learning to teach robots how to grasp objects.
The Fanuc robot was programmed by Preferred Networks. Fanuc, the world’s largest maker of industrial robots, invested $7.3 million in Preferred Networks in August last year. The companies demonstrated the learning robot at the International Robot Exhibition in Tokyo last December.
One of the big potential benefits of the learning approach, Hido says, is that it can be accelerated if several robots wok in parallel
and then share what they have learned. So eight robots working for one hour can perform the same learning as a machine going for eight hours. "Our project is oriented to distributed learning." Hido says. "You can imagine hundreds of factory robots sharing information."
and then share what they have learned. So eight robots working for one hour can perform the same learning as a machine going for eight hours. "Our project is oriented to distributed learning." Hido says. "You can imagine hundreds of factory robots sharing information."
This form of distributed learning, sometimes called "cloud robotics,|" is shaping to be a big trend both in research
and industry.
and industry.
"Fanuc is well place to think about this," says Ken Goldberg, a professor of robotics at the University of California Berkeley, because it installs so many machine in factories around the world. He adds that cloud robotics will most likely reshape the way that robots are used in the coming years.
Goldberg and colleagues (include several resuarchers in Google) are in fact taking this a step further by teaching robots how certain movements may be used to grasp not only specific objects
but also certain shapes. A paper on this work will appear at the IEEE International Conference on Robotics and Automation this May.
but also certain shapes. A paper on this work will appear at the IEEE International Conference on Robotics and Automation this May.
However, Goldberg notes, applying machine learning to robotics is challenging because controlling behavior is more complex than, say, recognizing objects in images. "Deep Learning has made enormous progress i pattern recognition ," Goldberg syas. "The challenge in robotics is that you are doing something beyond that. You need to be able to generate appropriate action for a huge range of inputs."
Fanuc may not be the only company developing robots that use machine learning. In 2014, a Swiss robot manufacturer ABB invested in another AI startup called Vicarious. The fruits of the investment has yet to appear, however.
Cloud Robotics and Automation
What if robots and automation systems were not limited by onboard computation, memory, or software?
Rather than viewing robots and automated machines as isolated systems with limited computation and memory, "Cloud Robotics and Automation" considers a new paradigm where robots and automation systems exchange data and perform computation via networks. Extending earlier work that links robots to the Internet,
Cloud Robotics and Automation builds on emerging research in cloud computing, machine learning, big data, open-source software, and major industry initiatives in the "Internet of Things", "Smarter Planet", "Industrial Internet", and "Industry 4.0."
Consider Google's autonomous car. It uses the network to index maps, images, and data on prior driving trajectories, weather, and traffic to determine spatial localization and make decisions. Data from each car is shared via the network for statistical optimization and machine learning performed by grid computing in the Cloud.
Another example is Kiva Systems approach to warehouse automation and logistics using large numbers of mobile platforms to move pallets using a local network to coordinate platforms and share updates on floor conditions.
Google's James Kuffner coined the term "Cloud Robotics" in 2010. Cloud Robot and Automation systems can be broadly defined as any robot or automation system that relies on data or code from a network to support its operation, i.e., where not all sensing, computation, and memory is integrated into a single standalone system.
There are at least four potential advantages to using the Cloud: 1) Big Data: access to updated libraries of images, maps, and object/product data, 2) Cloud Computing: access to parallel grid computing on demand for statistical analysis, learning, and motion planning, 3) Collective Learning: robots and systems sharing trajectories, control policies, and outcomes, and 4) Human Computation: use of crowdsourcing to tap human skills for analyzing images and video, classification, learning, and error recovery. The Cloud can also provide access to a) datasets, publications, models, benchmarks, and simulation tools, b) open competitions for designs and systems, and c) open-source software. It is important to recognize that Cloud Robotics and Automation raises critical new questions related to network latency, quality of service, privacy, and security.
The term "Singularity" is sometimes used to describe a punctuation point in the future where Artificial Intelligence (AI) surpasses human intelligence. The term was popularized by science fiction author Vernor Vinge and Ray Kurzweil. Superintelligence, a 2014 book by Nick Bostrom, explored similar themes that provoked Stephen Hawking, Elon Musk, and Bill Gates to issue warnings about the dangers of AI and robotics. My sense is that the Singularity is distracting attention from a far more realistic and important development that we might call "Multiplicity". Multiplicity characterizes an emerging category of systems where diverse groups of humans work together with diverse groups of machines to solve difficult problems. Multiplicity combines the wisdom of crowds with the power of cloud computing and is exemplified by many Cloud Robotics and Automation systems.
