Augmenting SLAM with deep learning

Some elements of the Spatial AI real-time computation graph. Click image to enlarge. Credit: SLAMcore

Simultaneous localization and mapping (SLAM) is the computational problem of constructing or updating a map of an unknown environment while simultaneously keeping track of a robot’s location within it. SLAM is being gradually developed towards Spatial AI, the common sense spatial reasoning that will enable robots and other artificial devices to operate in general ways in their environments.

This will enable robots to not just localize and build geometric maps, but actually interact intelligently with scenes and objects.

Enabling semantic meaning

A key technology that is helping this progress is deep learning, which has enabled many recent breakthroughs in computer vision and other areas of AI. In the context of Spatial AI, deep learning has most obviously had a big impact on bringing semantic meaning to geometric maps of the world.

Convolutional neural networks (CNNs) trained to semantically segment images or volumes have been used in research systems to label geometric reconstructions in a dense, element-by-element manner. Networks like Mask-RCNN, which detect precise object instances in images, have been demonstrated in systems that reconstruct explicit maps of static or moving 3D objects.

Deep learning vs. estimation

In these approaches, the divide between deep learning methods for semantics and hand-designed estimation methods for geometrical estimation is clear. More remarkable, at least to those of us from an estimation background, has been the emergence of learning techniques that now offer promising solutions to geometrical estimation problems. Networks can be trained to predict robust frame-to-frame visual odometry; dense optical flow prediction; or depth prediction from a single image.

When compared to hand-designed methods for the same tasks, these methods are strong on robustness, since they will always make predictions that are similar to real scenarios present in their training data. But designed methods still often have advantages in flexibility in a range of unforeseen scenarios, and in final accuracy due to the use of precise iterative optimization.

The three levels of SLAM, according to SLAMcore. Credit: SLAMcore”

The role of modular design

It is clear that Spatial AI will make increasingly strong use of deep learning methods, but an excellent question is whether we will eventually deploy systems where a single deep network trained end to end implements the whole of Spatial AI.  While this is possible in principle, we believe that this is a very long-term path and that there is much more potential in the coming years to consider systems with modular combinations of designed and learned techniques.

There is an almost continuous sliding scale of possible ways to formulate such modular systems. The end-to-end learning approach is ‘pure’ in the sense that it makes minimum assumptions about the representation and computation that the system needs to complete its tasks. Deep learning is free to discover such representations as it sees fit. Every piece of design which goes into a module of the system or the ways in which modules are connected reduces that freedom. However, modular design can make the learning process tractable and flexible, and dramatically reduce the need for training data.

Building in the right assumptions

There are certain characteristics of the real world that Spatial AI systems must work in that seem so elementary that it is unnecessary to spend training capacity on learning them. These could include:

  • Basic geometry of 3D transformation as a camera sees the world from different views
  • Physics of how objects fall and interact
  • The simple fact that the natural world is made up of separable objects at all
  • Environments are made up of many objects in configurations with a typical range of variability over time which can be estimated and mapped.

By building these and other assumptions into modular estimation frameworks that still have significant deep learning capacity in the areas of both semantics and geometrical estimation, we believe that we can make rapid progress towards highly capable and adaptable Spatial AI systems. Modular systems have the further key advantage over purely learned methods that they can be inspected, debugged and controlled by their human users, which is key to the reliability and safety of products.

We still believe fundamentally in Spatial AI as a SLAM problem, and that a recognizable mapping capability will be the key to enabling robots and other intelligent devices to perform complicated, multi-stage tasks in their environments.

For those who want to read more about this area, please see my paper “FutureMapping: The Computational Structure of Spatial AI Systems.”

Andrew Davison, SLAMcore

About the Author

Professor Andrew Davison is a co-founder of SLAMcore, a London-based company that is on a mission to make spatial AI accessible to all. SLAMcore develops algorithms that help robots and drones understand where they are and what’s around them – in an affordable way.

Davison is Professor of Robot Vision at the Department of Computing, Imperial College London and leads Imperial’s Robot Vision Research Group has spent 20 years conducting pioneering research in visual SLAM, with a particular emphasis on methods that work in real-time with commodity cameras.

He has developed and collaborated on breakthrough SLAM systems including MonoSLAM and KinectFusion, and his research contributions have over 15,000 academic citations. He also has extensive experience of collaborating with industry on the application of SLAM methods to real products.

How to explore inaccessible places by swarms of sensors

Since the dawn of humankind, exploration of certain places, ranging from the depths of the oceans to the edges of the universe, has led to numerous discoveries. However, there are also several environments that need to be examined but can't be directly observed, like chemical or nuclear reactors, underground water or oil distribution pipes, space and inside of the body. The EU-funded Phoenix project has been addressing this challenge by developing a new line of technology that will offer the opportunity to get to unreachable places.

Kollmorgen to present advanced motion control for commercial robots at Robotics Summit & Expo

Kollmorgen will exhibit its newest motion-centric automation solutions for designers and manufacturers of commercial robots and intelligent systems at the Robotics Summit & Expo 2019. Visitors are invited to Booth 202 to see and participate in a variety of product exhibits and exciting live demos.

Demos and other exhibits have been designed to show how Kollmorgen’s next-generation technology helps robot designers and manufacturers increase efficiency, uptime, throughput, and machine life.

Demonstrations

The AKM2G Servo Motor delivers the best power and torque density on the market, offering OEMs a way to increase performance and speed while cutting power consumption and costs. Highly configurable, with six frame sizes with up to five stack lengths, and a variety of selectable options (such as feedback, mounting, and performance capabilities), the AKM2G can easily be dropped into existing designs.

Robotic Gearmotor Demo: Discover how Kollmorgen’s award-winning frameless motor solutions integrate seamlessly with strain wave gears, feedback devices, and servo drives to form a lightweight and compact robotic joint solution. Kollmorgen’s standard and custom frameless motor solutions enable smaller, lighter, and faster robots.

AGVs and Mobile Robots: Show attendees can learn about Kollmorgen’s flexible, scalable vehicle control solutions for material handling for smart factories and warehouses with AGVs and mobile robots.

Panel discussion

Kollmorgen's Tom Wood will speak at the Robotics Summit & Expo

Tom Wood, Kollmorgen

Tom Wood, frameless motor product specialist at Kollmorgen, will participate in a session at 3:00 p.m. on Wednesday, June 5, in the “Technology, Tools, and Platforms” track at the Robotics Summit & Expo. He will be part of a panel on “Motion Control and Robotics Opportunities,” which will discuss new and improved technologies. The panel will examine how these motion-control technologies are leading to new robotics capabilities, new applications, and entry into new markets.

Register now for the Robotics Summit & Expo, which will be at Boston’s Seaport World Trade Center on June 5-6.

About Kollmorgen

Since its founding in 1916, Kollmorgen’s innovative solutions have brought big ideas to life, kept the world safer, and improved peoples’ lives. Today, its world-class knowledge of motion systems and components, industry-leading quality, and deep expertise in linking and integrating standard and custom products continually delivers breakthrough motion solutions that are unmatched in performance, reliability, and ease of use. This gives machine builders around the world an irrefutable marketplace advantage and provides their customers with ultimate peace of mind.

For more information about Kollmorgen technologies, please visit www.kollmorgen.com or call 1-540-633-3545.

Programming the forces of evolution

The genius of evolution is rarely seen in action, so the invisible hand guiding the direction of biological systems is often taken for granted. However, by applying the principles of natural selection to research questions and designing robots to carry out these tasks, scientists are creating the world's first evolutionary machines.