Neural network helps autonomous car learn to handle the unknown


Autonomous Vehicles

Shelley, Stanford’s autonomous Audi TTS, performs at Thunderhill Raceway Park. (Credit: Kurt Hickman)

Researchers at Stanford University have developed a new way of controlling autonomous cars that integrates prior driving experiences – a system that will help the cars perform more safely in extreme and unknown circumstances. Tested at the limits of friction on a racetrack using Niki, Stanford’s autonomous Volkswagen GTI, and Shelley, Stanford’s autonomous Audi TTS, the system performed about as well as an existing autonomous control system and an experienced racecar driver.

“Our work is motivated by safety, and we want autonomous vehicles to work in many scenarios, from normal driving on high-friction asphalt to fast, low-friction driving in ice and snow,” said Nathan Spielberg, a graduate student in mechanical engineering at Stanford and lead author of the paper about this research, published March 27 in Science Robotics. “We want our algorithms to be as good as the best skilled drivers—and, hopefully, better.”

While current autonomous cars might rely on in-the-moment evaluations of their environment, the control system these researchers designed incorporates data from recent maneuvers and past driving experiences – including trips Niki took around an icy test track near the Arctic Circle. Its ability to learn from the past could prove particularly powerful, given the abundance of autonomous car data researchers are producing in the process of developing these vehicles.

Physics and learning with a neural network

Control systems for autonomous cars need access to information about the available road-tire friction. This information dictates the limits of how hard the car can brake, accelerate and steer in order to stay on the road in critical emergency scenarios. If engineers want to safely push an autonomous car to its limits, such as having it plan an emergency maneuver on ice, they have to provide it with details, like the road-tire friction, in advance. This is difficult in the real world where friction is variable and often is difficult to predict.

To develop a more flexible, responsive control system, the researchers built a neural network that integrates data from past driving experiences at Thunderhill Raceway in Willows, California, and a winter test facility with foundational knowledge provided by 200,000 physics-based trajectories.

This video above shows the neural network controller implemented on an automated autonomous Volkswagen GTI tested at the limits of handling (the ability of a vehicle to maneuver a track or road without skidding out of control) at Thunderhill Raceway.

“With the techniques available today, you often have to choose between data-driven methods and approaches grounded in fundamental physics,” said J. Christian Gerdes, professor of mechanical engineering and senior author of the paper. “We think the path forward is to blend these approaches in order to harness their individual strengths. Physics can provide insight into structuring and validating neural network models that, in turn, can leverage massive amounts of data.”

The group ran comparison tests for their new system at Thunderhill Raceway. First, Shelley sped around controlled by the physics-based autonomous system, pre-loaded with set information about the course and conditions. When compared on the same course during 10 consecutive trials, Shelley and a skilled amateur driver generated comparable lap times. Then, the researchers loaded Niki with their new neural network system. The car performed similarly running both the learned and physics-based systems, even though the neural network lacked explicit information about road friction.

In simulated tests, the neural network system outperformed the physics-based system in both high-friction and low-friction scenarios. It did particularly well in scenarios that mixed those two conditions.

Simple feedforward-feedback control structure used for path tracking on an automated vehicle. (Credit: Stanford University)

An abundance of data

The results were encouraging, but the researchers stress that their neural network system does not perform well in conditions outside the ones it has experienced. They say as autonomous cars generate additional data to train their network, the cars should be able to handle a wider range of conditions.

“With so many self-driving cars on the roads and in development, there is an abundance of data being generated from all kinds of driving scenarios,” Spielberg said. “We wanted to build a neural network because there should be some way to make use of that data. If we can develop vehicles that have seen thousands of times more interactions than we have, we can hopefully make them safer.”

Editor’s Note: This article was republished from Stanford University.

The post Neural network helps autonomous car learn to handle the unknown appeared first on The Robot Report.

Robotics investments recap: March 2019

CloudMinds was among the robotics companies receiving funding in March 2019.

CloudMinds was among the robotics companies receiving funding in March 2019. Source: CloudMinds

Investments in robots, autonomous vehicles, and related systems totaled at least $1.3 billion in March 2019, down from $4.3 billion in February. On the other hand, automation companies reported $7.8 billion in mergers and acquisitions last month. While that may represent a slowdown, note that many businesses did not specify the amounts involved in their transactions, of which there were at least 58 in March.

Self-driving cars and trucks — including machine learning and sensor technologies — continued to receive significant funding. Although Lyft’s initial public offering was not directly related to autonomous vehicles, it illustrates the investments flowing for transportation.

Other use cases represented in March 2019 included surgical robotics, industrial automation, and service robots. See the table below, which lists amounts in millions of dollars where they were available:

CompanyAmt. (M$)TypeLead investor, partner, acquirerDateTechnology
Airbiquity15investmentDenso Corp., Toyota Motor Corp., Toyota Tsushu Corp.March 12, 2019connected vehicles
AROMA BIT Inc.2.2Series ASony Innovation FundMarch 3, 2019olofactory sensors
AtomRobotSeries B1Y&R CapitalMarch 5, 2019industrial automation
Automata7.4Series AABB March 19, 2019robot arm
Avidbots23.6Series BTrue VenturesMarch 21, 2019commercial floor cleaning
BoranetSeries AGobi PartnersMarch 6, 2019IIoT, machine vision
Broadmann1711Series AOurCrowdMarch 6, 2019deep learning, autonomous vehicles
Cloudminds300investmentSoftBank Vision FundMarch 26, 2019service robots
Corindus4.8private placementMarch 12, 2019surgical robot
Determined AI11Series AGV (Google Ventures)March 13, 2019AI, deep learning
Emergen Group29Series BQiming Venture PartnersMarch 13, 2019industrial automation
Fabu Technologypre-Series AQingsong FundMarch 1, 2019autonomous vehicles
FortnarecapitalizationThomas H. Lee PArtners LPMarch 27, 2019materlais handling
ForwardX14.95Series BHupang Licheng FundMarch 21, 2019autonomous mobile robots
Gaussian Robotics14.9Series BGrand Flight InvestmentMarch 20, 2019cleaning
Hangzhou Guochen Robot Technology15Series AHongcheng Capital, Yingshi Fund (YS Investment) March 13, 2019robotics R&D
Hangzhou Jimu Technology Co.Series BFlyfot VenturesMarch 6, 2019autonomous vehicles
InnerSpace3.2seedBDC Capital's Women in Technology FundMarch 26, 2019IoT
Innoviz Technologies132Series CChina Merchants Capital, Shenzhen Capital Group, New Alliance CapitalMarch 26, 2019lidar
Intelligent MarkinginvestmentBenjamin CapitalMarch 6, 2019autonomous robots for marking sports fields
Kaarta Inc.6.5Series AGreenSoil Building Innovation FundMarch 21, 2019lidar mapping
Kolmostar Inc.10Series AMarch 5, 2019positioning technology
Linear Labs4.5seedScience Inc., Kindred VenturesMarch 26, 2019motors
MELCO Factory Automation Philippines Inc.2.38new divisionMitsubishi Electric Corp.March 12, 2019industrial automation
Monet Technologies4.51joint ventureHonda Motor Co., Hino Motors Ltd., SoftBank Corp., Toyota Motor CorpMarch 28, 2019self-driving cars
Ouster60investmentRunway Growth Capital, Silicon Valley BankMarch 25, 2019lidar
Pickle Robot Co.3.5equity saleMarch 4, 2019loading robot
Preteckt2seedLos Olas Venture CapitalMarch 26, 2019machine learning automotive
Radar16investmentSound Ventures, NTT Docomo Ventures, Align Ventures, Beanstalk Ventures, Colle Capital, Founders Fund Pathfinder, Novel TMTMarch 28, 2019RFID inventory management
Revvo (IntelliTire)4Series ANorwest Venture PartnersMarch 26, 2019smart tires
Shanghai Changren Information Technology14.89Series AMarch 15, 2019Xiaobao healthcare robot
TakeOff Technologies Inc.equity saleMarch 26, 2019grocery robots
TartanSense2seedOmnivore, Blume Ventures, BEENEXTMarch 11, 2019weeding robot
Teraki2.3investmentHorizon Ventures, American Family VenturesMarch 27, 2019AI, automotive electronics
Think Surgical134investmentMarch 11, 2019surgical robot
Titan Medical25IPOMarch 22, 2019surgical robotics
TMiRobSeries B+Shanghai Zhangjiang Torch Venture Capital March 26, 2019hospital robot
TOYO Automation Co.investmentYamaha Motor Co.March 20, 2019actuators
UbtechinvestmentLiangjiang CapitalMarch 6, 2019humanoid
Vintra4.8investmentBonfire Ventures, Vertex Ventures, London Venture PartnersMarch 11, 2019machine vision
Vtrus2.9investmentMarch 8, 2019drone inspection
Weltmeister Motor450Series CBaidu Inc.March 11, 2019self-driving cars

And here are the mergers and acquisitions:

March 2019 robotics acquisitions

CompanyAmt. (M$)AcquirerDateTechnology
Accelerated DynamicsAnimal Dynamics3/8/2019AI, drone swarms
Astori AS4Subsea3/19/2019undersea control systems
BrainlabSmith & Nephew3/12/2019surgical robot
Figure Eight175Appen Ltd.3/10/2019AI, machine learning
Floating Point FXCycloMedia3/7/2019machine vision, 3D modeling
Florida Turbine Technologies60Kratos Defense and Security Solutions3/1/2019drones
Infinity Augmented RealityAlibaba Group Holding Ltd.3/21/2019AR, machine vision
Integrated Device Technology Inc.6700Renesas3/30/2019self-driving vehicle processors
MedineeringBrainlab3/20/2019surgical
Modern Robotics Inc.0.97Boxlight Corp.3/14/2019STEM
OMNI Orthopaedics Inc.Corin Group3/6/2019surgical robotics
OrthoSpace Ltd.220Stryker Corp.3/14/2019surgical robotics
Osiris Therapeutics660Smith & Nephew3/12/2019surgical robotics
Restoration Robotics Inc.21Venus Concept Ltd.3/15/2019surgical robotics
Sofar Ocean Technologies7Spoondrift, OpenROV3/28/2019underwater drones, sensors
Torc Robotics Inc.Daimler Trucks and Buses Holding Inc.3/29/2019driverless truck software

Surgical robots make the cut

One of the largest transactions reported in March 2019 was Smith & Nephew’s purchase of Osiris Therapeutics for $660 million. However, some Osiris shareholders are suing to block the acquisition because they believe the price that U.K.-based Smith & Nephew is offering is too low. The shareholders’ confidence reflects a hot healthcare robotics space, where capital, consolidation, and chasing new applications are driving factors.

In the meantime, Stryker Corp. bought sports medicine provider OrthoSpace Ltd. for $220 million. The market for sports medicine will experience a compound annual growth rate of 8.9% between now and 2023, predicts Market Research Future.

Freemont, Calif.-based Think Surgical raised $134 million for its robot-assisted orthopedic surgical device, and Titan Medical closed a $25 million public offering last month.

Venus Concept Ltd. merged with hair-implant provider Restoration Robotics for $21 million, and Shanghai Changren Information Technology raised Series A funding of $14.89 million for its Xiaobao healthcare robot.

Corindus Vascular Robotics Inc. added $5 million to the $15 million it had raised the month before. Brainlab acquired Medineering and was itself acquired by Smith & Nephew.

Driving toward automation in March 2019

Aside from Lyft, the biggest reported transportation robotics transaction in March 2019 was Renesas’ completion of its $6.7 billion purchase of Integrated Device Technology Inc. for its self-driving car chips.

The next biggest deal was Weltmeister Motor’s $450 million Series C, in which Baidu Inc. participated.

Lidar also got some support, with Innoviz Technologies raising $132 million in a Series C round, and Ouster raising $60 million. In a prime example of how driverless technology is “paying a peace dividend” to other applications, Google parent Alphabet’s Waymo unit offered its custom lidar sensors to robotics, security, and agricultural companies.

Automakers recognize the need for 3-D modeling, sensors, and software for autonomous vehicles to navigate safely and accurately. A Daimler unit acquired Torc Robotics Inc., which is working on driverless trucks, and CycloMedia acquired machine vision firm Floating Point FX. The amounts were not specified.

Speaking of machine learning, Appen Ltd. acquired dataset annotation company Figure Eight for $175 million, with an possible $125 million more based on 2019 performance. Denso Corp. and Toyota Motor Corp. contributed $15 million to Airbiquity, which is working on connected vehicles.

Service robots clean up

From retail to cleaning and customer service, the combination of improving human-machine interactions, ongoing staffing turnover and shortages, and companies with round-the-clock operations has contributed to investor interest.

The SoftBank Vision Fund participated in a $300 million round for CloudMinds. The Chinese AI and robotics company’s XR-1 is a humanoid service robot, and it also makes security robots and connects robots to the cloud.

According to its filing with the U.S. Securities and Exchange Commission, TakeOff Technologies Inc. raised an unspecified amount for its grocery robots, an area that many observers expect to grow as consumers become more accustomed to getting home deliveries.

On the cleaning side, Avidbots raised $23.6 million in Series B, led by True Ventures. Gaussian Robotics’ Series B was $14.9 million, with participation from Grand Flight Investment.

Robotics Summit & Expo 2019 logoKeynotes | Speakers | Exhibitors | Register

Wrapping up Q1 2019

China’s efforts to develop its domestic robotics industry continued, as Emergen Group’s $29 million Series B round was the largest reported investment in industrial automation last month.

Hangzhou Guochen Robot Technology raised $15 million in Series A funding for robotics research and development and integration.

That was followed by ABB’s participation in Series A funding of $7.4 million for Automata, which makes a small collaborative robot arm named Ava. Mitsubishi Electric Corp. said it’s spending $2.38 million to set up a new company, MELCO Factory Automation Philippines Inc., because it expects to grow its business there to $30 million by 2026.

Data startup Spopondrift and underwater drone maker OpenROV merged to form Sofar Ocean Technologies. The new San Francisco company also announced a Series A round of $7 million. Also, 4Subsea acquired underwater control systems maker Astori AS.

In the aerial drone space, Kratos Defense and Security Solutions acquired Florida Turbine Technologies for $60 million, and Vtrus raised $2.9 million for commercializing drone inspections. Kaarta Inc., which makes a lidar for indoor mapping, raised $6.5 million.

The Robot Report broke the news of Aria Insights, formerly known as CyPhy Works, shutting down in March 2019.


Editors Note: What defines robotics investments? The answer to this simple question is central in any attempt to quantify robotics investments with some degree of rigor. To make investment analyses consistent, repeatable, and valuable, it is critical to wring out as much subjectivity as possible during the evaluation process. This begins with a definition of terms and a description of assumptions.

Investors and Investing
Investment should come from venture capital firms, corporate investment groups, angel investors, and other sources. Friends-and-family investments, government/non-governmental agency grants, and crowd-sourced funding are excluded.

Robotics and Intelligent Systems Companies
Robotics companies must generate or expect to generate revenue from the production of robotics products (that sense, think, and act in the physical world), hardware or software subsystems and enabling technologies for robots, or services supporting robotics devices. For this analysis, autonomous vehicles (including technologies that support autonomous driving) and drones are considered robots, while 3D printers, CNC systems, and various types of “hard” automation are not.

Companies that are “robotic” in name only, or use the term “robot” to describe products and services that that do not enable or support devices acting in the physical world, are excluded. For example, this includes “software robots” and robotic process automation. Many firms have multiple locations in different countries. Company locations given in the analysis are based on the publicly listed headquarters in legal documents, press releases, etc.

Verification
Funding information is collected from a number of public and private sources. These include press releases from corporations and investment groups, corporate briefings, and association and industry publications. In addition, information comes from sessions at conferences and seminars, as well as during private interviews with industry representatives, investors, and others. Unverifiable investments are excluded.

The post Robotics investments recap: March 2019 appeared first on The Robot Report.

MapLite enables autonomous vehicles to navigate unmapped roads

Navigating roads less traveled in self-driving cars is a difficult task. One reason is that there aren’t many places where self-driving cars can actually drive. Companies like Google only test their fleets in major cities where they’ve spent countless hours meticulously labeling the exact 3-D positions of lanes, curbs, off-ramps, and stop signs. “The cars…

The post MapLite enables autonomous vehicles to navigate unmapped roads appeared first on The Robot Report.

9 Self-Driving Trucks Disrupting Logistics

The trucking industry is heading for an autonomous future. Self-driving trucks have been in the works for several years – Daimler introduced in 2014 the Mercedes-Benz Future Truck 2025 – but developments within the last year show the transformation is moving full steam ahead. The tests being done currently are limited to semi-autonomous systems that…

The post 9 Self-Driving Trucks Disrupting Logistics appeared first on The Robot Report.

Self-Driving Cars Have Power Consumption Problems

I recently chaired a UJA Tech Talk on “The Future Of Autonomous Cars” with former General Motors Vice-Chairman Steve Girsky. The auto executive enthusiastically shared his vision for the next 15-25 years of driving – a congestion-free world of automated wheeled capsules zipping commuters to and from work. Girsky stated that connected cars with safety…

The post Self-Driving Cars Have Power Consumption Problems appeared first on The Robot Report.