Fully autonomous smart tech can kill us, researchers warn

In a bid to “promote good security practices, and understand vulnerability risk,” the Carnegie-Mellon University’s Software Engineering Institute has conducted a study to look at the potential threats posed by autonomous transport systems, machine learning and smart devices in general. Their forecast is not good.

The Emerging Technology Domains Risk Survey is at its third iteration as part of an effort to aid the US Department of Homeland Security’s US-CERT to “identify vulnerabilities, promote good security practices, and understand vulnerability risk,” the team told The Register.

According to the Institute, intelligent transport systems – soon to be part of the IoT ecosystem – will communicate with each other to an unprecedented extent.

Everything from autonomous cars and trucks, to road infrastructure, bicyclists and even pedestrians will be in the matrix. When that happens, the risks will be sky high, according to the paper.

“A miscommunication in the system, whether accidental or intentional, could lead to numerous traffic accidents, causing property damage, injury, and possibly death … A compromise of a city-wide system could lead to a massive traffic jam or other major event,” the team writes.

Depending on the technologies applied, Intelligent Transport Systems (ITS) vary. ITS can include anything from basic management systems (car navigation, traffic signal control systems, variable message signs, number plate recognition, speed cameras, etc. ), to monitor applications (security CCTV systems), to more advanced implementations that aggregate live data from different sources simultaneously (for things like parking guidance, weather information, bridge de-icing systems, etc.)

On the topic of robots, the team had this to say:

“It is not difficult to imagine the financial, operational, and safety impact of shutting down or modifying the behaviour of manufacturing robots, delivery drones; service-oriented or military humanoid robots; industrial controllers; or … robotic surgeons.”

In other words, even if robots don’t ask for any sick leave or maternity leave, the trade-off is still pretty disconcerting.

Other key takeaways from the report touch cryptocurrency, robotic surgery, virtual personal assistants, and numerous Internet of Things (IoT) applications. Many of the team’s predictions are on par with Tesla CEO Elon Musk’s biggest fears about the future of autonomy and artificial intelligence in particular.

Earlier this year, Musk and other AI and robotics experts addressed an open letter to the United Nations urging the organization to reconsider its plans to deploy automated fighting drones, tanks and machine guns.

A month earlier, a fully autonomous mall security robot in Washington D.C. malfunctioned and was found floating in a public fountain.