Google uses human reviewers to listen to Home assistant recordings, investigation reveals

An investigation into Google’s practices for storing and analyzing audio files recorded via Google Home smart speakers has revealed that the tech giant hires people to systematically listen to recordings with the purpose of improving the technology.

A report from Belgian broadcaster VRT NWS titled “Google employees are eavesdropping, even in your living room” sent shockwaves around the Internet. Even though Google claims not to snoop on people’s conversations, it apparently does so, albeit to a limited extent, to improve its algorithms.

“Google employees are systematically listening to audio files recorded by Google Home smart speakers and the Google Assistant smartphone app,” according to the VRT NWS report. “Throughout the world – so also in Belgium and the Netherlands – people at Google listen to these audio files to improve Google’s search engine. VRT NWS was able to listen to more than a thousand recordings. Most of these recordings were made consciously, but Google also listens to conversations that should never have been recorded, some of which contain sensitive information.”

The publication talked to various people involved with the Google Home project to learn how the technology works. The sources asked to remain anonymous, but did reveal the detailed nature of the recordings:

“Speech recognition automatically generates a script of the recordings. Employees then have to double check to describe the excerpt as accurately as possible: is it a woman’s voice, a man’s voice or a child? What do they say? They write out every cough and every audible comma. These descriptions are constantly improving Google’s search engines, which results in better reactions to commands. One of our sources explains how this works.”

The Belgian outlet was able to access a number of recordings and pick up details like names, addresses and other types of personal information that they could then use to look up people on social media and identify them individually. Contacted by the publication, some of the people agreed to listen to the recordings and confirm it was indeed them in the recordings.

Raising serious ethical issues, in one case a Google contractor revealed he once listened to a recording that clearly involved domestic violence.

“One of our three independent sources says he had to describe a recording where he heard a woman who was in definite distress,” the report said. “What are employees supposed to do with such information? We are told that there are no clear guidelines regarding such cases […] Employees only receive specific directions when it comes to account numbers and passwords. Those are marked as sensitive information.”

All three major sellers of personal assistants — Google, Amazon and Apple — have taken fire recently for the way they store and use people’s data and recordings.

In a blog post likely reacting to the VRT NWS report, Google admits it has recently learned that one of its language reviewers violated the company’s data security policies by leaking confidential Dutch audio data.

“Our Security and Privacy Response teams have been activated on this issue, are investigating, and we will take action,” Google said. “We are conducting a full review of our safeguards in this space to prevent misconduct like this from happening again.”