What the Robot Saw

What the Robot Saw – the endless documentary of a social media robotSkip to content

Does the Internet suck? Or do just the parts we get to see suck?
In the new world order of robots and talking heads, a reality exists that’s part life, part cinema, and part algorithm. As algorithmic curators silently curate the social media world, an alternate reality also exists, starring the humans who win at losing the losing social media game — a secret cinematic world, seen only by robots. Who do social media algorithms render invisible?
‘What the Robot Saw’ is a live, continuously-generated, robot film, curated, analyzed and edited using computer vision, neural networks, and contrarian search algorithms.

^^^Please turn on the sound.^^^ … ‘What the Robot Saw’ is intended for fullscreen viewing if you can…
If the stream’s not live, or to scrub through the live or recent videos, view them on Twitch.
Thrice daily intermissions last four minutes; stream resumes on the hour.

Political trolls and YouTube celebs gain attention because social media algorithms reward and amplify addictive videos. This process reflects back to us a distorted view of culture, which potentially exacerbates divisiveness. It also incentivizes the production of content likely to gain attention, like sensational content and demographically stereotypical content which is likely to gain advertisers. Thus, the algorithms perpetuate a feedback loop between the public and the site that amplifies public perceptions and encourages production of problematic content.

‘What the Robot Saw’ uses contrarian ranking algorithms to constantly curate some of the least attention-grabbing new videos on online media. These videos are rendered largely invisible by commercial social media ranking algorithms — and so may only be seen by the algorithmic robots that rank them. Using computer vision facial and image analysis algorithms to curate videos and study their subjects, ‘What the Robot Saw’ continually assembles its film and identifies its “talking head” performers in a robofantastical cinematic style. It’s a Sunday drive through the awkward intersections of performance, surveillance, voyeurism, and robots in the age of the talking head.

The Robot curates clips from among the least viewed and subscribed YouTube videos uploaded in recent hours — videos that would be buried by YouTube’s engagement-centered ranking algorithms, which favor attention-grabbing content at the expense of less eye-catching, polished, search-optimized, stereotypically-aligned or sensational videos. As the Robot generates its film, it titles its subjects according to Amazon Rekognition’s demographically-obsessed algorithms.

Revealing a mix of underacknowledged media makers, performed selves, and obsessive surveillance algorithms, ‘What the Robot Saw’ enters into the complicated relationship between the world’s surveillant and curatorial AI robots and the humans who are both their subjects and their stars. It’s not about how robots actually see. ‘What the Robot Saw’ is a response to the contemporary collision of performed selves, screen-centric perceptions — and robots.

Humans have a complicated relationship with robots. An invisible audience of software robots continually analyze content on the Internet. Videos by non-“YouTube stars” that algorithms don’t promote to the top of the search rankings or the “recommended” sidebar may be seen by few or no human viewers. In ‘What the Robot Saw,’ the Robot is AI voyeur turned director: classifying and magnifying the online personas of the subjects of a never-ending film. A loose, stream-of-AI-“consciousness” narrative is developed as the Robot drifts through neural network-determined groupings.

Robot meets Resting Bitch Face and Other Adventures. As it makes its way through the film, the Robot adds lower third supered titles. Occasional vaguely poetic section titles are derived from the Robot’s image recognition-based groupings. More often, lower-third supers identify the many “talking heads” who appear in the film. The identifiers – labels like “Confused-Looking Female, age 22-34” – are generated using Amazon Rekognition, a popular commercial face recognition and analysis library. (The Robot precedes descriptions that have somewhat weaker confidence scores with qualifiers like “Arguably.”) The feature set of Rekognition offers a glimpse into how computer vision robots, marketers, and others drawn to off-the-shelf face analysis products often choose to categorize humans: age, gender, and appearance of emotion are key. The prominence of these software features suggest that in a robot-centric world, these attributes might better identify us than our names. (Of course Amazon itself is quite a bit more granular in its identification techniques.) The subjects’ appearance and demeanor on video as fully human beings humorously defy Rekognition’s surveillance and marketing enamored perceptions.

What the Robot Saw’s title is a reference to What the Butler Saw films. Neither the butlers nor the Robot could really understand the objects of their obsession. Despite their self-satisfaction, all they had was a squinted glimpse at a peep show.

The live streams run throughout the day; there are brief “intermissions” every few hours (and as needed for maintenance.) Extensive archives auto-documenting previous streams — and thus a piece of YouTube in realtime — are available on the Videos page.

‘What the Robot Saw’ is a non-commercial project by Amy Alexander
Twitter: @uebergeek