Auditing YouTube's Recommendation System

for Ideologically Congenial, Extreme, and Problematic Recommendations


Frequently Asked Questions

What are sock puppets?

A sock puppet is an automated browser instance that watches videos on YouTube. The sock puppets are “trained” by watching videos of a specific political ideology. We then “test” the homepage and up-next recommendations of the sock puppet for ideological bias.

Homepage


What are homepage recommendations?

The homepage recommendations are the recommendations that appear on the YouTube homepage. In the image below, the videos within the red border are the homepage recommendations collected from each sock puppet.

Homepage


What are up-next recommendations?

When a user watches a video, YouTube presents them with a series of videos to watch next. These recommendations are known as up-next recommendations and we refer to the currently playing video that triggered them as the “seed video”. In the image below, the video in the blue border is currently playing and is the seed video to the up-next recommendations in the red border.

Up-next


How do you estimate the ideology of a video?

The ideology of a video is estimated by analyzing its sharing patterns on Twitter. Specifically, we collect the tweets mentioning a given YouTube video using the Twitter API and check if the authors of those tweets follow a set of well-recognized partisan elites, i.e., landmarks, whose political ideology is clearly established as either liberal or conservative. Counting the total number of landmarks followed by the authors of those tweets, we estimate the slant on a scale of -1 (most liberal) to +1 (most conservative) as:

Slant = (conservative_landmarks - liberal_landmarks) / (conservative_landmarks + liberal_landmarks)