Recommendation algorithms are easier to understand than their reputation suggests, partly because the companies running them have published a surprising amount about how they work. The published material describes a two stage pipeline, a set of predicted behaviours, and an objective that is measurable rather than meaningful. Once you can see those three parts, most of the odd behaviour of a feed stops being mysterious, and the controls you do have start to make sense.
Updated October 2026. Descriptions are drawn from platform documentation and published research listed in Sources. Systems change and the details differ between products.
How recommendation algorithms work: two stages, not one
The clearest public description comes from the engineers who built YouTube’s system and published it at a research conference. Their paper describes a system comprised of two neural networks, one for candidate generation and one for ranking. Candidate generation takes events from a user’s activity history and retrieves a small subset, hundreds of videos, from a large corpus. Ranking then scores that short list in detail. The paper frames the first stage as extreme multiclass classification across millions of videos, and reports models learning approximately one billion parameters trained on hundreds of billions of examples.
The reason for the split is cost. Scoring millions of items richly for every user is impossible, so the system first throws almost everything away cheaply, then thinks hard about what survives. This is why a feed can feel both uncannily specific and strangely narrow: the expensive stage only ever sees what the cheap stage already selected.
What the system is actually optimising
The objective is whatever can be measured, and the YouTube paper is unusually direct about why that choice is not neutral. Its ranking model predicts expected watch time rather than click probability, because, in the authors’ words, “Ranking by click-through rate often promotes deceptive videos that the user does not complete (clickbait) whereas watch time better captures engagement.” That is both a sensible engineering decision and an admission that the chosen metric decides what the system will promote.
Platform documentation on the social side describes the same structure in less technical language. Instagram says there are thousands of signals, from when a post was shared to whether you are using a phone or the web, and that its recommendation algorithms make roughly a dozen predictions per ranking surface. For Feed, it names the five interactions it looks at most closely: how likely you are to spend a few seconds on a post, comment on it, like it, share it, and tap the profile photo. Meta’s system card for Instagram Feed adds predictions for how much time you are likely to spend and how likely you are to skip. Nowhere in any of that is there a prediction of whether something is true or good for you.
Why it feels like mind reading
Three ordinary things combine. The first is the breadth of the signal set, which includes time of posting, device, and your history with a particular account, not just explicit likes. The second is that recommendations are not limited to accounts you follow: Instagram says Feed considers recent posts from people you follow as well as posts from accounts you do not already follow. The third is selective memory. You remember the eerily apt recommendation and forget the hundreds of misses, which is the same reason horoscopes feel accurate.
Systems also explore deliberately. A model that only ever shows you what it is already confident about learns nothing new, so some slots are spent on items it is uncertain about. That is a large part of why a feed occasionally goes sideways for a day, and it is not evidence that something is listening to you. If you want the search engine version of the same mechanics, see how Google ranking works.
5 simple controls that actually do something
- Use the non-profiling option where it exists. Under Article 38 of the Digital Services Act, very large platforms must offer at least one recommender option that is not based on profiling. It is usually a chronological or following-only view.
- Give explicit negative signals. Not Interested, hide, mute and unfollow are inputs the documentation says are used. Scrolling past in irritation is an input too, and it reads as engagement.
- Curate the follow graph. Your history of interacting with an account is a named signal, so unfollowing and using favourites changes the candidate pool rather than just the ranking.
- Read the transparency text. Article 27 requires platforms to set out the main parameters of their recommender systems in plain and intelligible language, and Article 26 requires clear labelling of advertising and meaningful information about who paid for it.
- Separate deliberate use from grazing. Subscriptions, newsletters and search are pull mechanisms. A feed is a push mechanism optimised for duration, and the two produce different reading habits.
The one sentence worth keeping
A feed is not a list of what exists. It is a ranked prediction of what you will respond to, assembled by throwing away almost everything first. Treat it as a filtered sample rather than a view of the world and most of its strangeness becomes legible. The same caution applies to anything an automated system hands you, which is why checking an AI answer and decluttering your digital life belong in the same habit.
Common questions
How do recommendation algorithms decide what to show? In two stages. A cheap candidate generation step narrows a very large corpus to hundreds of items based on your activity history, then a ranking model scores that short list using predicted interactions such as time spent, likes, comments, shares and skips.
What are they optimising for? Measurable engagement. The published YouTube work predicts expected watch time rather than clicks, and Instagram describes roughly a dozen predictions per surface, naming time spent, comments, likes, shares and profile taps for Feed.
Can I turn personalisation off? Partly. Article 38 of the Digital Services Act requires very large platforms to offer at least one recommender option not based on profiling, which in practice is a chronological or following-only feed.
Does marking something Not Interested work? It is a documented input, and platforms list mute, hide, unfollow and report alongside it. Engaging angrily with content you dislike works against you, because the signal recorded is engagement.
Why does my feed suddenly show something unrelated? Partly deliberate exploration. A system that only shows what it is already confident about cannot learn, so some impressions test uncertain items.
Sources and further reading
Where the figures and rules above come from, so you can check them:
- Two stage architecture, corpus scale and the watch time objective: Covington, Adams and Sargin, Deep Neural Networks for YouTube Recommendations, RecSys 2016
- Thousands of signals, roughly a dozen predictions and the five Feed interactions: Instagram, Ranking Explained
- Inputs, predictions and user controls for Instagram Feed ranking: Meta Transparency Center system card
- Recommender transparency under Article 27 and the non-profiling option under Article 38: European Commission, user rights under the Digital Services Act
Photo credit: Amazon Prime Video Streaming (40845733273) by Ajay Suresh from New York, NY, USA, CC BY 2.0, via Wikimedia Commons.
Join the discussion