Back in the day when I used to work at a particular social media company I saw an internal report describing how ranking, or the process of changing the order of content in user feeds, was provably a good thing. This was in the face of both public pressure from scholars and commentators describing how content ranking algorithms created bias and opportunities for controversial, divisive content to go viral and other kinds of bad things.

The authors of the report had run a randomized experiment turning off ranking for a small portion of users for a few weeks. They showed that these users spent less time using the social media platform, scrolled past more stories, engaged less with meaningful content, and a whole host of other “negative” effects. Putting aside the obvious conflict of interest of some of these measurements (as in, maybe less time spent is actually a good thing, societally speaking), this is a great example of the research design not taking into account equilibrium effects.

For years the existence of the ranking algorithm has influenced both the types of content produced both by content creators and by the social media company itself. Turning off the ranking algorithm for a small subset of users for just a couple weeks is not a good hint at what the world would look like in the absence of a ranking algorithm.

If you have a ranking algorithm, there’s no penalty for just firehosing tons of content into the ether, relying on the algorithm to pick out the cream of the crop to bring to the top. If you turn off the algorithm, everyone will hate you because they are suddenly inundated with terrible content. But in a world that never had an algorithm in the first place, it’s unlikely that so much mediocre content would exist in the first place.