Netflix's rise to being the world's primary media streaming service was no fluke. It was based on a complex recipe of data manipulation and emotion that means the company knows what you want to watch even before you know yourself.

According to Netflix’s latest quarterly figures, the media streaming service is racking up almost 2 million new subscribers every month.

The reason for this relentless growth (whether you believe that’s good or bad) goes deeper than having a convenient and reasonably priced service to sell. It goes deeper than its mammoth marketing budget and strategies. And it certainly goes deeper than Netflix’s far-from-complete library.

It is Netflix’s secret sauce of algorithms, big data, and gut instinct that fuel this unstoppable growth. It’s this secret sauce that allows Netflix to not just consistently recommend content that users will (likely) love, but also to fund the creation of that content, confident that it will be a success.

Incredible Amounts of Big Data

It’s no surprise that big data plays a big part in Netflix’s ability to recommend and fund the right content. What is surprising, however, is the kind of data and amount of data that Netflix tracks every time you use the service.

Big Data

According to the official Netflix Tech Blog:

"Each time a member starts to watch a movie or TV episode, a 'view' is created in our data systems and a collection of events describing that view is gathered."

As part of this process, Netflix tracks your "entire viewing history for as long as [you] are subscribed". The system "gathers periodic signals throughout each view to determine whether a member is or isn’t still watching". It also tracks your searches, ratings, geo-location data, device information, browsing behavior, time of day/week that you’re watching, when you decide to ditch a show, to pause, and to fast-forward.

With millions of Netflix users streaming billions of hours of content each month, the amount of data the company collects is bewildering. This data is massively important to the success of the company.

In 2014 GigaOM paraphrased Netflix’s Senior Data Scientist, Mohammad Sabah, saying:

"75 percent of users select movies based on the company’s recommendations, and Netflix wants to make that number even higher."

This viewer data is huge, and it’s imperative to why the service can be so addictive. Combined with the huge range of data stored about each show, it becomes hard to disagree with David Carr’s theory that "Netflix is commissioning original content because it knows what people want before they do" (emphasis my own).

Ever-Improving Algorithms

On its own, data is of little use. As Jason Gilbert wrote; "[Netflix’s] success is based on how well it’s able to choose programming that its viewers like while still being profitable."

Netflix

To do this, Netflix uses algorithms. As Engineering Director, Xavier Amatriain, told Wired:

"[The company has developed] several algorithms, each optimized for a different purpose. In a broad sense, most of our algorithms are based on the assumption that similar viewing patterns represent similar user tastes. We can use the behavior of similar users to infer your preferences."

This focus on viewing patterns is proving far more reliable than looking primarily at the rating you give to a show.

As data about users and content are fed into these machine learning algorithms, viewer behaviors can be matched up with shows that have certain similarities — year of production, cast, director, etc. As we can see from the number of hours of media being streamed on Netflix each day, these algorithms are clearly working. But they are always a work in progress.

The company is constantly running large numbers of A/B tests (allowing user experience and algorithm changes to be rolled out and tested on small sub-sets of users) to iteratively improve each of those algorithms. According to Amatriain these tests "let us try radical ideas or test many approaches at the same time". The primary aim is almost always to improve "member engagement (e.g. hours of play) and retention".

Gut Instinct

In another Netflix Tech Blog post, Xavier Amatriain states:

"The abundance of source data, measurements and associated experiments allow us to operate a data-driven organization. Netflix has embedded this approach into its culture since the company was founded".

Watching Netflix

The idea of shows being manufactured and recommended based solely on data is somewhat disturbing. But the TV industry has always relied heavily on data (often in the form of focus groups and viewer numbers). However, Netflix is taking this quite a few steps further.

That being said, Joris Evers, the company’s director of global corporate communications wanted to ease users' minds. He told the New York Times:

"We don’t get super-involved on the creative side…We hire the right people and give the freedom and budget to do good work. That means that when Seth Rogen and Kristen Wiig are announced as special guests on coming episodes of Arrested Development it is not because a statistical analysis told Netflix to do so."

In other words, the value of big data and algorithms informs Netflix’s decision rather than dictates them. Creative ideas for movies to fund, and shows to license will come thick and fast. Those that feel good, will be subjected to the data. If it looks as if a large enough section of Netflix’s users will be interested, and the decision makers’ gut feeling says the show will be a hit, it’s given a thumbs-up and a large check.

This Recipe Seems to Work

This mix of data, continually improving algorithms, and gut instinct seems to be working for Netflix. So much so, in fact, that the company has the confidence to fund entire series of shows before releasing a pilot episode. Most other broadcasters work the opposite way.

Netflix HQ

Producers and directors can pitch creative ideas to Netflix. If the big data and gut feeling add up, and suggest that the costs can be recouped in terms of new subscribers gained and increased retention, Netflix is able to go all-in. House of Cards is one example, where the company invested $100 million in two seasons without even seeing a pilot episode. And it’s why 2016 will see Netflix producing more original content than most other broadcasters do in several years.

This would not be possible if Netflix was not able to be unnervingly reliable at understanding and predicting what you (or at least most people) would love to watch. Before you even know yourself.

Over to you: Do you find Netflix’s recommendations suit your tastes? If not, try these secret Netflix search codes. And do you feel comfortable having Netflix know this much about your viewing behavior, likes, and dislikes?

Image Credit: young couple by Andrey_Popov via Shutterstock, Server Room by Torkild Retvedt (Flickr)