A tool that’s quietly crept into almost all parts of our lives is a recommendation engine. When we go shopping online or turn on an OTT service, a recommender system is always at hand, easing our job of deciding what next.

For this newsletter, and in keeping with its goal, we shall limit ourselves to talking only about those recommendation engines that use content-based filtering. This kind uses inputs by an individual user or consumer ONLY.

What are recommendation engines?

It’s simple, really. It’s software that tasked to think on your behalf, take the pain of decision-making away, and to throw up “intelligent” offerings based on a user/customer’s past history. Basically, some claim it’s yet another way of making humans lazy, for it allows a machine to do the job of sifting/browsing.

Does it? In today’s world where content has grown manifold, it is impossible for a human to go through those catalogs or lists or reviews to decide which movie to watch next, which book to read or which shoe to buy.

The building and adoption of recommender software has gotten better, faster and more intelligent from its advent about 6 years ago. That’s tied-in with the progress made by artificial intelligence technology. This is one of those products that has the content developer, data analyst, IT engineer and marketer all working collaboratively.

Content-based filtering as compared to say its collaborative counterpart, works on a single user’s interactions and preferences. The offerings thrown up are based on the metadata collected from a user’s history and interactions.

Sharp marketing (and sales) people realized quickly the potential of a recommendation engine in drawing in new business. They understood it could:

• Engage leads and customers faster

• Throw up offers in real time

• Save marketing bucks when implementing the sales funnel

• Allow product bundling and increase the order value

So prevalent has the use of recommendation engines become today that now there are even DIY kits out there that make it fairly easy to make one. There are ‘How To’ articles and YouTube instructional videos with promises to help one make “Netflix-like” recommenders!

While development comes with challenges, implementation has its own. A good recommendation engine must have at least 2 things - accuracy and its ability to cover your entire inventory of content as far as possible. One more thing - it has to be scalable (like almost any other IT offering).

Whatever your business, if you do deploy this kind of software, remember to keep an eye on user experience to track if your recommender is doing its job adequately. Feedback is important.

Is there anything specific about content that you would want me to write on? Add your suggestions on topics for this newsletter in the ‘Comments’ section below. Better still, join our LinkedIn Group, All About Content.

Why can’t some recommendation engines hack it?

Without getting too technical, there are some reasons why a recommender system does not measure up when deployed, or over time.

From the content point of view, it must forever adapt to the fast-paced, ever-changing world of content. It must be robust enough to “cover” ALL content items, and must be constantly updated. It must be scalable to handle billions of requests made near-simultaneously by thousands of users, each of them with a different set of habits. And, any content-based recommendation system is as relevant as the content items that are tagged. In a large e-commerce firm with millions of items in its inventory, this can be quite a challenge. Not to forget the ever-changing moods and even habits of consumers. All of which boils down to the relevancy and recency of your data subsets.

These challenges are why the recommendation engine of your fav OTT service starts feeling jaded. If you are a war movie buff, for example, chances are that the recommender will keep throwing up the same 10-12 war movies time and again. Some of that problem lies with the inventory, of course.

No AI system is perfect and neither are recommender systems. That’s why they all have a constant, “Caution: Men At Work” warning hanging outside their hypothetical doors, an articulation of the philosophy on which machine learning is based.

Image by Terry Bouris from Pixabay