Recently at Hippo, a client of ours approached us with a fascinating foodie problem. Although, strictly speaking, that’s skipping a few steps.
They were for support in developing their loyalty platform, better integrating it with the company’s other services and helping the company to further extend its user base.
The interesting problem came in the form of the platform’s recommendation engine, which had not been properly revisited for several years, while the platform itself had undergone a period of intensive development.
Our investigations into how the current recommender was performing revealed a clear hurdle to overcome: a significant proportion of customers’ first, and in some cases only, points redemptions was for one specific snack food.
Which meant that very little other than a specific brand’s range of snack foods was being recommended for future rewards. There was meanwhile a clear favourite among that brand’s snack food range.
And thus, a seemingly perpetual cycle of repeat snackage was born.
Enter our hungry Hippos
First on our agenda was to do some more digging. We came to understand that our client’s most loyal, long standing customers were demonstrating some clear earning and spending behaviours, beyond the snack cycle.
There were some clear product category and brand preferences, but with other factors at play, such as account age, which of the client’s services was the customer’s entry point, and their current points balance.
The key issue remained that the majority of customers were acquired in the past year or two, and hadn’t yet generated the wealth of data we typically would like for a complex recommender approach, such as collaborative filtering.
However, while the client continues to grow their portfolio of products and services that earn customers points, as well as those customers can be rewarded by, they need a highly-performing recommendation engine to support them – so it was not a case of waiting for more data.
Our other priorities were to introduce a system that could scale and be iterated upon as the company grew. Down the line, this might mean that parts of the process are retired or swapped out, that new data points are added, or that different algorithms become more appropriate.
It needed to be efficient, not introducing excessive data costs, or time consuming processes causing bottlenecks elsewhere. It also needed to be maintainable by a small team of Data Scientists, so the fewer steps and lines of code the better.
Serving up some (non-food related) recommendations
The ‘sparse-in-places’ data we did have lent itself well to one possible methodology for the recommendation engine (of which there are quite a few).
The approach involves forming clusters from customer purchases and recommending relevant products to those that have behaved similarly in the past.
In our snack-based case, these behaviours cover everything from long-standing, points-rich customers able to do their full Christmas shop on points, right through to brand new customers who have just redeemed their very first, much-loved snack.
After evaluating a few possible methodologies, we selected the Non-Negative Matrix Factorisation (NMF) algorithm to build our clusters. NMF worked well for our dataset (example below), for a few reasons:
- Our dataset was inherently non-negative (customers had either transacted against products or categories, or they hadn’t – and we were able to use transaction counts to make clusters even clearer)
- We had plenty of customers who hadn’t transacted against particular products or categories of products. NMF deals well with these gaps.
- The clusters drawn out were meaningful and interpretable – we could easily label them in a way that made sense to humans (particularly, stakeholders).
A simplified example of our dataset
| Product Category 1 | Product Category 2 | Product Category 3 | Price Category 1 | Price Category 2 | |
| Customer 1 | 2 | 0 | 0 | 1 | 1 |
| Customer 2 | 9 | 2 | 1 | 5 | 2 |
We also opted to cluster customers based on the categories of products purchased, rather than the products themselves. We could then rotate the recommended products to each cluster, prioritising the categories of interest.
Given the patterns we’d seen around product favouritism (that favourite snack that no-one seemed to want to pass on) this allowed us a level of flexibility.
By that, I mean we were able to tweak the recommendations provided based on these categories, and assign a few into the mix that were either aspirational points-wise, or slightly out of the customer’s comfort zone (so that we might break the snack food cycle).
We could also increase the likelihood that products on a time-sensitive deal would be recommended to those likely to transact.
Given that the process so far was also relatively light on computational cost (and therefore time for the code to run), the recommendations provided did not necessarily have to be the same for all customers within a cluster.
Their cluster could simply inform us as to how many recommendations should come from each category. And that in turn opens up an opportunity to test different approaches within the same cluster.
The takeaway – and did we deliver?
There is no step-by-step guide to building the perfect recommendation engine. We listened to our client’s needs, their strategy for growth and change – and we reacted accordingly.
We also took some time to truly understand the data we were working with. These aren’t ones and zeros, after all; they are people looking for a good deal, a treat, a reward for their loyalty.
In this crucial stage in our client’s development, proposing a method that works for the current context, but provides the foundations for something bigger later on absolutely felt like the right thing to do.
And hopefully soon, deal-seekers all over will be tempted to try one of the huge variety of rewards on offer on our client’s platform – not just a simple snack (as excellent and well-loved as it is).
