Customer Response, Retention and Valuation Concepts (RFM Model) by Jim Novo

Customer Response, Retention and Valuation Concepts (RFM Model)
by Jim Novo

Jim’s Intro:  Here’s a more complex model using Recency and Frequency to rank the LifeTime Value and likelihood to respond of customers relative to each other.

Have you ever heard somebody refer to his or her customer list as a “file”? If you have, you were probably listening to someone who has been around the catalog block a few times.   Before computers (huh?), catalog companies used to keep all their customer information on
3 x 5 cards.

They’d rifle through this deck of cards to select customers for each mailing, and when a customer placed an order, they would write it on the customer’s card.  These file cards as a group became known as “the customer file,” and even after everything became computerized, the name stuck.

Who cares? It happens that while going through these cards by hand, and writing down orders, the catalog folks began to see patterns emerge.  There was an exchange taking place, and the data was speaking.  What the data said to them, what they heard, were 3 things:

1.  Customers who purchased recently were more likely to buy again versus customers who had not purchased in a while

2.  Customers who purchased frequently were more likely to buy again versus customers who had made just one or two purchases

3.  Customers who had spent the most money in total were more likely to buy again.  The most valuable customers tended to continue to become even more valuable.

So the catalog folks tested this concept, the idea past purchase behavior could predict future results.  First, they ranked all their customers on these 3 attributes, sorting their customer records so that customers who had bought most Recently, most Frequently, and had spent the most Money were at the top.  These customers were labeled “best.”   Customers who had not purchased for a while, had made few purchases, and had spent little money were at the bottom of the list, and these were labeled “worst.”

Then they mailed their catalogs to all the customers, just like they usually do, and tracked how the group of people who ranked highest in the 3 categories above (best) responded to their mailings, and compared this response to the group of people who ranked lowest (worst).  They found a huge difference in response and sales between best and worst customers.  Repeating this test over and over, they found it worked every time!

The group who ranked “best” in the 3 categories above always had higher response rates than the group who ranked “worst.”  It worked so well they cut back on mailing to people who ranked worst, and spent the money saved on mailing more often to the group who ranked best.  And their sales exploded, while their costs remained the same or went down.  They were increasing their marketing efficiency and effectiveness by targeting to the most responsive, highest future value customers.

The Recency, Frequency, Monetary value (RFM) model works everywhere, in virtually every high activity business.  And it works for just about any kind of “action-oriented” behavior you are trying to get a customer to repeat, whether it’s purchases, visits, sign-ups, surveys, games or anything else.  I’m going to use purchases and visits as examples.

A customer who has visited your site Recently (R) and Frequently (F) and created a lot of Monetary Value (M) through purchases is much more likely to visit and buy again.  And, a high Recency / Frequency / Monetary Value (RFM) customer who stops visiting is a customer who is finding alternatives to your site.  It makes sense, doesn’t it?

Customers who have not visited or purchased in a while are less interested in you than customers who have done one of these things recently.  Put Recency, Frequency, and Monetary Value together and you have a pretty good indicator of interest in your site at the customer level.  This is valuable information to for a business to have.

How is RFM implemented?  Customers are ranked based on their R, F, and M characteristics, and assigned a “score” representing this rank.   Assuming the behavior being ranked (purchase, visit) using RFM has economic value, the higher the RFM score, the more profitable the customer is to the business now and in the future.  High RFM customers are most likely to continue to purchase and visit, AND they are most likely to respond to marketing promotions.  The opposite is true for low RFM score customers; they are the least likely to purchase or visit again AND the least likely to respond to promotions.

For these reasons, RFM is closely related to another customer marketing concept: LifeTime
Value
 (LTV).  LTV is the expected net
profit a customer will contribute to your business over the LifeCycle, the period of time a customer remains a customer.  Because of the linkage to LTV and the LifeCycle, RFM techniques can be used as a proxy for the future profitability of a business.

High RFM customers represent future business potential, because the customers are willing and interested in doing business with you, and have high LTV.  Low RFM customers represent dwindling business opportunity, low LTV, and are a flag something needs to be done with those customers to increase their value.

Once you have scored customers using RFM, you will be able to:

  • Decide who to promote to and predict the response rate
  • Optimize promotional discounting by maximizing response rate while reducing overall discount costs
  • Determine which parts of the site or activities attract high value customers and focus on them to increase customer loyalty and profitability

The Drilling Down book teaches RFM customer scoring methods and how to use the scores to create high ROI marketing and site designs.  The software that comes with the book automates the customer scoring process, importing your customer transaction files, creating a customer database, and assigning a score to each customer based on RFM theory.

You will learn the original RFM scoring method and theory, plus the updated and modified version based on my experience using RFM with interactive environments.  This version simplifies customer analysis by emphasizing the visual display of customer behavior to aid in marketing decision making.  You don’t need a Ph.D. to use the Drilling Down method.

For example, the standard approach to RFM analysis is a “snapshot” method, measuring the customer at a point in time.  The methods described in the Drilling Down book modify this approach.  These new methods make use of the RFM parameters over time in unique ways, and are not dependent on purchase dollars (monetary) as the original RFM model is.  The result of using these methods is very high ROI marketing campaigns & site designs.

Want a sample of this method at work?  You can see it applied to the true ROI of ad spending by taking the tutorial Comparing the
Potential Value of Customer Groups
.

Step by step instructions for creating future value and likelihood to respond scores for each customer, and for using these scores to create high ROI marketing campaigns and site designs are in the Drilling Down book.

Jim’s Blog

RFM post by Kevin Hillstrom

I was recently asked to evaluate how a catalog selects names for upcoming mailings. The Executive told me that her vendor asked her company to switch from model-based selections to … are you ready for this … to RFM … prompting me to offer a predictable response.
And I laughed and laughed. What idiots! My goodness. The vendor community is really failing my client base … again.
One problem.
In my arrogance, I forgot the original request – to evaluate how this company should select names for catalog mailings.
So I evaluated models against the RFM strategy.
The RFM strategy performed to within 0.3% of the prior modeling strategy – a modeling strategy that while not outstanding was at least credible.
Why would a 40 year old methodology perform almost as well as a credible regression-based modeling strategy? Several reasons.
  1. The annual repurchase rate of reactivation candidates at the margin is only about 7% in this example. When repurchase rates are low, RFM is competitive.
  2. The organic percentage is about 40%. So if the annual repurchase rate is just 7% at the margin, and 40% of the 7% will happen with/without aid of catalogs, then the effective annual repurchase rate is actually (1 – 0.40)*0.07 = 4.2%. A lower effective annual repurchase rate makes RFM more competitive.
Instead of judging a vendor for using a 40 year old methodology, I should have judged my pre-conceived notion that I am right and a vendor is wrong. I should have let the data make the case for using the methodology.

Read full article on Kevin Hillstrom’s site

RFM Video Series

Customer Segments Part (1 of 3)

An overview of RFM.  Recency, Frequency; Monetary (RFM) continue to form the basis of many behavior analysis variables. Learn about the Direct Marketing basics of RFM in this series of videos.

Watch Out with RFM (2 of 3)

RFM Revisited – learn about the real-world danger in committing too completely to this simple formula.

RFM Secrets (3 of 3)

While Recency Frequency Monetary is fundamental to customer scoring and modeling, yet there is very little literature about exactly how to do it. Here, John lays out simple concepts for building your RFM scores. Near the end, he even hints at some powerful secrets slightly beyond normal RFM.

RFM: Not a substitute for predictive modeling by Kevin Macdonell

RFM: Not a substitute for predictive modeling

Recency, Frequency, Monetary value. The calculation of scores based on these three transactional variables has a lot of sway over the minds of fundraisers, and I just don’t understand why.

It’s one of those concepts that never seems to go away. Everyone wants a simple way to whip up an RFM score. Yet anyone who can do a good job of RFM is probably capable of doing real predictive modeling. The persistence of RFM seems to rest on some misconceptions, which I want to address today.

First, people are under the impression that RFM is cutting-edge. It isn’t. In his book, “Fundraising Analytics: Using Data to Guide Strategy,” Joshua Birkholz points out that RFM is “one of the most common measures of customer value in corporate America.” It’s been around a long time. That alone doesn’t mean it’s invalid — it just isn’t anything new, even in fundraising.

Second, it’s often misconstrued as a predictive tool, and therefore the best way to segment a prospect pool. It’s not. As Josh makes clear in his book, RFM has always been a measure of customer value. It does not take personal affinities into account, nor any non-purchasing activities, he writes.

Note the language. RFM is borrowed from the for-profit world: retail and sales. Again, this doesn’t discredit it, but it does make it inappropriate as the sole or primary tool for prediction. Because it’s purely transactional in nature, all RFM can tell you is that donors will become donors. It CAN’T tell you which non-donors are most likely to be acquired as new donors. The RFM score for a non-donor is always ZERO.

It also can’t tell you which lapsed donors are most likely to be reactivated, or which donors are most likely to be upgraded to higher levels of giving. In the eyes of RFM, one person who gave $50 last year is exactly the same as any other person who gave $50 last year. They’re NOT.

Third, we’re often told that RFM is easy to do. RFM is easy to explain and understand. It’s not necessarily easy to do. Recency and Monetary Value are straightforward, but Frequency requires a number of steps and calculations, and you’re probably not going to do it in Excel. Josh himself says it’s easy to do, but the demonstration in his book requires SPSS. If you’re using a statistics software package such as SPSS and you’ve mastered Frequency, then true predictive modeling is easily within your grasp. Almost all the variables I use in my models are simpler to derive than Frequency.

Is RFM useless? No, but we need to learn not to pick up a hammer when what we really need is a saw. RFM is for ranking existing donors according to their value to your organization, based on their past history of giving. Predictive modeling is for predicting (who knew?), and answering the three hard questions I listed above (acquisition, reactivation, upgrade potential.)

You could, in fact, use both. Your predictive model might identify the top 10% of your donor constituency who are most likely to renew, while your RFM score set will inform you who in that top 10% have the highest potential value to your organization. A matrix with affinity (predictive model) on one axis and value on the other (RFM) would make a powerful tool for, say, segmenting an Annual Giving donor pool for Leadership giving potential. Just focus on the quadrant of donors who have high scores for both affinity and value.

If you want to use RFM in that way (that is, properly), then fill your boots. I recommend Josh Birkholz’s book, because he lays it out very clearly.

The real danger in RFM is that it can become an excuse for not collecting more and better data.

For any institution of higher education, the idea that RFM is the bee’s knees is patently untrue. Institutions with alumni have a trove of variables that are informative about ALL of their constituency, not just those who happen to be donors. Expand that to arts-based nonprofits, and you’ll find member-based constituencies and the very same opportunities to model in a donor/non-donor environment. Neither of these types of institutions should be encouraged to rely exclusively on RFM.

For the rest, who don’t have the data for their constituency but could, the idea that pure donor transaction data is all you need cuts off the chance of doing the work now to get more sophisticated about collecting and organizing data that will pay off in the years ahead.

Kevin’s Blog

Making Your Database Pay Off – Arthur Hughes

Making Your Database Pay Off Using Recency Frequency and Monetary Analysis

The principal obstacle to effective database marketing is the development of profitable strategies for use of the database. It is relatively easy to construct a workable marketing database. Many service bureaus are experienced at this work and can do a very satisfactory job. What the service bureau normally cannot help you with, however, is figuring out how to make your database pay off. These strategies you will have to work out yourself.

Read full article

RFM versus Predictive Modeling by Jim Novo

RFM versus Predictive Modeling

First published 2/21/02

This article was written after this article ran describing how “predictive modeling techniques outperformed Recency-Frequency-Monetary value (RFM) targeting in a back-to-school campaign.”  I received a ton of e-mail asking for an explanation of this confusing claim.

For those of you not well versed in what behavioral modeling is all about, this article provides a look inside and addresses some very Frequently Asked Questions on modeling.

For those looking for some resolution on issues brought up in the DM news article, I decided to just write this response and point all the queries to it (saves much typing!).  Thanks to all the fellow Drillers out there who thought there was something a bit off in this article.

Introduction

Let me make it clear upfront that I don’t know either company involved and am not making any judgments on the way this promotionwas designed or executed.  I do however have a problem with the presentation of the article, especially the opening paragraph – “predictive modeling outperformed RFM” – which at best is very misleading based on the facts provided, and at worst is an intentional obscuring of the facts to push a particular agenda.

The following is my best guess as to what is going on here and why the results ended up as they did based on the facts provided.

RFM as Straw Man?

Think about this campaign: it was a back-to- school promotion.  It’s held at a fixed point in time, happens every year.  The people running the campaign seem to have a lot of experience using RFM, both on the agency and client side.

One thing they should know given the type of promotion and experience of the players is this:  RFM is not a valid scoring approach for at least one segment of the population – heavy cyclical buyers.  These are the folks who are primarily promotional buyers, not “regular customers.”  Given back-to-school is the first major promotion in the retail calendar, it may have been quite some time since these promotional buyers had made a purchase in the promoted categories – perhaps since the after holiday blow-out sale.

Knowing all this, they would certainly be aware RFM scoring would demote this promotional buyer because they are not “Recent.”  So a sub-optimal scenario is set up relative to the usage of RFM scoring.  RFM has at least one hand tied behind its back on this promotion, because some (perhaps many, high volume) known heavy buyers are intentionally excluded.  Under these conditions, it’s not surprising just about any model, including “let’s mail to heavy buyers who bought last year” would beat RFM if you were in fact mailing the entire population in a controlled test.

So let’s look at some possible scenarios to explain the results claimed in this case.

They’re smart, but awful case writers, or

the case was edited and many of the key facts people would want to know excluded

There is no mention of methodology in this case, not even the phrase “controlled test” and there are no ROI comparisons.  To make the statement about “beating RFM” one would expect some shred of evidence besides the top line “spent 2.5 times more per direct mail piece than those chosen through RFM.”  OK, but what was the profit comparison?  How much did the model cost?  Was there discounting, and if so, what about subsidy costs?  Were control groups used to measure subsidy costs?  And on and on.  You get what I mean.  If this group included heavy cyclical buyers, my first question is this: how many of them would have bought anyway without mailing them?  If you don’t know the answer to this question, any claims become suspect.

They’re smart, but not terribly honest

There were 40,000 customers chosen with the model and they “would not have mailed to any of the 40,000 using RFM.”  Yet they mailed 60,000 customers using RFM.  Why?  If the model was so much better at selecting targets, why use RFM at all, and in such a big way?  Clearly, they mailed a lot more people than you need to execute a controlled test.

This is not normally how one would execute this promotion – unless one knew they were working with different populations (one Recent, one not) and used different scoring approaches for each.  If this is the way they did it, that’s smart.  But in no way does it support the statement “predictive modeling outperformed RFM”; different groups were scored differently, and the gig was rigged.  Any claims under this scenario could be assumed to be intentionally designed to mislead a reader, or represent a significant lack of experience on the part of people making this kind of claim.

They’re not as smart as they seem

Serendipity is a wonderful thing and my favorite part of direct marketing.  Yea, it’s all pretty scientific, but sometimes you just get results that you didn’t expect or plan for – one way or the other.  What if they simply said, “Hey, let’s run a model on everyone we didn’t pick with RFM, and see what happens if we mail them.”  Essentially a model test, but with a huge percentage of the population, which is a bit strange if you don’t already have “gut feel.”

In this case, they were not thinking of the heavy cyclical buyers at all, and not thinking of the obvious impact of using RFM scoring on a population “rigged” to fail – they would simply run a model and follow the output.  And it worked very well, because the model teased out a pretty obvious mailing strategy from the customer base (as models frequently do).  They simply were not aware of and had not thought of the implications underlying the results and made an inappropriate comparison.

In fact, look at the parameters of this model they provided us with:

  • customer purchase behavior, such as the average number of months between purchases and the amount spent

Well folks, that’s a Latency model if I’ve ever heard one, and certainly implies this group had a Recency problem, at the very least.  RFM would be rigged to fail under this scenario

  • only a half a percent the model selected were previous junior apparel buyers or previous children’s apparel purchasers

Hard to tell what this means without knowing the full story, but here’s one thought – product history didn’t matter a bit.  These were just buyers who bought whatever, whenever prompted at the right time with the right offer – the classic sign of a discount prone, highly subsidized promotional buyer.

In this scenario, the players are innocent of any intentional malice – but still cannot make any claims about modeling versus RFM.  They intentionally created two populations and scored them differently, and got rewarded for trying something new.  Hey, that’s great!

OK, now that we’ve gone through these examples, let me address some issues on RFM and custom modeling in general.  Hopefully, this information will be of value to people when they are faced with interpreting data and making decisions in the analytics area.

You say Tomato, I say Celery

Let’s talk briefly about populations and target selection.  Those of you who know RFM and response models in general know they are ranking systems.  They rank the likelihood of people to respond to the promotion, from highest likelihood to lowest likelihood.  People at the “top” of the ranking are the very most likely to respond; people at the bottom of the ranking are the very least likely to respond.  Offline, the top 20% of the ranking usually has a response rate from 5 to 40 times higher than the bottom 20% of the ranking.  Online, the difference is even greater.

On any scored population, RFM or customer model, I can select how far down into the ranking to mail.  Do I want to mail the top 10% most likely to respond, the top 20%?  As you include more and more people, the average likelihood to respond drops rapidly.

If you mailed deeply into an RFM scored population, let’s say covering the top 50% of the rankings, and did a very shallow mailing to the custom model population, say covering the top 10% of the rankings, then I have no doubt in my mind you could get the per mailerresults and comparative stats mentioned:

“Names selected using predictive modeling had a four times higher average monthly spending rate…  a three times higher purchase rate… spent 2.5 times more per direct mail piece than those chosen through RFM.”

“Selected” is the operative word here.  If only the best and most likely to respond were selected using the model, but on the RFM side you mailed much more deeply into the scores, including lots of people with lowered likelihood to respond, you end up with a completely self-fulfilling prophesy, not a “predictive model that beats RFM.”  Not even close.

I’m not saying this happened in the article we just looked at.  I’m saying a statement along the lines of “the top 20% most likely to respond groups in both the RFM and custom model populations were selected” is something you always, always look for when you are in this space.  If you have people pitching you any kind of analytics, make sure you are dealing with fair comparisons.  You can make anything look fantastic by fooling with the knobs and levers in the background.

Kissin’ Cousins

Folks, RFM is a predictive model.  It predicts behavior based on past activity; RFM is no different in that respect than a “predictive model” you paid some modeler $50,000 for.  So to make the statement “predictive modeling beat RFM” is just a bit circular in the first place, and one wonders what the intent of making a statement like that could be.  If you said “A Latency model beat a Recency model in a Seasonal Promotion” then I’d have no problem with that at all, but would wonder why it’s a news item.  As explained above, it’s pretty much common sense.

Latency is nothing more than Recency with a twist; instead of counting “days since” using today, you count “days since” using a fixed point in time.  Latency can work much better than Recency when there are external cyclical factors involved – like seasonal promotions.

For example, if you have not filed a tax return Recently, it does not mean you are less likely to file one in the future.  All it means is there is an external cyclical event (April 15th in the US) controlling your behavior.  If you had not filed one in 18 months (18 months Latent), then I would start to question likelihood to file.

The optimum solution is often to use RFM (Recency Frequency Monetary) and LFM (Latency Frequency Monetary) in tandem targeting the appropriate populations, as was (apparently) done in this promotion.  Smart.

Crop dusting with the SST

If you are not doing any data modeling at all, the ROI of implementing an advanced model can be substantial.  But the real question is this: will the improvement gained by using an advanced predictive model be enough to cover the cost of it relative to the improvement gained by using a simple model?

Given that most advanced “response models” like the one in the article use Recency or Latency and Frequency as the primary driving variables, it’s a valid question to ask.  Here’s a dirty little modeling secret: most, if not every “response model” built includes Recency / Latency and Frequency as primary variables, whether created “top down” by a human or “bottom up” by a machine (so called data mining).  The primary difference is this: they add 3rd, 4th, 5th etc. variables which incrementally improve ROI – all else equal.

In other words, RFM is the low hanging fruit, often buying you 10x or 20x response rate improvement.   You want the next 10%?  Get a custom model, and make sure the price you will pay is worth the diminishing returns.

Just because RFM is a simple, easy to implement, standardized predictive model, people pick on it.  They want you to pay through the nose for a “good model” because, my simple friend, you could not possibly do any modeling yourself.  Now, am I saying that RFM is better than a model created by a roomful of modelers?  Of course not.  The question, as always, is this: will it improve your performance enough to cover the modeling cost; what is the ROI?

RFM Slandered – again?

Speaking of picking on RFM, I was wondering what’s up with this statement in the article:

“When working with RFM, you are really only looking at three elements, and you never get to see the rest of the prospects in a database that have other characteristics that could lead them to become buyers in a given area.”  Well, that may be the way they use RFM, but it’s certainly not the only way.

There is no reason you can’t load up on any variable you want with RFM scoring.  Those who have read my book know this approach is fundamental to the Drilling Down method.  RFM is the Swiss Army knife of behavioral models, and can be used in very many ways.  Choosing to use the original, pre-computers, late 1950’s version of RFM is simply that – a choice.  Or you could choose to use a totally bastardized version from who knows where.  Like any tool, you need to really know how to use it to get the most out of it.

I’m a solution, I’m a problem

To conclude, I have nothing against custom models.  I use them when appropriate.  I have nothing against the design or execution of the promotion.  I have a big problem with the way the article was presented, resulting in a claim appearing to lack sufficient backup.

Those of us in the modeling space need to help people understand how behavioral modeling works by presenting clear and clean examples.  Fast and loose “cheerleading” is what got the CRM folks into the mess they are in, and we don’t want Business Intelligence or Customer Analytics or whatever “space” we are in this month to experience the same fate.

If anyone, including the original retail and agency players in the article above have comments on my analysis or in general on this topic, I’d be glad to post them.  Heck, if the players have “the whole” case study available for review, I’ll provide the download link right here.  It was a sweet promotion, really.

But what I want to know is exactly what happened with all the glorious details – so I can learn something from it, or use the stats to confirm what I already know.  And we owe those folks just beginning to get a grip on behavioral modeling the same courtesy.

That’s why we’re all here.  To learn.

Thoughts on RFM Scoring – John Miglautsch

RFM Basics

Direct marketing is fundamentally the scientific control of customer acquisition and contact.  The recurring question is whether Customer A merits an additional contact based on his past purchase behavior. This question applies equally to direct mail, catalog, phone, field or Internet contact.
The process of making this decision is customer segmentation. Not all customers have purchased identical amounts. Some have ordered more often, some have ordered more recently. Consequently, not all customers should be contacted with the same effort and expense. The cornerstone of direct marketing segmentation is RFM (Recency, Frequency and Monetary

values).

Since direct marketing segmentation is a science, it is important to quantify customer behavior so that we can test the short and long term effect of our segmentation formulae. The purpose of RFM is to provide a simple framework for quantifying that customer behavior. Once customers
are assigned RFM behavior scores, they can be grouped into segments and their subsequent profitability analyzed. This profitability analysis then forms the basis for future customer contact frequency decisions.
RFM Scoring
The purpose of RFM scoring is to project future behavior (driving better segmentation decisions). In order to allow projection, it is important to translate the customer behavior into numbers which can be used through time.

Why Use RFM? by John Miglautsch

Why Use RFM?

RFM is short for Recency, Frequency and Monetary. These three variables when applied to a customer file become the backbone of mailing segmentation. Recency is based on when the most recent purchase was made. Frequency relates to the entire number of purchases made in a customers life-to-date. Monetary is the total money spent.

For any given mailer, a very small percentage of the customers spend most of the money. This is best captured by the Pareto Principle (80/20 Rule). 20% of your customers spend 80% of the money. As your customer file ages, it becomes less and less productive to mail all your customers. To identify the best 20% (or whatever proportion of your customer file is productive to mail/contact) we apply scores to your customers, produce reports and allow you to select as deeply as you feel appropriate. As results are generated, mailings become fine tuned with circulation increasing in the best seasons and decreasing when business is traditionally slower. Your circulation plan thus maximizes both sales and profits.

“To be useful, recency, frequency and monetary value must evolve hour by hour, day by day, week by week, month by month, quarter by quarter, and year by year. The value lies in the ability to see change over time; indeed, that may be the only value as it becomes a replicable measurement of consistently improving profitability created from increasingly better decisions.” Donald R. Libey, Author of Libey on Recency, Frequency, and Monetary Value (The Libey New Century Library).

RFM vs. Models

So you’ve made the decision to try statistical segmentation or modeling against your traditional RFM. To cut through all of the hype, you vow to carefully test the new method against the old. But we’re not talking about an A/B split of one list, or measuring one rental against another. You will be comparing two different approaches to your customers. So what is the best way to make this comparison?

The first point is not to get too fine in your mailings. You won’t find anything out about customers you don’t mail. Use a mailing where you would ordinarily go pretty far down into your RFM schema. You also want to mail pretty far into the model. Even without mailing every cell, you might want to take a few thousand from even the lowest cells just to see what would have happened. Again, you don’t want to bet the farm, but if you are mailing most of your customers anyway, then the negative risk is minimal.

The key in the comparison is to find a method which isn’t too cumbersome, is easy to understand and perhaps most important is fair to both techniques. Both RFM and segmentation models attempt to predictively rank customers from good to bad. We need to keep not only the results but their interpretation in mind as we compare. There are three common approaches: random allocation, unique comparison and ID match.

Random allocation involves matching the customers selected by RFM with the customers selected by the model. Two tapes are created, many names happen to be common. Using an exact match (the names came from exactly the same source) the two tapes are compared at your service bureau. The matching names are then randomly reallocated first to one tape, then the next match to the other tape. In this way, each segment retains its original key code. Unfortunately, the segments with the most matches lose some of their original count. At the same time, it is fair to both techniques. Assuming both tapes had about the same total number of names, each had the same number of matches, therefore, we can simply compare total sales and profit and see which technique did the best.

This method preserves the complete original key coding of both tapes. If you don’t have equal quantities you can still easily compare the two results. Assuming you have an expected rank of your RFM cells, it is a good idea to line up results with quantities as the X axis and sales or profit on the Y axis. This graph illustrates the dramatic difference between RFM and modeling results.

Compare Uniques is probably the easiest both to do and to understand. Simply match the two tapes, preserve the RFM key codes for the matched names. Key the unique names of each tape separately. The assumption is that if the new technique finds names RFM would have missed and the names do better, then the model will more than pay for itself. Mailers who care mostly about “beating control” focus on this most common method. In a recent test, our model uniques generated $15 sales per catalog, the RFM uniques generated $1 per book. A great win “right?”

This technique separates the unique names from their original segments. Little insight is gained by the comparison. Further, to the extent that RFM is usually pretty good at finding the best names, there might not be much difference at the top end. The uniques that are compared often lie far down into the moderate or poor customers. Bottom line, this technique tends to destroy analysis of the results. This overly simplistic comparison reduces the subtleties of both RFM and statistical segmentation to a horse race. A few months ago, a client may have manipulated this comparison by first selecting all the 24 month buyers, then they required selection of modeled names far down into the poorer names. The net effect was to force a comparison between their unique RFM names the model missed with the forced uniques way down in the heap.

ID Match requires some computer processing to understand what happened in the mail. Two tapes are created. Customer ID’s and key codes are preserved. When orders come in, they are matched back to each tape. In this way, results actually recreate how both mailings would have done. Most importantly, this technique perfectly preserves both customer rankings and cell counts. Assuming we again compare sales and profit as mail quantities increase, we can lay both techniques side by side as seen in the graph above.

Conclusion

If you are planning to spend the money to tip toe into modeling, make sure you also take the time to plan your testing and evaluation techniques. Comparing uniques might be the most common way to assess payback, but the other two provide a much richer picture of what is really going on in your customer file. Good luck