Lifts and Gains

Models are created to predict or classify. So one common way to assess a model’s performance
is to compare its performance to the results expected if no model was used. We can assess the
value of a predictive model by using the model to rank or score a set of customers and then
contacting or targeting them in that order.

Lifts and gains are commonly used performance measures. Lift indicates how much better a
model performs than the ‘no model’ or average performance. For example, assume that Meals à
la Minute wanted to assess the effectiveness of targeting its customer list based on recency of
last purchase. A sample of 50,000 customers is randomly selected from the customer list and
emailed the offer. After all the orders have been received, the analysis begins. The 50,000
customers are divided into deciles with the most recent customers in decile 1 and the least recent
customers in decile 10.

To show how lift is calculated, consider the results in Exhibit 1 that summarize the number of
customers and number of buyers by recency decile for the Meals à la Minute test involving the
offer to purchase the White Truffle Risotto meal. Exhibit 1 reports:

 Recency Decile: Note there are nine rather than ten deciles as a result of large numbers
of customers having the same value for weeks since last purchase close to the ‘dividing
line’ between deciles.
 # Customers: The number of customers in that decile
 # Buyers: The number of customers who bought the White Truffle Risotto meal

Exhibit 1: Recency Decile Summary

Recency Decile # Customers # Buyers

1 (top) 3748 670
2 7424 1058
3 3820 459
4 6254 638
5 6158 521
7 6229 474
8 6184 389
9 5346 203
10 (bottom) 4837 110
Total 50000 4522

T h i s d o c u m e n t i s a u t h o r i z e d
A E M 4 0 1 5 / N B A 6 3 4 0 : C u s t o
A n y u n a u t h o r i z e d u s e o r r
1. Lift and Cumulative Lift

From these raw numbers, we can compute the following as shown in Exhibit 2:

 Cumulative # customers: The number of total customers up to and including that decile

 Cumulative % customers: The percent of total customers up to and including that decile

 Cumulative # buyers: The number of buyers up to and including that decile

 Response rate (%): The actual response rate for each decile, computed by the number of
buyers divided by the number of customers for each decile

 Lift: (Response rate for each decile) ÷ (Overall response rate) × 100

 Cumulative response rate (%): (Cumulative # buyers) ÷ (Cumulative # customers)

 Cumulative lift: (Cumulative response rate) ÷ (Overall response rate) × 100

Exhibit 2: Lift Calculations

# Cum. # Cum. % # Cum. # Res. Cum. Cum.

Decile Cust. Cust. Cust. Buyers Buyers Rate (%) Lift Res. Rate (%) Lift
1(top) 3748 3748 7.5 670 670 17.88 198 17.88% 198
2 7424 11172 22.3 1058 1728 14.25 158 15.47% 171
3 3820 14992 30.0 459 2187 12.02 133 14.59% 161
4 6254 21246 42.5 638 2825 10.20 113 13.30% 147
5 6158 27404 54.8 521 3346 8.46 94 12.21% 135
7 6229 33633 67.3 474 3820 7.61 84 11.36% 126
8 6184 39817 79.6 389 4209 6.29 70 10.57% 117
9 5346 45163 90.3 203 4412 3.80 42 9.77% 108
10 4837 50000 100.0 110 4522 2.27 25 9.04% 100
Total 50000 4522 9.04

For example, below are the calculations leading to the numbers for decile 2:

 Cumulative # customers = 3,748 + 7,424 = 11,172

 Cumulative % customers = 11,172 ÷ 50,000 = 0.223, or 22.3%

 Cumulative # buyers = 670 + 1,058 = 1,728

 Response rate = 1,058 ÷ 7,424 = 0.1424, or 14.24%

 Lift = 14.25 ÷ 9.04 × 100 = 158

 Cumulative response rate = 1,728 ÷ 11,172 = 0.1547 or 15.47%

 Cumulative lift = 15.47 ÷ 9.04 × 100 = 171

Lift is an index that indicates the model’s ability to beat the ‘no model’ case or average
performance. For example, from Exhibit 2 we see that the lift for the top decile is 198. This
indicates that by targeting only these customers we would expect to yield 1.98 times the number
of buyers found by randomly emailing the same number of customers. In contrast, the last decile
(decile 10) has only one-quarter (0.25 times) the number of buyers as one would expect in a
random sample of the same size.
2

This document is authorized for use by Julia DeNardis, from 2/8/2021 to 2/26/2021, in the course:
AEM 4015 / NBA 6340: Customer Analytics & Strategy - Park (Spring 2021), Cornell University.
Any unauthorized use or reproduction of this document is strictly prohibited*.
From the cumulative lift column, we see that by targeting the top two deciles, we would expect to
yield 1.71 times the number of buyers as compared with a random emailing. As a larger percent
of the customers are included, cumulative lift will decrease, reaching 100 (or average response)
when 100% of customers are included.

Lift indices that exceed 100 indicate better than average performance or response, whereas lift
indices less than 100 indicate poorer than average performance or response. Note that lift is a
relative index – a lift of 400 could refer to a predicted 8% response rate or a predicted 80%
response rate – depending on whether the overall or average response rate is 2% or 20%. A
chart depicting the cumulative lift is shown in Exhibit 3.

Exhibit 3: Cumulative Lift Chart

Note that most standard spreadsheet or statistical programs contain options to create line charts
and scatter plots. Though the results may look similar, there is a key difference between these
two. A scatter plot is appropriate for plotting two metric variables (such as sales and profits). A
line chart is used to plot one metric variable (e.g., sales) against a categorical variable (e.g.,
week). With a line chart, the values for the categorical variable are equally spaced along the
horizontal axis. With a scatter plot, the values for the horizontal axis variable are scaled by their
value. So, if you use a line chart with two metric variables, the values for the x-axis will be equally
spaced regardless of their actual value. For lift charts (cumulative lift on the y-axis versus
cumulative % of customers on the x-axis) and gains charts (cumulative gains on the y-axis versus
cumulative % of customers on the x-axis) a scatter plot is the appropriate choice.

T h i s d o c u m e n t i s a u t h o r i z e d
A E M 4 0 1 5 / N B A 6 3 4 0 : C u s t o
A n y u n a u t h o r i z e d u s e o r r
2. Gains and Cumulative Gains

A different way to summarize a model’s performance is with gains and cumulative gains. Again,
we begin with the raw numbers in Exhibit 1 and create the following shown in Exhibit 4:

 Gains (%): The proportion of responders in each decile

 Cumulative gains (%): The proportion of responders up to and including the decile, or
simply the sum of the gains up to that decile

Exhibit 4: Gains and Cumulative Gains

# Cum. # Cum. % # Cum. # Cum.

Decile Cust. Cust. Cust. Buyers Buyers Gains (%) Gains (%)
1(top) 3748 3748 7.5 670 670 15 15
2 7424 11172 22.3 1058 1728 23 38
3 3820 14992 30.0 459 2187 10 48
4 6254 21246 42.5 638 2825 14 62
5 6158 27404 54.8 521 3346 12 74
7 6229 33633 67.3 474 3820 10 84
8 6184 39817 79.6 389 4209 9 93
9 5346 45163 90.3 203 4412 4 98
10 4837 50000 100.0 110 4522 2 100
Total 50000 4522

For example, below are the calculations leading to the numbers for decile 2:

 Gains = 1,058 ÷ 4,522 = 0.23, or 23%

 Cumulative gains = 1,728 ÷ 4,522 = 0.38 or 38%

The cumulative gains chart in Exhibit 5 is a useful visual representation for comparing a model to
the ‘no model’ case or average performance. All models start at the 0-0 point. If 0% of the
customers are emailed or targeted, then we will yield 0% of buyers. Similarly, all models end at
the 100-100 point. If 100% of the customers are targeted then we will yield 100% of buyers.

The diagonal line represents the no model or baseline case. For example, if we randomly select
10% to email or target, then we would expect to get 10% of the buyers. Similarly, if we randomly
select 50% to target, then we would expect to get 50% of the buyers, and so on. The cumulative
gains for the model reveal what proportion of responders we can expect to gain from targeting a
specific percent of customers using the model. For example, results of using a recency model to
target customers for the White Truffle Risotto meal show that by targeting the 7.5% most recent
customers, we would gain 15% of total buyers. By targeting the top 22.3% most recent
customers, we would gain 38% of customers.

Cumulative gains charts are sometimes known as ‘banana’ charts because of their banana-like
shape. The larger the distance between the model and no model lines (i.e., the fatter the
banana), the stronger or more powerful the model is.

T h i s d o c u m e n t i s a u t h o r i z e d
A E M 4 0 1 5 / N B A 6 3 4 0 : C u s t o
A n y u n a u t h o r i z e d u s e o r r
Exhibit 5: Cumulative Gains Chart

3. Comparing Models

Lifts and gains can also be used to compare two or more alternative models, to track a model’s
performance over time, or to compare a model’s performance on different samples. A cumulative
gains chart comparing the recency model to a model using monetary value for the Meals à la
Minute White Truffle Risotto emailing is shown in Exhibit 6. Clearly, the recency model is a more
powerful predictor of response compared with the monetary model.

Exhibit 6: Cumulative Gains for Recency and Monetary Models

In summary, lift is a relative measure of the effectiveness of a predictive model. It is computed as

the ratio between the results obtained with the model to the results with no model. For a model
5

This document is authorized for use by Julia DeNardis, from 2/8/2021 to 2/26/2021, in the course:
AEM 4015 / NBA 6340: Customer Analytics & Strategy - Park (Spring 2021), Cornell University.
Any unauthorized use or reproduction of this document is strictly prohibited*.
predicting response, lift reveals how much more likely we are to get responders if we use the
model than if we contact a random sample of customers.

For a model predicting response, gains show the percent of total possible responders gained by
targeting a specific percent of the customers scored or ranked by a model. Cumulative lift and
gains charts are useful visual tools for measuring and comparing a model’s performance. Both
charts include a baseline or no model case – the greater the difference between the lift or gains
curve and the baseline, the better the model.

This document is authorized for use by Julia DeNardis, from 2/8/2021 to 2/26/2021, in the course:
AEM 4015 / NBA 6340: Customer Analytics & Strategy - Park (Spring 2021), Cornell University.
Any unauthorized use or reproduction of this document is strictly prohibited*.