| Course | BUS 655 Sports Business Analytics |
|---|---|
| Topic | Topic 7 |
| Paper type | Test plan |
| Length | About 990 words, 5 pages |
| Format | APA 7 student paper |
| School | Grand Canyon University (GCU) |
| Program | MBA |
| Updated | October 2026 |
Free sample paper for BUS 655 Topic 7
Roll It Out or Drop It: A Holdout Test Plan for a Lapsed-Buyer Bundle at a Composite Soccer Club, With the Decision Rule Signed Before the First Email
[Author Name]
Colangelo College of Business, Grand Canyon University
BUS-655: Sports Business Analytics
Topic 7 Assignment
[Instructor Name]
[Date]
The organization, setting and figures below are a composite written as a model document. No real employer, client, colleague or patient is described.
Roll It Out or Drop It: A Holdout Test Plan for a Lapsed-Buyer Bundle at a Composite Soccer Club, With the Decision Rule Signed Before the First Email
The Decision
Riverbend FC's marketing team has argued for weeks about a bundle for lapsed single-game buyers: two tickets, two drinks and a club scarf for $59, against $76 for the two tickets alone at face value. The marketing director says it will win back buyers who drifted away; the ticketing manager says it will mostly discount tickets for people who were coming anyway. Both could be right. This plan turns the argument into a test whose result decides one thing: whether the bundle is offered to the full list of lapsed buyers for the rest of the season, or dropped. All figures are illustrative composites.
Who Is Eligible
Eligible buyers are defined from the CRM as accounts that bought single-game tickets in either of the past two seasons, bought nothing this season, have a valid email address and have not opted out of marketing. Season members and group buyers are excluded, since the bundle is not meant for them. The list contains 6,400 accounts. Because some households hold two accounts, accounts sharing a mailing address are grouped and treated as one unit, leaving 6,050 households.
Randomizing Buyers, Not Matches
The unit randomized is the household. A random 20 percent, 1,210 households, are withheld and receive no bundle offer; the other 4,840 receive the offer by email. Randomizing buyers is the only design that answers the question cleanly. Offering the bundle for some matches and not others would mix the bundle's effect with differences between opponents, dates and weather, and buyers who saw the offer for one match would be influenced in their choice of another. Comparing buyers who used the bundle with buyers who did not would be worse, since people who act on an offer were already the likeliest to buy, which flatters the bundle. Experiments in advertising have repeatedly found that non-randomized comparisons overstate effects, sometimes by a large margin (Gordon et al., 2019).
The Primary Measure
The test does not count bundles sold. A bundle sale to someone who would have paid $76 anyway is a loss of $17 in ticket revenue plus the cost of drinks and a scarf, and a bundle count would record it as a success. The primary measure is incremental margin per contacted household: total margin from all ticket purchases by the offered group over the rest of the season, minus the same for the holdout group, divided by the number of households in each.
Margin per bundle is $59 less about $9 for the drinks and scarf, or $50. Margin on a full-price purchase of two tickets is $76, since the club's other ticketing costs do not change. Secondary measures, reported but not used for the decision, are the share of households that buy anything, the share that buy at least twice and unsubscribe rates.
How Big a Test
Table 1
Incremental Margin Under Two Illustrative Outcomes
| Outcome | Offered group buys bundle | Offered group buys full price | Holdout buys full price | Margin per household, offered | Margin per household, holdout | Incremental margin |
|---|---|---|---|---|---|---|
| Bundle wins back buyers | 7.0% | 1.0% | 4.0% | $4.26 | $3.04 | +$1.22 |
| Bundle mostly discounts | 5.0% | 1.5% | 4.5% | $3.64 | $3.42 | +$0.22 |
Table 1 shows what the measure would look like under two illustrative outcomes.
In the first outcome the bundle would add about $7,400 across all 6,050 households over the season; in the second it would add about $1,300, close to nothing. The test needs to tell these apart. A holdout of about 900 households would be enough to detect a difference of three percentage points in the share buying anything, with conventional confidence. Margin is noisier than a yes-or-no purchase, so the plan uses a holdout of 1,210, larger than the minimum. Even so, small effects on revenue are hard to measure precisely, and the team should expect a range rather than a single number (Lewis & Rao, 2015).
Contamination Risks
Four things could blur the comparison, and each has a safeguard.
1. Forwarded promo codes: each offered household gets a single-use code tied to its account, so a forwarded code cannot be used by a holdout household.
2. Social posts about the bundle: the bundle is not advertised publicly during the test, and the ticket office does not mention it to callers who do not have a code.
3. Staff overrides: ticket office staff are told not to apply the bundle price to holdout households who ask about it, and any exceptions are logged.
4. Other offers: no other promotion is sent to the eligible list during the test. Offers to the general public continue, and they reach both groups equally.
Well-run tests check that the two groups really look alike before results are read, for example by comparing their past purchase rates, and stop to investigate if they do not (Kohavi et al., 2020).
Timing
The offer goes out four weeks before the midpoint of the season and remains valid for any of the next eight home matches. Results are read once, after the eighth match. The analysis will not be stopped early because results look good after two matches, and it will not be extended because results are disappointing.
The Rollout Rule
The rule below was agreed and signed by the marketing director, the ticketing manager and the general manager before the first email was sent: "If incremental margin per contacted household is above zero with at least 90 percent confidence, the bundle will be offered to the full list of lapsed buyers for the remainder of the season and next season. Otherwise it will be dropped. The decision will not be revisited on the basis of bundle sales counts."
Writing the rule down first matters as much as the randomization. Without it, each side could read an ambiguous result as support for its view, and the argument would continue with data added.
References
Gordon, B. R., Zettelmeyer, F., Bhargava, N., & Chapsky, D. (2019). A comparison of approaches to advertising measurement: Evidence from big field experiments at Facebook. Marketing Science, 38(2), 193-225. https://doi.org/10.1287/mksc.2018.1135
Kohavi, R., Tang, D., & Xu, Y. (2020). Trustworthy online controlled experiments: A practical guide to A/B testing. Cambridge University Press. https://doi.org/10.1017/9781108653985
Lewis, R. A., & Rao, J. M. (2015). The unfavorable economics of measuring the returns to advertising. The Quarterly Journal of Economics, 130(4), 1941-1973. https://doi.org/10.1093/qje/qjv023
The BUS-655 Topic 7 assignment instructions
BUS-655 Topic 7 asks students to design a controlled test of a marketing or pricing action. Plans that compare people who used an offer with people who did not miss the point. The assignment rewards a plan that states the decision first, defines the eligible audience from the organization's records, randomizes a holdout group and explains why that unit was chosen. Students should choose a primary measure that captures the real economics, size the test for the effect that matters, name contamination risks with safeguards, fix the timing and write the decision rule down before launch so neither side can redefine success afterward.
Inside the BUS 655 Topic 7 example
The plan opens with the bundle and both sides of the marketing argument. The eligible list is defined from the CRM and grouped by household. A section explains why households, not matches, are randomized and why a comparison of responders would mislead. The primary measure is defined with the margin on each kind of purchase. Table 1 works through two outcomes, one where the bundle wins buyers back and one where it mostly discounts, and the holdout is sized. Four contamination risks are paired with safeguards, timing rules prevent early stopping and the signed rollout rule closes the plan.
BUS 655 Topic 7 rubric: what Excellent looks like
Instructors grading the test plan reward a decision stated before any design. Top plans define the audience precisely and randomize at a level that keeps groups separate. Credit goes to a primary measure that counts the cost of discounts given to people who would have bought anyway, and to a test sized with the measure's noise in mind. The rubric values practical safeguards against forwarded codes, staff overrides and overlapping offers, along with balance checks before results are read. The highest band fixes the reading date and has the rollout rule signed in advance by everyone involved. Plans that count bundles sold as success usually lose points.
BUS 655 Topic 7 help: mistakes that cost marks
Plans on this topic most often lose credit for measuring the wrong outcome. Counting bundles sold treats every discounted purchase as a win, including those from buyers who would have paid full price, and a promotion can post strong sales while losing money on exactly that group. A comparison group chosen by convenience, such as buyers who happened not to open the email, reintroduces the self-selection randomizing was meant to remove. Randomizing across a few games instead of many buyers leaves too few units to separate the offer from the schedule. Plans with no decision rule written before launch invite the result to be read in favor of whoever proposed the promotion. Graders also notice when a pricing test randomizes the posted price itself across customers without addressing fairness, a concern many sections expect a graduate plan to raise.
Get a BUS-655 Topic 7 example written to your instructions
Send us the BUS-655 Topic 7 instructions, the rubric from your classroom and the pricing or promotion decision your case poses. We write a custom example to those criteria, with the decision stated, a randomized holdout designed, incremental margin as the measure, contamination addressed and the rollout rule fixed before launch, in 24 to 48 hours. The first one is free.
More BUS 655 papers and MBA samples
- BUS 655 Topic 1: Commercial Question Register
- BUS 655 Topic 2: Attendance Driver Model
- BUS 655 Topic 3: Price Elasticity Estimate
- BUS 655 Topic 4: Fan Data Audit
- BUS 655 Topic 5: Promotion Attribution Dq Post
- BUS 655 Topic 6: Renewal Risk Model
- BUS 655 Topic 8: Analytics Function Charter
- BUS 660 Topic 7: Simulation and Inventory Analysis
- BUS 635 Topic 7: Property Revenue Strategy
- ECN 601 Topic 7: Policy Transmission Analysis
- ACC 650 Topic 7: Standard Cost Variance Report
BUS 655 Topic 7 questions, answered
Why hold out customers instead of comparing before and after?
Because too much else changes between periods. Opponents, weather, the standings and other campaigns all shift from one stretch of the season to the next, and a before-and-after comparison credits the offer with all of it. A randomly withheld group lives through the same season at the same time, so the difference between the groups isolates what the offer did.
Is it acceptable to withhold a discount from some customers?
Withholding a marketing offer from part of a list is standard practice, since every campaign reaches some buyers and not others. Randomizing posted prices for the same seat is a different matter and raises fairness and trust concerns that a plan should address. The example tests an emailed offer for that reason, and your instructions may specify what your organization or case allows.
How large should the holdout group be?
Large enough that a gain worth acting on would stand out from chance, which depends on the expected response rate and the smallest effect that would change the decision. The example sizes it from illustrative response rates and explains the reasoning. A tiny holdout often produces a result nobody trusts, which wastes the test and usually ends in the very argument the plan was meant to replace.
Where can I find a free BUS 655 Topic 7 sample paper?
The whole bundle test plan is above: the decision, the eligible households, randomization, the primary measure, the sizing table, contamination safeguards, timing and the signed rollout rule. Tell us the offer you want to test and we will set up a free plan around it.
Why not count bundles sold in BUS 655 Topic 7?
Some buyers would have paid $76 anyway, so selling them a $59 bundle with drinks and a scarf loses money. Incremental margin per contacted household counts that cost; a bundle count hides it.