Is the sales drop really just prayer cards?
John flagged "3 days in a week performing 25% under norm" and asked whether the A/B tests are to blame. We pulled ground-truth revenue and the GA4 export to answer two things: (1) where the drop actually is, and (2) whether any running test could have caused it. Short version: the decline is concentrated almost entirely in the prayer-card category, and no running test can produce a prayer-card-only effect.
The headline
Real revenue is down 9% week-over-week, not 25%. Of the category-level decline, prayer cards explain effectively all of it — every other meaningful category is flat or up.
The "25%" feeling comes from comparing against the abnormally high Mother's Day prep week (May 11–14, ~$10.5k/day). Against a normal baseline (~$7,429/day), the recent week (~$6,795/day) is a mild dip — and it is not spread across the store.
1. Where the drop is — by category
Prayer cards (cat 24) is the dominant category by volume and the only large one that fell. Sessions held flat (−2%) while conversion dropped from 4.6% to 3.2% — a revenue fall of $593/day (-33%).
| Category | Sess/day (prior→now) | Conv rate | Rev/day (prior→now) | Δ rev |
|---|---|---|---|---|
/c/24/ prayer-cards ← | 201 → 198 | 4.6% → 3.2% | $1,821 → $1,228 | -33% |
/c/23/ memorial-posters | 75 → 96 | 4.2% → 4.2% | $540 → $742 | +37% |
/c/98/ prayer-cards (sub) | 50 → 62 | 0.9% → 2.7% | $126 → $417 | +231% |
/c/49/ shop-standard-program | 60 → 58 | 1% → 1.2% | $156 → $264 | +69% |
/c/19/ funeral-guest-books | 10 → 10 | 1.4% → 6.7% | $34 → $160 | +371% |
/c/29/ catholic-prayer-cards | 50 → 57 | 2% → 0.6% | $230 → $32 | -86% |
/c/10/ single-photo-posters | 57 → 32 | 3.5% → 2.6% | $237 → $96 | -59% |
Prayer cards alone account for ~125% of the net category-attributed revenue decline — meaning if prayer cards had held flat, the rest of the store combined would have been slightly up. Small categories (catholic-prayer-cards, single-photo-posters, etc.) swing hard in percentage terms but move only single-digit dollars per day — that's purchase-count noise, not signal.
2. Within prayer cards — by traffic source
Traffic volume barely moved on any source. The conversion fell hardest on google / organic (5.3 → 3.2 purchases/day, revenue −46%). Notably, google / cpc was up on prayer cards (+16% revenue) — so this is not a paid-traffic-quality story the way it first looked.
| Source | Sess/day | Purchases/day | Rev/day | Δ rev |
|---|---|---|---|---|
| google / organic | 149 → 149 | 5.3 → 3.2 | $1,013 → $544 | -46% |
| (direct) | 45 → 44 | 2.4 → 1.8 | $533 → $314 | -41% |
| bing / cpc | 38 → 47 | 1.4 → 1.3 | $344 → $334 | -3% |
| google / cpc | 83 → 90 | 3.0 → 3.2 | $613 → $709 | +16% |
3. Can any running test cause a prayer-card-only drop? No.
We checked the URL targeting and DOM selector of every running experiment. None is scoped to prayer cards specifically — they're either sitewide, all-categories, checkout-only, or posters-only. A test that fires everywhere cannot produce a decline that shows up only on prayer cards. If it could, we'd see the same conversion crash on every other category and on the homepage. We don't.
| Test | Where it fires | Verdict | Why |
|---|---|---|---|
| FP-34 Paper thickness radio cards | Paper-thickness PDPs (incl. prayer cards) | PAUSED | Had a real bug (radio cards vanished on dropdown change, did not return). Paused May 28. PDP-level purchase data was still fine (13 vs 11). |
| FP-38 Category card design button | Every /c/ page (sitewide categories) | cleared | Fires on all categories, not prayer cards specifically. All 3 arms moved together — a broken arm would show one arm down vs control. Kept running. |
| FP-40 Satisfaction guarantee next to CTA | Sitewide (anchors to footer logo, on every page) | cleared | Verified sitewide. Cannot produce a prayer-card-only effect. |
| FP-37 Move coupon code from banner to cart | Sitewide | cleared | Sitewide shipping banner. Cannot produce a category-specific effect. |
| FP-95 Show only delivery date in checkout | Checkout only (/onepagecheckout) | cleared | Checkout-stage only, identical for all products. |
| FP-31 Rolled vs foam core mounting buttons | Poster PDPs only (GTM tag) | cleared | Posters only. Does not touch prayer cards at all. |
The one test with a confirmed bug, FP-34, was paused. Its bug (the radio-card UI disappeared when a dropdown changed and didn't return) only affected prayer-card product pages, not the category page where the drop is concentrated — and the FP-34 PDP purchase data was actually fine (13 variant vs 11 control). So even the buggy test is not the explanation for the category-level decline.
Conclusion
- Yes — the drop is essentially just prayer cards. Other large categories (memorial posters, standard program, guest books) are flat or up.
- It is not an A/B test. No running test is scoped to prayer cards; the sitewide ones would have moved every category equally, and they didn't.
- It is a conversion problem, not a traffic problem. Sessions held flat; fewer of them bought. The biggest fall was on google / organic.
- It is not a 25% sitewide crash. Real revenue is down ~9% week-over-week; the "25%" came from comparing to the Mother's Day spike week.
Most likely causes (to investigate next)
- Organic ranking / SERP shift on prayer-card keywords. Google organic took the biggest revenue hit while its session count held — check Search Console positions for prayer-card queries, last 14 days.
- Prayer-card category template / content change. Conversion fell with flat traffic — consistent with an Infigo or theme change on that category. Nicole's video confirmed at least one UX regression in this area.
- Memorial Day weekend. Real revenue's single worst day was Sunday May 24 (−42%). Part of the dip is simply the post-holiday weekend, but that alone wouldn't isolate to one category.
Caveats
- GA4 undercounts purchases (the purchase tag is DOM-based and misses orders inconsistently — 9–24% normally, and 30–50% on some days in this window). All category and source splits above are GA4 and carry that undercount, but it applies equally to both weeks, so the direction and ratios are reliable. Absolute sitewide revenue uses the Revenue Tracker sheet, which is ground truth.
- 6 days vs 7 days — all figures are day-normalized to make the weeks comparable.
- Category attribution counts a session under a category if it viewed any
/c/{id}/page in that session; a session that browsed two categories is counted in both.