# Measure impact after the tap

- **Canonical URL:** https://playbook.affpartners.io/en/practices/push-measure/index.html
- **Markdown version:** https://playbook.affpartners.io/en/practices/push-measure/index.md
- **Module:** Push notifications
- **Time:** 90 minutes for the first report

An open shows that the copy attracted attention; it does not prove value. Connect the send to the destination screen and target action, then add a comparison and negative signals.

## Outcome

A report for one scenario covering sends, deliveries, opens, useful actions, a no-push comparison group, and trust impact.

## A tap is the middle of the journey, not the result

A push can collect many opens with loud copy while people do nothing after the tap or turn notifications off. The report must show the whole chain and answer one question: did the useful action start happening more often because of the message?

## What you will need

- **Event chain:** Send, delivery, open, the destination screen, and one confirmed useful action.
- **Comparison:** A small random group without the message, or at least a comparable period before launch.
- **Negative signals:** Notification opt-outs, complaints, uninstalls, and repeat opens without the target action.

## Terms in plain language

- **Push notification — A short app message shown outside the app:** Definition: It appears on the device and can return a user to one specific useful action. It is sent only when the user has granted notification permission. Example: A message about a new item in a saved topic opens that item through a deep link.
- **Open rate — The share of delivered messages that were opened:** Definition: Shows how many users tapped a delivered message. Open rate alone does not show whether they completed a useful action after the tap. Example: 200 of 1,000 delivered push messages were opened: open rate is 20%.
- **Analytics event — A record of a specific user action:** Definition: The app sends an event when a user opens a screen, taps a button, or completes an action. Example: registration_started and registration_completed reveal how many people abandon registration.
- **Conversion — The share of people moving to the next step:** Definition: Shows how many people who started a step completed it or reached the next one. Example: 80 registrations completed from 100 started: 80 ÷ 100 × 100% = 80% conversion.
- **Segment — Users sharing a behavior or characteristic:** Definition: A segment groups people by platform, market, interest, or action so the team does not send the same message to everyone. Example: Android users who started but did not complete registration in the past 24 hours.
- **Retention — The share of users who return:** Definition: Shows how many people reopen the app after a set period from installation or registration. Example: If 18 of 100 new users return after seven days, D7 retention is 18%.

## When to use it

The team reports delivery and open rate but does not know whether users completed the promised action — or whether it would have happened without the message.

## Where to assemble the chain

The send platform and product analytics must join by scenario and user. If the join is incomplete, state that as a limitation.

- **Send platform**
  - **Source:** Take the number of eligible users, sends, deliveries, and opens for one campaign ID.
  - **If access is unavailable:** Without an automatic export, export the campaign summary and record the exact send window.
- **Product analytics**
  - **Source:** Count destination-screen opens and the target action within a pre-chosen window after the push.
  - **If access is unavailable:** If the events are not joined, add campaign_id to the deep link and the screen event before the next launch.
- **Trust and control**
  - **Source:** Put opt-outs alongside, plus the results of the group that matched the scenario but randomly received no message.
  - **If access is unavailable:** Without a control group, compare with a similar week and state plainly that causal conclusions are limited.

## Set up measurement for one scenario

1. **Name one useful action.** Before launch, define the post-tap result and the measurement window. Do not use a generic “app visit.”
   - **Where to do it:** In the scenario specification and the report title.
   - **Example:** Read at least half of the new analysis within 24 hours of delivery.
2. **Join the events into one chain.** Use a single campaign_id and trace the journey from the send platform to the screen and the action in product analytics.
   - **Where to do it:** In the push parameters, the deep link, and the app events.
   - **Example:** Send → delivery → open → screen view → half the material read, all with campaign_id=topic_update_01.
3. **Add a comparable group.** Randomly hold back a small share of the eligible audience. It shows the baseline level of the action.
   - **Where to do it:** In the launch settings and a separate row of the report.
   - **Example:** 10% of the audience gets no push; the action: 3.2% versus 4.9% among recipients.
4. **Put negative signals alongside.** Watch opt-outs and complaints in the same window. A lift in the action does not count as a win if the scenario visibly erodes trust.
   - **Where to do it:** In the last rows of the report.
   - **Example:** Opt-outs at 1% against the usual 0.3% — the scenario needs rework despite the lift.
5. **Make one of three decisions.** Keep, change, or switch off the scenario. Record the reason and the date of the next check.
   - **Where to do it:** In the report conclusion and the communications log.
   - **Example:** Keep the audience, cut the frequency to a single send, and repeat the comparison in two weeks.

## Practical examples

- **Open rate 20%, action 4.9%:** Of 8,600 delivered pushes, 1,720 were opened, but 420 completed the useful action. The control group came in at 3.2% — there is an incremental effect, but far smaller than the open rate.
- **The lift in actions did not cover the opt-outs:** The target action grew, but opt-outs hit 1% against the usual 0.3%. The decision — reduce the frequency and repeat the test rather than scale right away.

## Finished artifact: Funnel of one push scenario

The numbers must belong to one audience, one window, and one scenario. Show the comparison next to them, not only the absolute result.

| Step | Users | Conversion | What it means |
| --- | --- | --- | --- |
| Matched the scenario | 10,000 | — | Eligible audience |
| Received the push | 8,600 | 86% | Check permissions and tokens |
| Opened the message | 1,720 | 20% | The copy caught attention |
| Opened the target screen | 1,410 | 82% | The deep link mostly works |
| Completed the action | 420 | 30% | Final conversion: 4.9% of deliveries |
| Turned notifications off | 86 | 1% | Compare with the usual level |

Control group without the push: 3.2% completed the useful action. The +1.7 percentage point difference makes the scenario a candidate to continue — if opt-outs stayed within the usual range.

## Safety and constraints: Do not optimize for taps alone

Loud copy can collect curiosity opens while eroding trust. Judge the scenario by the whole chain and the negative signals.

## Report checklist

Every metric must belong to the same scenario, audience, and measurement window.

- [ ] The push has one target action and a pre-defined window.
- [ ] Send, open, screen, and action are joined by a single campaign ID.
- [ ] There is a comparable group without the message, or the comparison limitation is stated plainly.
- [ ] Opt-outs and complaints are shown next to the useful action.
- [ ] The report ends with a decision: keep, change, or switch off the scenario.

## Which conclusions the team needs

- **Technical funnel:** Delivery, opens, and arrival on the right screen show where the route breaks.
- **Incremental effect:** The difference in the target action between push recipients and a comparable group without the message.
- **Trust:** Opt-outs, complaints, and later retention do not degrade beyond the acceptable level.

## Key rule

A good report lets you switch off useless pushes as confidently as you scale the genuinely useful ones.

---

- [HTML version](https://playbook.affpartners.io/en/practices/push-measure/index.html)
- [All practices](https://playbook.affpartners.io/en/)