Segmenting for Success: The Case of the Vanishing Mobile Users
A media company has a mobile app and a responsive website. They have implemented Adobe Analytics with a single report suite for both, using a prop to distinguish between 'App' and 'Web' traffic. The marketing team wants to understand why mobile app engagement (measured by average time on site) appears to be much lower than web engagement. They created a segment for 'App' users using the prop and compared it to 'Web' users. The results show App average time on site is 30 seconds, while Web is 3 minutes. The marketing team is concerned that the app is not engaging. However, the analytics team notices that the app's implementation does not send a 'page view' beacon on every screen; instead, it sends a single beacon on app launch and then uses custom links for screen views. The 'Average Time on Site' metric is calculated based on the time between the first and last beacon in a visit.
1. Why is the App average time on site so low, and is the comparison valid?
Read the worked answer
The App average time on site is low because the app only sends one beacon on launch and then custom link beacons for screen views. The 'Average Time on Site' metric in Adobe Analytics is calculated as (total time spent / visits), where time spent is the difference between the first and last hit in a visit. If the app only sends a launch beacon and then custom links, the time between the first and last hit may be short if the user quickly navigates away, or if the custom link beacons are not being included in the time calculation. Actually, Adobe Analytics includes all hits in time spent calculation, but if the app sends a single beacon on launch and then no further beacons (e.g., if the user stays on the same screen), the time spent is zero. Moreover, the app might not be sending a beacon when the app is backgrounded or closed, so the last hit is the launch, resulting in near-zero time. The comparison is not valid because the app and web are measured differently. The web sends a page view beacon on every page, so time between page views is captured. The app's custom links may not be treated the same way for time calculations. Also, the segment is based on a prop that may not be set on all hits, leading to data quality issues.
2. What would you recommend to accurately measure and compare engagement across app and web?
Read the worked answer
To accurately measure engagement, you should: 1) Implement a consistent tracking approach: in the app, send a page view beacon (or a custom link with a specific name) on each screen view, similar to web page views. 2) Ensure that the app sends a beacon when the app goes to background or closes to capture the end of the session. 3) Use a common metric like 'Screen Views' or 'Page Views' per visit, or 'Time Spent per Visit' calculated consistently. 4) Consider using Adobe's Mobile SDK with lifecycle metrics that automatically track time spent. 5) Create a calculated metric that normalizes time spent per hit or per screen view. 6) Validate the segment by checking if the prop is set on all hits; if not, use a more reliable method like a report suite or a virtual report suite. 7) Compare like-for-like: e.g., compare 'time per screen' vs 'time per page'.