Recommendations Recommending the Wrong Thing
A media company deploys a Recommendations activity using the 'People Who Viewed This, Viewed That' algorithm on article pages. After three weeks, click-through on recommended articles is 1.8%, below the 2.4% baseline for a manual 'most popular' module. The catalog contains 40,000 articles, many published within the last 48 hours. Analytics shows 70% of article traffic is from search engines and lands on a single article, then bounces. The recommendations feed only includes items that have at least one co-view event. The team proposes switching to 'Recently Viewed Items' and calling it done.
1. Diagnose why the co-view algorithm underperforms here, and evaluate whether 'Recently Viewed Items' is the right fix.
Read the worked answer
Diagnosis. 'People Who Viewed This, Viewed That' depends on co-view co-occurrence data. With 70% bounce traffic landing on one article and leaving, the co-view graph is sparse: most articles have few or zero co-view partners, so the algorithm has little to work with and the feed is thin or dominated by a handful of heavily co-viewed items. Newly published articles (many within 48 hours) have essentially no co-view history, so they can never be recommended - which is bad for a media site whose freshest content matters most. The algorithm is data-starved, not broken.
Is 'Recently Viewed Items' the fix? Weak. Recently viewed is a session-continuity feature, not a discovery engine; on a bounce-heavy, search-landing audience there is often only one prior view, so the module degenerates into showing the article the user just read or nothing. It does not solve the cold-start problem for new articles and does not improve discovery.
Better direction. Consider a content-similarity or popularity-based algorithm that works with sparse behavioral data (e.g., 'Most Popular' or a similarity algorithm keyed on article attributes), ensure the catalog feed includes new items with required attributes, and set up a fallback so the module never renders empty. Test against the manual baseline before rolling out.
2. What configuration and data-quality checks would you run before changing the algorithm?
Read the worked answer
Run these checks in order:
- Catalog/feed health. Confirm the feed is ingesting all 40,000 articles, that new articles appear within the expected refresh window, and that required attributes (entity ID, name, category, image, URL) are populated. Missing attributes silently drop items from recommendations.
- Exclusions and filters. Check whether any exclusion rules (e.g., exclude already-purchased/viewed, category filters) are over-filtering and shrinking the candidate pool to near-empty.
- Fallback behavior. Verify what renders when the algorithm returns fewer items than the design slot count - an empty or single-item module depresses CTR independent of algorithm quality.
- Collection vs. environment. Confirm the activity points at the correct collection/environment and that the production catalog is the one being queried.
- Measurement. Confirm the success metric (click-through) is firing correctly, that the baseline 2.4% was measured on a comparable audience and placement, and that the 1.8% is not an artifact of impression counting (e.g., module rendered below the fold).
- Segmentation. Break results by traffic source; search-landing bounce traffic may behave differently from returning readers, and the aggregate may mask a segment where recommendations do work.