Link Prediction Recommendations with Popularity Bias and Temporal Leakage
A social platform has 2.3 million :User nodes and 38 million directed :FOLLOWS relationships. The growth team wants top 10 'people you may know' per active user, refreshed weekly. They build a GDS link prediction pipeline with features commonNeighbors, adamicAdar, sameCommunity (community IDs written by Louvain on the full FOLLOWS graph), and preferentialAttachment. They configureSplit with testFraction 0.1, validationFraction 0.1, negativeSamplingRatio 1.0, train logistic regression, and get test AUC 0.91. In production, many recommended pairs already follow each other, the top slots concentrate on about 50 celebrity accounts, and 18% of active users get fewer than 10 recommendations after filtering. The team runs on a 32 GB heap and wants a weekly batch that finishes in 2 hours.
1. Why are already-following pairs returned, and how would you produce exactly 10 valid recommendations per active user?
Read the worked answer
The prediction procedure ranks candidate pairs by model probability; it does not guarantee that existing FOLLOWS relationships are excluded, and the pipeline's topN may fill slots with pairs that are already connected. Filtering after a small topN can therefore leave fewer than 10 rows. Generate a larger candidate set per source, then filter and re-rank.
Use the prediction output with topN set well above 10, for example 50 or 100. In Cypher, exclude self-pairs, blocked users, and existing edges with a pattern such as NOT EXISTS { MATCH (a)-[:FOLLOWS]->(b) }. Then rank by probability per source user and take the first 10. If some users still have fewer than 10, either increase the candidate topN for those users or apply a documented fallback such as community-popular accounts. Measure coverage explicitly, because 18% of users receiving fewer than 10 is a product defect, not a modelling detail.
2. The recommendations are dominated by about 50 celebrity accounts. How would you diagnose and reduce this popularity bias?
Read the worked answer
Preferential attachment and common-neighbour counts reward high-degree targets, so a global ranking can collapse onto celebrities. Measure the target-degree distribution of recommended pairs and compare it with the candidate pool. Also measure per-user recall@10 on a temporal holdout and coverage: what fraction of active users receive 10 distinct, non-existing recommendations?
Reduce bias by changing features and candidates, not just by adding a post-hoc penalty. Use Adamic-Adar and sameCommunity to reward specific shared context rather than raw popularity. Restrict candidates to 2-hop neighbourhoods or community members so celebrities are not globally eligible. If a popularity correction is still needed, divide the score by a function of target degree and tune it on validation data. Always keep a non-personalized fallback separate so bias in the personalized model is visible. Re-evaluate with per-user metrics, not only global AUC.
3. sameCommunity uses Louvain IDs computed on the full FOLLOWS graph. What is the risk, and how should the team evaluate the model for weekly production?
Read the worked answer
Computing communities on the full graph includes the edges that the pipeline later holds out for test. The sameCommunity feature can therefore encode future connections, inflating AUC. The pipeline's random edge split is also optimistic for a weekly system because it mixes older and newer edges. For production, use a temporal split: build the training graph from edges before a cutoff, compute communities and any other node properties from that graph only, and evaluate on FOLLOWS edges that appear after the cutoff.
Write the training-only community property with a clear name and regenerate it each training cycle. Do not reuse a property derived from a later graph snapshot. Evaluate per-user recall@10, coverage, and popularity bias on the temporal holdout, and monitor the live recommendation acceptance rate. A global AUC of 0.91 from a random split is not sufficient evidence that next week's recommendations will work.