Finding a deleted cache key between two release tags
An e-commerce Rails app's p95 latency doubled after the v2.4.0 release. The setting cache.ttl existed in config/cache.yml at tag v2.3.0 but is absent at v2.4.0. There are 200 commits and several merge commits between the tags. git blame on the surrounding lines points to a RuboCop reformat that touched 300 files, but the key is gone, so blame cannot show the deletion. One suspect is a merge commit that resolved a conflict in cache.yml by taking the wrong side. The release is deployed; main is protected and already contains the bad merge. The team needs the exact commit that removed the key, then a safe way to restore it without rewriting shared history. Production config is loaded from the repository at deploy time; no feature flag covers it.
1. Which history-inspection commands would you use to find the commit that removed `cache.ttl`, and why is `git blame` insufficient? Include how you would limit the search to the tag range.
Read the worked answer
git blame attributes lines that exist in the current file. When cache.ttl is deleted, there is no line to blame, so blame points at the last commit that touched the surrounding context, such as the reformat. Use the pickaxe to find commits where the number of occurrences of the string changed:
git log --oneline -S'cache.ttl' v2.3.0..v2.4.0 -- config/cache.yml
git log -p -S'cache.ttl' v2.3.0..v2.4.0 -- config/cache.yml
-S finds commits that change the count of the string; -G would find commits whose diff contains a matching line even if the count is unchanged, which is useful for a renamed key. Limit to the file and tag range to avoid noise. Merge commits can be hidden by history simplification, so if the result is empty or suspicious, add --full-history -m and inspect the merge:
git log --full-history -m --oneline -S'cache.ttl' v2.3.0..v2.4.0 -- config/cache.yml
git show <sha> -- config/cache.yml
Also compare the endpoints directly with git diff v2.3.0 v2.4.0 -- config/cache.yml. The commit that removed the key is the one whose diff shows the deletion, not necessarily the commit blame names.
2. You find the key was dropped in a merge commit, not in a normal commit. Explain how to confirm which parent still has the key and how you would revert or restore it. Why is `git revert <merge>` different from a normal revert, and what future merge problem can it create?
Read the worked answer
Confirm which parent retained the key:
git show --no-patch --format='%P' <merge>
git diff <merge>^1 <merge> -- config/cache.yml
git diff <merge>^2 <merge> -- config/cache.yml
git show <merge>^1:config/cache.yml | grep cache.ttl
git show <merge>^2:config/cache.yml | grep cache.ttl
The parent whose file still contains the key is the side that was dropped. A normal git revert <commit> reverses that commit's diff. A merge commit has multiple parents, so Git cannot know which side to reverse; git revert -m 1 <merge> reverses the changes introduced relative to parent 1. That records a new commit that undoes the merge result, but it also tells Git the branch from parent 2 should not be merged again: a later merge of that branch may not reintroduce its changes. Reverting the revert is then required.
For a single missing config key, prefer a forward fix: open an MR that adds cache.ttl back with a test that asserts the key exists. This avoids the merge-revert trap and keeps main's history append-only. Use git revert -m 1 only if the whole merge was wrong and must be undone.
3. A junior engineer proposes `git reset --hard v2.3.0` on `main` and force-pushing. Explain why that is unsafe here and what safeguards in GitLab would block or detect it.
Read the worked answer
git reset --hard v2.3.0 moves main back and discards the 200 commits from the branch tip; it does not create a revert commit. Force-pushing that would rewrite protected shared history: every clone, open MR, and CI ref based on the old main becomes divergent, and audit logs no longer match the branch. On GitLab, main protected with allowed-to-push No one should reject the force-push even for a Maintainer, depending on the 'Allowed to force push' setting. If that setting is on, the push may succeed but still violate policy. The protected-branch rule and audit events would expose it, but prevention is better.
The safe rollback is additive:
git switch -c revert/cache-ttl v2.3.0
# or restore just the key on a branch from main
git revert --no-commit v2.3.0..v2.4.0
git commit -m 'Revert v2.4.0 changes'
For a production rollback, deploy the existing v2.3.0 tag rather than rewriting main, then forward-fix the key in an MR. If the whole release must be undone in Git, revert the range on a branch, run CI, and merge through the protected-branch MR. Tags are immutable references; do not move v2.4.0 to hide the bad release.