Weekly releases with a shared Common module
Helios Insurance has 14 applications and 260 modules on O11, deployed through Development, Quality, and Production. Releases happen every two weeks and use a manual deployment plan built by whoever remembers to run dependency analysis. Last quarter 41% of production deployments required a rollback, mostly because a shared module named Common was changed at the last minute by another team. Common has 190 public actions and is referenced by every application; it changes five times a week in Development. There are 30 timers in Production with no documented owner, 12 of which fail daily. The performance monitor shows 18 screens slower than 5 seconds, but nobody reviews it. Test coverage is manual. The CIO wants weekly releases within three months.
1. Diagnose the lifecycle health problems and rank them by risk.
Read the worked answer
Ranked by risk
- Common module coupling. 190 public actions and a dependency from every application means each change has portfolio-wide blast radius. A last-minute edit forces a broad deployment plan and is the direct cause of the rollback pattern. This is the highest risk because one change can affect all 14 applications.
- Manual deployment planning and late changes. A 41% rollback rate shows the process cannot detect impact before production. The plan depends on memory, so omitted dependencies surface only at deploy time.
- Unowned failing timers. Twelve daily failures mean silent data staleness, and no owner means no fix path. This is a correctness risk that may already be producing bad business data.
- Unreviewed performance. Eighteen slow screens are a growing cost and user-experience issue, but they degrade gradually rather than breaking a release.
The ranking matters: fixing timers before the Common module would leave the release process just as fragile.
2. Design a release process that supports weekly releases safely.
Read the worked answer
Structure: split Common into foundation and canonical modules with named owners. Version them and require a deprecation window for public actions. This alone shrinks the deployment plan.
Flow: branch per team, merge to a release branch, and freeze shared modules during the release window. Generate deployment plans from dependency analysis through lifecycle management rather than memory; attach the impact analysis to every plan. A module owner signs off on changes to shared modules; the release manager owns the schedule.
Automation: unit tests for public actions, integration tests for critical flows, smoke tests after deployment, and automated checks for BPT and timer health. Keep the previous version deployable and use feature flags for risky behavior so rollback does not require a redeploy.
Cadence: two-day code freeze, automated regression, staged rollout to a canary environment or a subset of users, then production. Track change failure rate and lead time weekly; if the rollback rate does not fall, the freeze or the test gate is not working.
3. What should Helios do about the 30 Production timers with no owner?
Read the worked answer
Build an inventory before changing anything: for each timer record its module, schedule, business purpose, last successful run, failure mode, and downstream consumers. Assign an owner through the module's lifecycle, not through a spreadsheet. Move timers into process modules owned by the relevant domain so they do not ride on UI deployments.
For the 12 failing timers, classify the failures: timeout, data error, dependency outage, or overlapping runs. Convert critical ones to BPT with retries, error logging, and alerting; delete redundant timers; add a lock or cap Max Instantiations to prevent overlap. Set realistic wake-up intervals and timeouts, and alert when a timer is still running at its next scheduled start.
Publish a timer health dashboard with owner, last run, duration, and failure count. Unowned timers are an operational debt that will quietly undermine any weekly release, because a deployment can change their behavior without anyone noticing.