Quality Goals Without Numbers
A hospital is building a scheduling system for 2,000 clinicians. Stakeholders demand 'fast, secure, user-friendly and maintainable'. At shift change, 2,000 users log in within 10 minutes; current login takes 12 seconds. The system must be available during clinic hours, but no numeric target exists. Audit logs must be retained for 10 years and be tamper-evident. Role-based access is required. Change lead time is six weeks and defect escape rate is high. The team proposes an architecture review based on a checklist: layered architecture, microservices, event sourcing, API gateway. The CIO asks whether the system is 'good enough' and how to prove it before go-live. No quality scenarios or metrics exist, and the review is scheduled in four weeks.
1. Convert the vague goals into at least four measurable quality scenarios. Use stimulus, response and measure.
Read the worked answer
A quality scenario makes the goal testable by naming source, stimulus, environment, response and measure. Examples for this system:
- Performance: during shift change, 2,000 clinicians authenticate within 10 minutes; the system authenticates each user and p95 login response is under 3 seconds, with less than 0.1% failed logins caused by the system.
- Availability: during clinic hours (07:00-19:00), a component failure does not prevent schedule viewing or booking; monthly availability is at least 99.9%, and recovery from a single node failure takes under 5 minutes.
- Audit integrity: when a clinician changes a schedule, the system records an audit event within 1 second; the event is tamper-evident and retrievable for 10 years; an auditor can verify a sample of 100 events without manual reconstruction.
- Security: an unauthorised user attempting to view another clinician's schedule is denied within 500 ms, the attempt is logged, and no schedule data is returned.
- Maintainability: a typical scheduling-rule change is implemented, tested and deployed in under 10 working days, with no increase in defect escape rate.
The measures turn 'fast' into a testable latency target, 'secure' into an access-control response, and 'maintainable' into change lead time. Without measures, the checklist cannot prove anything.
2. Identify conflicts between these quality goals and propose how to resolve them with stakeholders.
Read the worked answer
The goals interact, and pretending otherwise hides risk.
- Security versus login speed: strong authentication can slow the 2,000-user shift-change login. Resolve with risk-based authentication: fast path on trusted devices, step-up authentication for sensitive actions, and load-tested identity providers. Do not silently drop MFA; document the risk decision.
- Audit completeness versus latency: synchronous writes to a tamper-evident store may slow scheduling changes. Resolve with a durable local audit buffer and asynchronous shipping, while guaranteeing that no acknowledged change lacks an audit record. Test the failure case.
- Availability versus consistency: if the system must remain usable during a network partition, schedule conflicts may appear. Decide which operations can degrade (viewing) and which must block (booking a slot that requires authoritative availability).
- Maintainability versus microservices: microservices can improve team autonomy but add operational and debugging complexity, which may worsen the current six-week change lead time. Start with modular boundaries and extract only where scaling or ownership demands it.
Resolution mechanism: a quality-attribute workshop with clinical, security, audit and operations representatives. Rank scenarios, make trade-offs explicit in decision records, and assign owners. The architect's job is to expose the conflict and its consequences, not to satisfy every adjective simultaneously.
3. How would you evaluate the architecture against these scenarios rather than the pattern checklist?
Read the worked answer
Use scenario-based evaluation and evidence, with patterns treated as means.
- Agree the scenarios and priorities with stakeholders before reviewing diagrams. The output is a ranked set of quality scenarios, not a list of preferred technologies.
- Walk through each scenario against the proposed architecture: components involved, runtime behaviour, failure modes, data flow and operational responsibilities. Record risks, sensitivity points and trade-offs.
- Prototype or benchmark the uncertain parts: login load at 2,000 users in 10 minutes, audit write and retrieval, failover, and a representative scheduling-rule change. A four-week review can include a focused load test and a threat-model workshop.
- Inspect the audit chain end to end: who writes, where it is stored, how tampering is detected, retention and deletion. 'Event sourcing' on a checklist does not prove tamper evidence.
- Define fitness functions for continuous checks: p95 login latency, audit write success, availability, change lead time and defect escape rate. Review them after go-live.
- Report fitness against scenarios: which are met, which are at risk, which are not met, and what decision is needed. The CIO gets evidence, not a pattern count.
The reasoning: a layered or event-sourced architecture can still fail the shift-change login or the audit requirement. Evaluation must test the required responses and measures, and make remaining risks explicit.