Introduction to AI and AI-based Systems
What "AI" actually covers
Artificial intelligence is an umbrella term. Knowledge-based (symbolic) AI encodes human expertise as explicit rules - expert systems, rule engines, decision tables. Machine learning (ML) instead induces a model from data: supervised learning (labelled targets), unsupervised learning (structure discovery), and reinforcement learning (reward-driven policy learning). Deep learning is a family of ML using multi-layer neural networks. Also distinguish narrow AI (competent at one task) from general AI (hypothetical, human-level across domains).
Anatomy of an AI-based system
An AI-based system is more than a model. It combines: the ML model(s), the data used for training and operation, non-ML software (pre/post-processing, business logic, UI, integrations), and the environment in which it operates (users, sensors, other systems, regulations). Failures emerge at all four levels, so testing must span them.
Why testing AI is different
In conventional software the specification is the oracle. In ML, behaviour is learned, so requirements are often statistical or emergent, outputs are probabilistic and non-deterministic, and there is no general oracle for "correct" output. Some systems are opaque (limited interpretability). This shifts testing toward data testing, metric-based evaluation, and statistical reasoning.
Traps in exam questions
- Equating "AI" with "machine learning" - a rule-based expert system is AI but not ML.
- Assuming autonomy implies general AI.
- Claiming conventional test techniques are useless for AI systems - they still apply to the non-ML parts, and equivalence partitioning/boundary values still apply to input validation.
Rule of thumb
If the behaviour can be fully written down as rules in advance, treat it as conventional software testing even if the product is marketed as AI. If the behaviour only exists as a pattern in data, you need data- and metric-driven testing.