What happens in anomaly detection?
Quick answer
Anomaly detection identifies data points that deviate from expected patterns. A model learns what 'normal' looks like from historical data, then flags observations that fall outside it. Approaches include statistical thresholds (z-score / IQR), distance and density methods (k-NN, Local Outlier Factor), tree-based Isolation Forest, one-class SVM, and autoencoders that flag high reconstruction error. Because anomalies are rare and often unlabelled, it is usually framed as unsupervised learning and evaluated with precision/recall rather than accuracy. It is widely used for fraud, intrusion, and equipment-failure detection.
Short answer: A model learns what "normal" looks like from historical data, then flags observations that fall outside it. The main method families are statistical thresholds (z-score, IQR), density/distance methods (k-NN, Local Outlier Factor), tree-based Isolation Forest, one-class SVM, and autoencoders that flag high reconstruction error. Because anomalies are rare and usually unlabelled, it's typically unsupervised and judged on precision/recall — not accuracy.
Anomaly detection is the process of identifying unusual patterns or events in data that don't conform to expected behaviour — fraud, intrusions, equipment malfunctions, and other rare events worth investigating.
The core idea: build a model of normal, then measure how far each new observation sits from it. Points that are far enough out — by whatever distance, density, or reconstruction measure the method uses — are flagged as anomalies.
Why it's usually unsupervised
Anomalies are rare and often unlabelled — you rarely have a clean, balanced "fraud vs not-fraud" training set. So most anomaly detection is unsupervised: the model sees mostly-normal data and learns its shape, without labels. This also means accuracy is the wrong metric — a detector that flags nothing scores 99.9% accuracy on data that's 0.1% anomalies. Use precision, recall, and the precision–recall AUC instead.
The main method families
| Family | How it decides "anomalous" | When it fits |
|---|---|---|
| Statistical (z-score, IQR) | Value is many standard deviations / IQRs from the centre | Low-dimensional, roughly Gaussian data |
| Distance / density (k-NN, LOF) | Point is far from its neighbours, or in a lower-density region | Clusters of varying density |
| Isolation Forest | Point is isolated by random splits in few steps | High-dimensional, large datasets |
| One-class SVM | Point falls outside a learned boundary around normal data | Smaller datasets, clear normal region |
| Autoencoder | High reconstruction error — the net can't rebuild it | Complex, high-dimensional (images, signals) |
A concrete example: Isolation Forest
Isolation Forest is a strong default: it randomly partitions the data, and anomalies — being few and different — get isolated in fewer splits. In scikit-learn:
from sklearn.ensemble import IsolationForest
import numpy as np
# mostly-normal data with a few outliers
X = np.concatenate([np.random.normal(0, 1, (200, 2)),
np.array([[6, 6], [-5, 7], [7, -6]])])
model = IsolationForest(contamination=0.02, random_state=0)
pred = model.fit_predict(X) # -1 = anomaly, 1 = normal
print((pred == -1).sum(), "anomalies flagged")contamination is the expected fraction of anomalies — it sets the decision threshold, so tune it to your domain rather than leaving it at the default.
Local Outlier Factor (density-based)
When "normal" has regions of different density, LOF compares each point's local density to its neighbours':
from sklearn.neighbors import LocalOutlierFactor
lof = LocalOutlierFactor(n_neighbors=20, contamination=0.02)
pred = lof.fit_predict(X) # -1 = anomalyWhere it's used
- Fraud detection — a card used only for small local purchases suddenly making large high-end ones.
- Network security / intrusion — a host that normally receives a handful of connections suddenly receiving thousands.
- Manufacturing — a machine whose stable temperature starts fluctuating, signalling impending failure.
- Healthcare — a patient's vitals departing from their established baseline.
The technique you pick depends on the data's dimensionality, whether you have labels, and how you'll handle the flagged points — anomaly detection surfaces candidates for investigation, it doesn't decide on its own that something is wrong.
Related guides
Sources
Key takeaways
- •Anomaly detection identifies data points that deviate from expected patterns.
- •A model learns what normal looks like from historical data, then flags observations outside it.
- •Approaches include statistical thresholds, distance/density methods (k-NN, LOF), Isolation Forest, and autoencoder reconstruction error.
- •Common uses are fraud, intrusion detection, and equipment failure.
Frequently asked questions
How does anomaly detection work?
A model learns normal behaviour from historical data and flags observations that fall significantly outside it.
What methods are used for anomaly detection?
Statistical thresholds, distance and density methods such as k-NN and LOF, tree-based Isolation Forest, and autoencoders.
Software Engineering Leader & Technical Author · Updated September 9, 2026