Forms of Discriminant Analysis Explained in Details
Quick answer
Discriminant analysis classifies observations into groups from predictor variables. The main forms are Linear Discriminant Analysis (LDA), which assumes a shared covariance matrix across classes and gives linear boundaries; Quadratic Discriminant Analysis (QDA), which allows per-class covariance and curved boundaries; Regularised Discriminant Analysis (RDA), which interpolates between them for high-dimensional data; plus flexible and mixture variants for non-linear cases. LDA also doubles as a supervised dimensionality-reduction method.
Short answer: Discriminant analysis classifies observations into known groups from predictor variables by modelling each class's distribution. LDA assumes all classes share one covariance matrix → linear boundaries; QDA lets each class have its own covariance → curved boundaries; RDA shrinks between the two for high-dimensional data. LDA also serves as a supervised dimensionality-reduction technique. The trade-off is bias vs variance: LDA is more stable, QDA more flexible but needs more data.
Discriminant analysis predicts which group a new observation belongs to, based on predictor variables. It's a generative classifier: it models what each class's data looks like (assumed roughly Gaussian) and assigns new points to the most probable class via Bayes' rule.
The key distinction: shared vs per-class covariance
The forms differ mainly in one assumption — how they model each class's covariance:
| Form | Covariance assumption | Decision boundary | Best when |
|---|---|---|---|
| LDA | One shared matrix for all classes | Linear | Classes have similar spread; less data |
| QDA | Each class has its own | Quadratic (curved) | Classes have different spread; more data |
| RDA | Shrinks per-class toward shared | Between the two | High-dimensional / small samples |
LDA (Linear Discriminant Analysis) assumes every class shares the same covariance matrix. That constraint makes it simple, stable, and hard to overfit — a strong default when data is limited.
QDA (Quadratic Discriminant Analysis) relaxes that: each class gets its own covariance, producing curved boundaries. More flexible, but it estimates far more parameters, so it needs more data and can overfit.
RDA (Regularised Discriminant Analysis) interpolates between LDA and QDA (and shrinks toward a diagonal), which stabilises the covariance estimates when you have many features and few samples.
Two further variants you'll see: Mixture DA models each class as a mixture of Gaussians (for multi-modal classes), and Flexible DA replaces the linear step with non-linear regression.
LDA and QDA in scikit-learn
from sklearn.discriminant_analysis import (
LinearDiscriminantAnalysis, QuadraticDiscriminantAnalysis
)
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
X, y = load_iris(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=0)
lda = LinearDiscriminantAnalysis().fit(X_tr, y_tr)
qda = QuadraticDiscriminantAnalysis().fit(X_tr, y_tr)
print("LDA:", lda.score(X_te, y_te))
print("QDA:", qda.score(X_te, y_te))Bonus: LDA as dimensionality reduction
Unlike PCA (which is unsupervised), LDA uses the labels to find the axes that best separate classes — useful for projecting to 2-D before plotting:
X_2d = LinearDiscriminantAnalysis(n_components=2).fit_transform(X, y)Choosing a form
- Start with LDA — it's the robust baseline.
- Move to QDA if classes clearly have different spread and you have enough data.
- Reach for RDA when features outnumber (or approach) samples.
- Use LDA's transform when you also want a supervised low-dimensional projection.
Related guides
Sources
Key takeaways
- •Discriminant analysis classifies observations into groups from predictor variables.
- •Linear Discriminant Analysis (LDA) assumes a shared covariance matrix and produces linear boundaries.
- •Quadratic Discriminant Analysis (QDA) allows per-class covariance and curved boundaries.
- •Regularised and flexible variants handle high-dimensional or non-linear cases.
Frequently asked questions
What is the difference between LDA and QDA?
LDA assumes one shared covariance matrix and gives linear boundaries; QDA lets each class have its own covariance, giving curved boundaries.
When would I use discriminant analysis?
To classify observations into known groups from predictor variables, especially when class distributions are roughly Gaussian.
Software Engineering Leader & Technical Author · Updated September 9, 2026