Concept Bottleneck Models

Prelude

The interpretability promise

Saliency tells you "the model used these pixels"; concept bottleneck (Koh et al., 2020) tells you "the model used these named features — here are their values, and if you change them by hand, here's how the prediction changes". The bottleneck is a known set of interpretable concepts; the head is a small classifier on top of them.

During training, you supervise both stages: the bottleneck must predict the right concept values from the image, and the head must predict the right label from the concept vector. At inference, you can replace any predicted concept with a human-set value (an intervention) and see the label respond.

Step 1

Watch a CBM predict and accept interventions

Synthetic 4-class problem. Each shape has 4 concepts: is-dark, is-large, has-stripes, is-warm. Class is determined by a fixed boolean recipe over those concepts. Toggle a concept off (or on) and watch the predicted class flip.

true class — predicted class —

image

predicted concepts (drag to override)

Each bar is the model's predicted concept value (0..1). Click a bar to override (forces the concept to 0 or 1). Refresh "New image" to clear.

Step 2

Three training recipes

Let $g_\theta$ map image → concept vector and $f_\phi$ map concept vector → label. With per-concept loss $\mathcal{L}_c$ and label loss $\mathcal{L}_y$, the three standard recipes are:

Sequential. Fit $\theta^\star = \arg\min \mathcal{L}_c(g_\theta(x), c)$ first; freeze; then fit $\phi^\star = \arg\min \mathcal{L}_y(f_\phi(g_{\theta^\star}(x)), y)$. Most interpretable: concepts cannot leak label information because the label signal never touches $g$.
Joint. Train $g$ and $f$ together with $\mathcal{L} = \mathcal{L}_y + \lambda \mathcal{L}_c$. Slightly higher accuracy but the bottleneck can encode spurious info to help $f$ — called leakage. $\lambda$ controls the trade-off: $\lambda \to \infty$ recovers sequential.
Independent. Train $f$ on ground-truth concepts during training; at test use predicted concepts. Best test accuracy when concepts are hard to predict.

Step 2½

Why interventions matter

A CBM lets you do something no neural net usually allows: run a counterfactual. "If striped were 0 instead of 1, would the prediction change?" — and the answer is computed by a single forward pass through $f$ on the edited concept vector.

Test-time auditing. Domain expert disagrees with the predicted "is-tumor" concept. They override; the CBM still produces a class prediction, but now from the expert's concept value, not the model's.
Mistake debugging. Run intervention sweeps on the failing examples; the concept whose flip fixes the prediction tells you exactly where $g$ is mispredicting.
Causality, sort of. Under the assumption that the named concepts mediate the image→label relationship, interventions on $\hat c$ yield the do-operator on $c$. The mediation assumption is often wrong, hence the leakage trap below.

Step 3

The leakage trap, with a live demonstration

If the concept layer has more channels than the ground-truth concept count, the model can use the spare channels to encode arbitrary features. Then "intervening" on a named concept doesn't actually change the prediction — the spare channel still encodes the answer. Several recent papers (Margeloiu et al., 2021; Mahinpei et al., 2021) document this.

Below: same 4 named concepts, but the model has $K$ extra "spare" channels that get trained jointly on the label loss (joint recipe with high $\lambda_y$). As $K$ rises, the accuracy under intervention drops — the spare channels have stolen the predictive signal.

spare hidden channels K 0 label-loss weight λ_y 0.5

Intervention faithfulness vs spare channels

"Clean accuracy" = label correctly predicted on held-out data. "Intervention accuracy" = on inputs where flipping one concept should change the class, does the CBM's prediction flip too? If yes, the bottleneck is faithful. When spare channels exist, the gap widens.

The fix. Use the concept layer dimension equal to the number of named concepts. Use sequential training. Verify with intervention experiments — if forcing a concept to its opposite doesn't change predictions on the right subset of inputs, your bottleneck has leaked.

Step 3½

2023–2026 follow-ups

Probabilistic Concept Bottleneck Models (Kim et al., 2023) — concept values are distributions, so the head can express uncertainty over interventions.
Label-free CBMs (Oikarinen et al., 2023) — derive concept vocabulary from CLIP text embeddings instead of human labelling. Lower accuracy than supervised CBMs but no annotation burden.
Concept Embedding Models (CEMs) (Espinosa-Zarlenga et al., 2022) — replace scalar concept values with low-dim concept embeddings, restoring most of the accuracy gap to a black-box net.
Intervention training — at train time, randomly intervene on concept values; teaches $f$ to actually use the bottleneck instead of routing around it.

Step 4

Where CBMs are the right tool

Medical imaging. Radiology concepts (lesion size, edge type, surrounding tissue density) are already in the doctor's vocabulary. Predicting them explicitly lets the clinician interrogate the model in their own language; original CBM paper used CUB-200 birds and OAI knee X-rays.
Bioacoustics & ecology. Bird-call classifier with concepts like "pitch range", "trill present", "burst rate". Domain experts can audit mispredictions by intervening on the wrong concept.
Auditable policy decisions. Any model whose output feeds a high-stakes decision (credit scoring, disease screening, fairness audits) benefits more from "the prediction was X because concepts $c_1, c_3$ fired" than from a raw probability with no interpretable structure.
Concept-conditioned generation. The "concept" interface generalises beyond classifiers — CBMs for image generators let you steer outputs by toggling human-named attributes.

Final takeaway. Concept bottlenecks trade a small amount of accuracy for a large amount of accountability. For any model whose predictions feed a policy decision, this trade is usually a bargain — provided you guard against leakage and verify interventions actually steer the prediction.

Step 5

Reading list

Koh, Nguyen, Tang, Mussmann, Pierson, Kim, Liang (2020) — Concept Bottleneck Models. The original paper.
Margeloiu et al. (2021) — Do Concept Bottleneck Models Learn as Intended? The leakage paper.
Mahinpei et al. (2021) — Promises and Pitfalls of Black-Box Concept Learning.
Espinosa-Zarlenga et al. (2022) — Concept Embedding Models.
Oikarinen et al. (2023) — Label-Free Concept Bottleneck Models.