Concept Bottleneck Models

We begin with the black box: what evidence did the model use, and can a person inspect it?

A neural network makes a prediction. How can a person inspect, and correct, the intermediate evidence it used?

Here is a photograph of a gull, and a real model, CLIP. The name stands for Contrastive Language-Image Pre-training.

It is asked to choose among six gull species, by comparing the image with a short description of each.

Image in, an encoder, and a score for every species. It prefers American herring gull, with a softmax share of 0.54. Glaucous-winged gull comes second, at 0.21.

The answer happens to be right. But what made the model prefer it?

A common response is a heatmap. This one is computed: we grey out one patch of the photograph at a time, and measure how much the herring gull score drops.

The largest drops sit on the bill and the head.

So what did the model recognise there? The yellow bill? The red spot near its tip? The shape of the head? Or the blue water just behind it?

The map marks where the score is sensitive. It does not say which property the model detected, and it gives a person nothing specific to correct.

Suppose instead the model had to state its evidence as named properties: wingtip colour, bill shape, head colour. Each claim could then be checked.

There are two ways to get an explanation from a model. You can train a black box and explain it afterwards, which is called post-hoc. Or you can build the model so that its reasoning can be read from the start, which is called ante-hoc, or interpretable by design.

Start with post-hoc. The model f is already trained. An explanation method takes the model and one input, and returns some evidence about the prediction.

Start with a real zebra photo and a trained image classifier, a Residual Network with 50 layers, or ResNet-50. Choose its zebra score before softmax. Keep the model fixed and slightly increase the red value at the marked pixel. The zebra score changes by a tiny amount; the predicted class need not change. Sensitivity is the size of the score change divided by the size of the pixel change. In symbols, x i is one pixel channel, s k is the chosen class score, and the derivative measures sensitivity to an infinitesimal nudge. The numerical ratio shown here is an approximation.

Keep this exact photo and model. After softmax, zebra is the top prediction; tiger has a very low probability. First choose zebra: backpropagate its score to every input pixel and show the largest absolute gradient across the colour channels. Brighter means a larger sensitivity; this gives the zebra saliency map. Now choose tiger and repeat, without changing the photo or the model. Its score is low, but it can still be sensitive to pixels. Both maps use one brightness scale and neither has been smoothed. Notice the background lights up too. A saliency map shows which tiny pixel changes affect a chosen score; it does not mean that a tiger is present, or trace the outline of the animal.

A second family changes the input. Grey out one patch, run the model again, and measure how much the score drops. That is the occlusion map we computed for the gull in Part 1.

The input is a gull photo. A trained model produces class scores; choose the herring gull score as the output to explain. The explanation method takes the photo, the model and this choice of class. For plain gradient saliency, it computes the pixel sensitivities from the formula we just saw. It displays them as a heatmap, with brighter pixels more sensitive. This is an explanation of the score, not another prediction.

Now keep the photo, the herring gull class and the explanation method fixed. Compare the heatmap from the trained model with one made after replacing its learned weights with random values. For some methods, such as guided backpropagation, Adebayo and colleagues found the maps barely changed. That is a warning: the picture can survive even after learning is removed. Before trusting a heatmap, check that it depends on the learned weights. Plain gradients did change in their test, but passing this check alone does not prove an explanation is correct.

Zech and colleagues studied a chest X-ray classifier. Focus on the model trained using data from both Mount Sinai and the National Institutes of Health Clinical Center, or NIH. Each training example pairs an X-ray with a pneumonia label derived from its radiology report. The label says whether pneumonia is reported. In the collected datasets, about a third of Mount Sinai X-rays have a positive label, compared with about one in a hundred at NIH. These are dataset label rates, not rates for all hospital patients. At test time, a new X-ray goes into the trained model, without its label, and a pneumonia score comes out. The intended clue is disease in the lungs, but the hospital already gives a clue to the label. That is the shortcut we will examine next.

A network could tell which hospital an X-ray came from almost perfectly, from markers and scanner details rather than from the lungs. Since the hospital predicts pneumonia, the detector could use the hospital as a shortcut.

A heatmap might light up the hospital marker in the corner, and expose the shortcut. But it cannot remove it. On X-rays from a third hospital, the detector did worse. The only fix was new data and retraining.

The third limit is vocabulary. A heatmap says where the score was sensitive. It does not say what the model saw there. On the gull: the yellow bill, the red spot, or the shape of the head?

Experts reason in named findings: a narrowed joint space, a bone spur, black wingtips. Even when an expert spots the mistake, the model has no named part to correct. There is no setting for wingtip colour that a person could switch from grey to black.

Ante-hoc models take the other route. The prediction is computed from quantities a person can read. A sparse linear model on named features is one example. A short decision tree is another. Prototype networks compare parts of an image with stored example parts.

Rudin argued that for high-stakes decisions we should prefer models like these over black boxes explained afterwards.

A concept bottleneck model is an ante-hoc model for deep networks. First, predict named properties of the photo, its concepts. Then predict the answer from those concepts alone.

In symbols: x is the input, c hat is the vector of predicted concepts, and y hat is the prediction. A concept predictor, g, maps x to c hat. A task head, f, maps c hat to y hat.

Now we put named concepts between the image and the prediction.

Here is that idea on a real photograph, with numbers small enough to follow. Between the image and the answer, we insert a layer of named variables.

A concept predictor, g theta, maps the image to a vector of concept values, c hat. A task head, f phi, maps those values to the prediction, y hat.

Here the concepts are wingtip grey, wingtip black, bill pointed, head white, and tail rounded. Each receives a value between 0 and 1.

The essential rule: the task head receives only these five numbers. No other path from the image reaches it. That narrow interface is the bottleneck.

A black box maps the image straight to an answer. A strict concept bottleneck model, or CBM, first predicts the concepts, then predicts the task from the concepts alone.

These concept values and this head are written by hand, for teaching.

This model believes the wingtips are grey, at 0.8, and black, at only 0.2. So it ranks glaucous-winged gull first, 0.71 against 0.29.

Look at the photograph: the wingtips are black. An observer corrects the wing colour concepts. Grey becomes 0, and black becomes 1.

The task head is unchanged. It simply recomputes, and American herring gull now receives 0.91.

A person edited an intermediate belief, and the prediction followed.

Compare multi-task learning, where a concept head sits beside the class head.

Editing that concept output changes nothing, because the class head never reads it.

In a concept bottleneck, the concepts lie on the path to the prediction.

The gull is not a random choice. The standard benchmark for concept bottlenecks is a bird dataset: Caltech-UCSD Birds, or CUB.

It has 11,788 photographs of 200 species. Eight of the species are gulls.

People annotated every photograph with 312 yes-or-no attributes, in 28 groups: bill shape, wing colour, wing pattern, eye colour, and so on.

Each attribute is a question with a yes or no answer. Does this bird have a hooked bill? Is its wing grey? Yes is a one, and no is a zero.

Annotators also recorded how sure they were: definitely, probably, guessing, or not visible. About one label in ten is not visible.

Labels on single photos are noisy, so Koh and colleagues took a majority vote within each species. After the vote, every photo of a herring gull has the same labels.

They then kept the 112 attributes that are present in at least ten species.

One training record is a photograph, 112 zeros and ones, and a species.

In symbols, the concept labels c form a vector with 112 entries, each zero or one, and the label y is one of 200 species. The model predicts c hat, a value between zero and one for each concept.

Before building a bottleneck, we need to decide what counts as a concept.

Let us slow down. What exactly is a concept? It is more than a neuron with a name attached: it is a promise that a person can check the claim in the input.

Three things must be in place. The first is meaning: a definition that a domain expert can apply. For example, grey wingtip: the tips of the folded flight feathers are grey, not black.

The second is grounding: where in the input is the evidence? For wingtip colour, it is this region. The estimate should depend on these feathers, and not on the water or the rock.

The third is representation: how the concept becomes a number. The simplest is binary: c j is zero or one.

A soft concept passes a logit, z, through the sigmoid, giving a value between zero and one. A logit of 1.4 gives 0.80.

A soft score of 0.8 is not automatically a calibrated probability. Whether eighty percent of such cases really have grey wingtips has to be checked on data.

Some concepts are categorical. Wing colour takes one value from a list, so its scores form a softmax over the options, and they sum to one.

Others are continuous, such as bill length in millimetres, predicted by regression.

Richer models give each concept a vector, an embedding, or a whole probability distribution. We will meet these in Part 10.

And sometimes the evidence is missing. In CUB, a wingtip hidden behind a rock is labelled not visible. Unknown is not the same as absent, and the representation should be able to say so.

So a concept can be stored in six ways: binary, soft, categorical, continuous, a vector or distribution, and unknown.

The same pattern applies across sensors. In a sleep EEG recording, a concept might be: sleep spindle present.

A spindle is a brief burst of brain waves, about 11 to 16 cycles per second, lasting roughly half a second to two seconds. Sleep scorers treat spindles as a sign of stage two sleep.

In a customer message, a concept might be: duplicate charge reported, meaning the customer says one purchase was billed twice.

Each is a named claim about the input that a specialist could check.

Next, we follow one image from its predicted concepts to class scores and probabilities.

Start with an image. The concept predictor estimates the concepts in it. The task head reads those estimates and produces class scores. Softmax converts the scores into class probabilities. The task head can be a single linear layer or a multilayer perceptron, an MLP. We will use one linear layer so that we can see every calculation.

Our possible classes are zebra, horse and giraffe. For every image, we predict the same three concepts, in this order: stripes, horse-like shape and long neck. These concept names are fixed before we train the model.

The concept predictor, g theta, is a neural network that takes the image as input. For this hand-chosen example, suppose it predicts stripes 0.1, horse-like shape 0.9 and long neck 0.1. These are assumed outputs for this photograph, not measurements from a trained network. Together they form c hat, the predicted concept vector for this image. The hat means predicted.

Now pass that same concept vector to the task head, f phi. It takes the three concept values and returns one score for each class. Its input is the predicted concepts. It does not see the image directly. This is where we could use a linear layer or an MLP.

For our example, choose a single linear layer with no bias. Its weight table, W, has a row for each class and a column for each concept. The zebra row is 3, 1, 0. These are weights, not another image's concept values. We normally learn the weights during training and keep them fixed when predicting new images. Here, both the weights and the concept values are hand-chosen to make the arithmetic easy.

Let us compute just the zebra score. Stripes contributes 3 times 0.1, which is 0.3. Horse-like shape contributes 1 times 0.9, which is 0.9. Long neck contributes 0 times 0.1, which is 0. Add these contributions to get 1.2. This is one output of the linear layer.

Repeat that calculation with the horse row and the giraffe row. The layer returns zebra 1.2, horse 1.6 and giraffe 0.2. These scores are also called logits. Only now do we write the shorthand: scores equal W times c hat. This is just the linear layer applied to the predicted concept vector.

Pass the score vector through softmax. Exponentiate each score and divide by the total. This gives class probabilities that sum to one. Horse has the largest score, 1.6, and the largest probability, about 0.52. The predicted class is horse.

Now an observer looks at the image and sees stripes. They correct the stripes value from 0.1 to 1. The other concept values stay the same. This edit replaces one entry in the predicted concept vector.

Feed the corrected concept vector into the same linear layer. Its weights have not changed. The new scores are zebra 3.9, horse 0.7 and giraffe minus 0.7. The image and the concept predictor are unchanged. We only replace one concept value before running the task head again.

Apply softmax to the new scores. Zebra now has probability about 0.95. The predicted class changes from horse to zebra.

Only the stripes value changed, by 0.9. The zebra weight for stripes is 3. So the zebra score increases by 3 times 0.9, which is 2.7. That takes it from 1.2 to 3.9. The other contributions stay the same.

The full path is still image, concept predictor, concept vector, task head, scores and softmax. A person can correct a concept in the middle, then run the same task head again. The answer can change without retraining either network.

We now compare how the concept predictor and task head learn from their labels.

So far the concept values were given. Where do they come from? The concept predictor learns them from labelled examples.

Each training example carries three things: an input x i, concept labels c i, and a task label y i. For a herring gull in CUB: the photo, zeros and ones, and the species.

The concept loss, L c, compares the predicted concepts with the labels: typically a binary cross-entropy for each concept, averaged.

One term: grey wing has label 1; suppose the predictor says 0.8. The term is minus log 0.8, about 0.22. Closer to 1 costs less.

The task loss, L y, is typically a cross-entropy over the classes: minus the log of the share given to the correct class.

A weighted sum combines the two; lambda c and lambda y set how much each counts.

In the original paper, Koh and colleagues compared three ways to train.

Independent: fit the predictor to the concept labels, and separately fit the head on the true concepts. At test time, though, the head receives predicted concepts.

Sequential: fit the predictor, freeze it, then fit the head on the concepts it predicts. The head learns from the predictor's actual mistakes.

Joint: train the whole system at once, with both losses.

In joint training, the task loss sends its gradient back through the head and the concept values into the predictor, which is then shaped by what helps the task.

That can improve task accuracy. It can also hurt concept fidelity, if a concept score drifts toward whatever helps the classifier. That has to be measured.

The experiment we will meet later used joint training, with the concept loss weighted 2 and the task loss weighted 1.

Part 5 described three training schemes. Does the choice matter on real photographs? We ran a small version on CUB.

Each photo goes through a frozen CLIP image encoder, giving 512 features. A linear concept predictor maps them to 112 concepts, and a linear task head maps those to 200 species.

For our gull photo, the predictor is sure the throat is white and the wings are grey. The head spreads its probability over several gull species.

Joint training is the most accurate, close to a standard model with no bottleneck. Sequential falls in between, and independent training is the least accurate.

Now intervene. Pick a few attribute groups, replace their predicted values with the true values, and let the same head recompute.

The independent model improves fastest. Its head was trained on true concepts, so it trusts corrected ones. The joint model improves least.

Next, a person corrects a concept and we follow the effect on the prediction.

Concepts are useful because a person can check them. When the model gets one wrong, someone who can see the mistake can correct it, with no retraining.

An intervention has four steps. The model predicts concept values. A person checks one and corrects it. The fixed task head recomputes. A new prediction comes out.

Back to the gull. Wing colour is a categorical concept, so the observer corrects the group together: grey to 0, black to 1.

Changing only one of them could leave an impossible state, with both colours on, or neither.

The head's score for American herring gull moves from 0.29 to 0.91.

The same mechanism works on a signal. Here is a schematic EEG trace, drawn for teaching rather than recorded from a patient. Near the middle is a short burst at about 13 hertz, the shape of a sleep spindle.

The concept predictor gives spindle present only 0.12. K-complex present gets 0.8, and slow-wave dominance 0.15.

With these values, an illustrative stage head favours other stage, at 0.62, over an N2 candidate, at 0.38.

A sleep expert inspects the trace and marks the spindle as present. 0.12 becomes 1.

Same head, new input. The N2 candidate rises to 0.86.

This demonstrates the mechanism, not a clinical result. Real sleep staging applies scoring rules to thirty-second epochs, and this head was written by hand.

A correct concept edit can still hurt the final prediction, so we need to understand how the head uses it.

So far, every correction has helped. Does a correct edit always help? Here is a case where it does not.

A support message reads: The app works, but I was charged twice. Please refund the extra payment. It belongs in the billing queue.

The concept predictor rates duplicate charge at only 0.2, and refund requested at 1.

The task head computes a billing score. It starts at 1.2, takes away 2 for each unit of duplicate charge, and adds 0.6 for each unit of refund requested.

In symbols, the score s is 1.2, minus 2 times duplicate charge, plus 0.6 times refund requested. A sigmoid turns that score into the billing share.

Before any edit, the score is 1.2, minus 0.4, plus 0.6, which gives 1.4. Billing receives 0.80, so the message is already routed correctly.

A reviewer reads the message and correctly sets duplicate charge to 1.

Now the score is 1.2, minus 2, plus 0.6, which gives minus 0.2. Billing falls to 0.45, and the message goes to general support.

The edit was correct. The fault lies downstream: this head learned a negative weight for duplicate charge, the wrong sign for billing.

Correcting a concept does not repair an incorrect decision rule. An intervention is only as good as the head that receives it, so its effect has to be measured rather than assumed.

This head was built faulty on purpose, to show that the failure can happen. It says nothing about how often it does.

What happens when the concept vocabulary leaves out information needed for the task?

A second limit sits in the vocabulary itself: the bottleneck passes on only what we chose to name.

Take two objects, a blue circle and a blue square. They belong to different classes.

Suppose the concept vocabulary has only two entries: colour and size. Both objects are blue. Both are small.

So both map to exactly the same concept vector. c of x 1 equals c of x 2, and yet y 1 differs from y 2.

Watch what reaches the task head. Shape was present in the image. At the bottleneck, it disappears.

Any task head that reads only this vector must give both objects the same output. No head, linear or a large neural network, can recover a distinction that never reached it.

On these two examples, the best possible accuracy is fifty percent.

The ideal property is called sufficiency. Once the concepts are known, the input says nothing more about the label.

In symbols: Y is independent of X, given C. Here it fails: the label depends on shape, which C lacks.

A strict CBM guarantees something weaker. Its own prediction depends on the input only through the concepts. Whether the true label does is a property of the chosen vocabulary.

More accurate colour and size estimates would not help. The fix is a richer vocabulary: add shape. Now the two vectors differ.

We now compare newer architectures by the supervision they need, from concept labels to class labels to no fitting on the new task.

The original concept bottleneck needs a person to label every concept on every training image. That is its main cost, and most newer methods try to reduce it.

One supervised training record looks like this: the image, a 0 or 1 for each named concept, and the class. On CUB, that meant 112 concept labels per photograph.

The newer methods form a ladder of shrinking supervision. On the first rung, CBMs and concept embedding models need the image, the concept labels and the class.

Our post-hoc phrase example drops the concept labels on the task images. It still learns from images with class labels. A pretrained model supplies visual knowledge, and concept names can come from people or a language model.

Classification by description drops training on the new task. It still uses a pretrained model. We supply class descriptions and a new image, without fitting a model to labelled examples of these classes.

Use our CLIP phrase example for rung two. A language model can suggest descriptions, such as black and white stripes, which we review. Pretrained CLIP compares each image with these descriptions and produces match scores. During training, class labels teach a task head how to combine those scores. For a new photo, the learned head predicts the class. We need class-labelled images, but no person has to label every concept in every image. The diagram illustrates the path, without showing measured scores.

Now keep the pretrained CLIP model, but replace the learned head with a fixed rule. A language model supplies descriptions for each class. CLIP compares a new photo with each description. We average the match scores within each class and choose the class with the highest average. No class-labelled examples or fitting are needed for this task. The difference is the combination: rung two learns it from class labels; rung three uses a fixed average.

Dropping per-image concept labels saves labelling, but removes a direct check. A person who supplies a concept label has checked it against the image. On the higher rungs, a match score alone does not establish that the named concept is visible. We need to check this another way.

So for each variant, watch for one question: who checks that a named concept is really visible in the image? We come back to this question in the recap and evaluation.

Newer variants change four things: where the concepts come from, how each concept is detected, how it is represented, and what reaches the task head.

We will walk through them on one running example: photographs of zebras, horses and giraffes, the classes from Part 4. We train on 24 photos and test on 12.

For each variant, we first say in one line what changed compared with the plain CBM. Then we write down how the path changes: from x, the image, to c hat, the concepts, to y hat, the class.

On rung one, every training image has concept labels and a class label; concept embedding models keep this supervision but give each concept a richer representation.

First, concept embedding models, or CEMs. One number per concept is a narrow channel, and when the vocabulary is incomplete, accuracy suffers, as we saw in Part 8.

A prototype usually means a representative example. Here the clearer term is embedding: a short list of learned numbers. CEM computes a present vector and an absent vector for the current image. Both exist even when stripes are clearly visible. They are not stored example photos or fixed class averages. The coordinates are learned features, not individually labelled properties. These numbers are illustrative.

Where do the vectors come from? An image encoder first extracts features. For stripes, two learned layers turn those features into the present and absent vectors. Every concept has its own pair of layers. Training learns the weights of these layers. With those weights fixed, a different image can still produce different vectors. Nobody supplies target values for the vector coordinates.

Now the mechanics. An embedding is a short list of numbers that describes something; here each one has two numbers. From the image, the model computes two embeddings for every concept: one describing it as active, one as inactive.

A learned scoring layer reads both vectors and predicts the probability, p, that the concept is present. Suppose that, as in Part 4, it misses the stripes on this photo: p is only 0.1. The concept labels will teach this probability during training.

The concept's embedding mixes the two: p times the active embedding, plus one minus p times the inactive one.

With p equal to 0.1, take the first entry: 0.1 times 0.82, plus 0.9 times 0.22, gives 0.28. The second entry works the same way, and the mixed vector sits close to inactive.

At test time, only the new image is needed. The trained model computes both vectors and a probability for every concept, then mixes them. The head reads all three mixed vectors, six numbers in all. Our illustrative linear head produces class scores, and softmax gives the displayed probabilities. Here it favours horse. All weights stay fixed, and no true concept or class label is used for this ordinary prediction.

To intervene, a person sets p, not the vector coordinates. Setting p to 1 swaps in the active embedding already computed for this image. The same head now makes zebra lead in this example. The two candidate vectors and all network weights stay fixed; there is no retraining.

During training, each image comes with concept labels and a class label. For this zebra, stripes is labelled present and the class is zebra. A concept loss compares the predicted probability with the concept label. A task loss compares the class prediction with the class label. Backpropagation from these losses updates the encoder, the vector-generating layers, the probability scorer and the task head together. We supervise the probability and the final answer, rather than supplying the vector coordinates.

Compare the two stages. The input during training is an image with concept and class labels. At test time it is just a new image. The weights are updated during training and kept fixed at test time. The vectors and probabilities are computed from each image in both cases.

In symbols, for concept j out of k: g now outputs a probability and two embeddings, each m numbers long. c hat j is the mixed vector, and f reads all of them. Training still needs images, concept labels and class labels.

On rung two, the image and the saved concept phrases go through frozen CLIP encoders. Their cosine similarities become the concept scores, with any saved scaling applied, and a learned class head combines them. Class labels train the head, while the CLIP encoders remain fixed. We will follow one post-hoc example from the phrase bank to a prediction.

We now build a post-hoc concept bottleneck model, or PCBM, using frozen CLIP encoders. Start with the animal classes: zebra, horse and giraffe. An optional helper is a large language model, or LLM. Ask it to suggest visible properties that help distinguish these animals, without using their class names. It might suggest black and white stripes, a long neck and a flowing mane. This exchange is illustrative: the LLM proposes a vocabulary, rather than classifying our photos.

Review the suggestions before fitting. Keep clear visible properties. Reject zebra because it is the class name, not a property. Merge striped coat with black and white stripes because they repeat the same idea. These are illustrative review examples. Check phrase scores across the training photos too; remove phrases that never give useful matches. For the saved animal experiment, we keep the original bank of eight hand-chosen phrases. An LLM could help propose such a list, but these saved results do not come from an LLM-generated bank. Every image will be scored against the same list, including phrases that do not describe it. Freeze the list before fitting the task head.

Take this zebra photo and the phrase black and white stripes. CLIP's frozen image encoder gives an image vector, and its frozen text encoder gives a phrase vector. Compare these vectors using cosine similarity. For this real photo and phrase, the score is 0.274. This is a match score, not the probability that stripes are present. We reuse CLIP as it is; neither encoder is trained on our animals.

Keep the zebra photo fixed. Repeat the cosine comparison for every phrase in the bank. The result is eight named scores, in a fixed order. These scores are not probabilities. Next, put each score on a common training scale, then learn a task head that predicts the animal class. The LLM is not needed when scoring a new image.

Keep the prompts fixed and change only the image. Here are three held-out photos: a zebra, a horse and a giraffe. Each column uses exactly the same phrase for every photo. Each cell is the raw cosine similarity between that photo and that phrase. This gives one score per image and concept. These scores are not probabilities.

Each prompt has its own usual score and its own spread. To find them, score all 24 training photos with the same prompts. Compute the average and standard deviation separately down each column, across images. Save those training statistics and reuse them for every new photo. The three test photos on the previous slide do not set the scale.

Each cell is standardised separately. Take the zebra and stripes cell. Subtract its average over the 24 training photos, then divide by its spread, the standard deviation. For stripes, that gives 1.41.

Now apply the same rule to every cell, using its column's saved mean and spread. The photos and prompts have not changed; only the units have changed. The zebra's stripes score becomes 1.41, while the giraffe's long-neck score becomes 1.70. A positive value means above that prompt's training average, zero means at the average, and a negative value means below it. A negative score does not prove that the concept is absent. Within each column, the order of the images stays the same. The head receives these standardised values.

These eight numbers are the concept bottleneck. Each belongs to a named phrase. For this zebra photo, black and white stripes scores highest, 1.4 standard deviations above its average on the training photos.

Now use the task training photos, each with its animal class label. The frozen encoder and concept bank turn each photo into eight scores, using the saved training scale. Feed those scores to a linear head. Compare its prediction with the true class and update only the head weights and constants. The encoders, phrase vectors and scaling stay fixed. This learns which named concepts help distinguish zebra, horse and giraffe.

The head is also sparse. An L1 penalty adds the size of every weight to the loss, so weights that do not help are pushed to exactly zero. Of 24 possible weights, 8 concepts times 3 classes, only 4 are non-zero: zebra reads stripes, giraffe reads a long neck, and horse reads a flowing mane.

For this zebra, only the stripes weight feeds the zebra score: the stripes score times its weight, plus the constant, gives about 2.03. The softmax turns the three scores into shares, and zebra gets 0.77.

At test time, give the system a new image. The frozen image encoder produces its features. Cosine similarities with the saved phrase vectors give raw concept scores, and the saved training statistics put the scores on the same scale. The trained head produces class scores; softmax gives probabilities. No concept labels or class label are supplied, and nothing is fitted again. This is the complete prediction path.

All 12 held-out photos are classified correctly. This is a tiny teaching example, not a benchmark.

Lower the stripes score to its usual value on other animals, and zebra falls to 0.14.

The concept list may miss useful evidence in the image. The hybrid post-hoc concept bottleneck model, or PCBM-h, adds a residual: an extra linear head that reads the same image features directly. Add its class scores to the concept head's scores, then apply softmax. For our zebra, the score is about 2.03 plus 0.31, giving 2.34 before softmax. During training, keep the concept model fixed and fit only the extra head using class labels. At test time, both paths read the same image. This can improve prediction when the concept list is incomplete.

Now lower the stripes score, just as before. The concept head sees the edit, but the residual still reads the same image features, so its contribution stays unchanged. After the same edit, the concept model gives zebra probability 0.14; with the residual it is 0.20. The prediction can still change, but some evidence bypasses the concepts. The trade-off is better predictive capacity with less control through named concepts.

Compared with the plain CBM, three of the four things changed. The concepts come from a reviewed bank of phrases instead of per-image concept labels, frozen CLIP encoders and cosine similarity replace the trained predictor, and in the hybrid a residual also reaches the head. Each concept is still one number.

In symbols, for concept j out of k, compare the image vector e with the saved phrase vector v j using cosine similarity. Standardise each score using its saved training mean and spread to obtain the concept value. The encoders and phrases are frozen. The task head f is sparse and linear. Only f is trained from images and class labels, with an additional residual r in the hybrid.

On rung three, a pretrained model scores descriptions and a fixed rule combines the scores, without fitting a head to labelled examples from the new task.

We just used class labels to learn a head on CLIP scores. The last step on the ladder removes that fitting on the new task. The pretrained CLIP encoders remain. Menon and Vondrick call this classification by description, or CBD in our comparison table. Use a fixed average of description scores for each class instead of learning how to combine them.

The intuition is a checklist. A language model writes a short checklist for every class. CLIP ticks each item softly, with a score for how well the photo matches it. The class with the best average tick wins.

A language model writes a few descriptors for every class: short phrases about what the class looks like. For a herring gull: a white body with black wingtips, a yellow beak and legs, a red spot on its beak.

CLIP scores the image against each descriptor: that score is the soft tick. The class score is simply the average of its descriptors' scores, and the highest average wins.

Where does a score come from? CLIP has two encoders. One, phi, turns the image into a vector; the other, psi, turns the descriptor into a vector in the same space. The score is the cosine similarity of the two: near one when they point the same way, smaller when they do not.

Then y hat k, the score of class k, averages the descriptor scores over D k, the class's list of descriptors. Nothing is fitted to the target data.

Every prediction comes with its reasons: the descriptor scores that were averaged. On our gull photograph, the five herring gull descriptors score between 0.32 and 0.34, and their average is about 0.33.

But the slaty-backed gull's descriptors average slightly higher, so it edges ahead by a tiny margin. Our photo shows a herring gull, so this prediction is a near miss.

So when is this attractive? When you have no labelled images at all. And a new class needs only a new description: write its checklist, and it can be scored at once, with no retraining.

But the descriptors themselves can be wrong: the language model says a California gull has a black head, which it does not.

Look at our herring gull photo: its head is white and its bill is yellow. The California gull checklist says black head and bill, yet CLIP gives that phrase a positive similarity score. That number compares the photo with the whole phrase; it is not a probability that each detail is true. The fixed average still includes it, so a misleading description can affect the class score. Other descriptions may be impossible to verify from a photo, or describe its background. The takeaway: check that the words are correct and visible, then test whether changing that feature changes its score.

In our rung-two phrase example, the head was a linear layer trained from class labels. Here the class score is a fixed average of description scores, so nothing is trained and no labelled image is needed. The main change is how we combine the CLIP scores.

In symbols: g is CLIP, c hat d is the similarity to descriptor d, and f is a fixed average over each class's descriptors. The data: none. The cost: nothing guarantees that the descriptors are correct, or grounded.

Use our supervision rungs to organise the examples. This table puts them beside the original CBM, one row per method, and we fill it one column at a time.

The first column is the data needed for training. Here x is an image, c its concept labels, and y its class label. Supervised CBMs and concept embedding models need all three.

Post-hoc models drop the concept labels on the task images. Classification by description, CBD in the table, needs no training data for the new task.

The second column says who supplies the concepts. For CBMs and concept embedding models, people do, through their labels. Our post-hoc phrase example uses a reviewed concept bank. Classification by description uses descriptions from a language model.

The third column is what the task head actually reads. A CBM passes one number per concept, a concept embedding model a vector per concept, and our post-hoc phrase example its scaled cosine scores. The optional hybrid version adds a residual to the class scores. Classification by description averages raw CLIP scores.

The last column is what gets trained. CBMs and concept embedding models train the concept predictor and the head. Post-hoc models train only the head. Classification by description trains nothing on the new task.

One question sums up the table: who supplies the concepts, and who checks them? In CBMs and concept embedding models, people label the concepts, so people check them. In post-hoc models, a concept bank supplies them, checked only by whoever built it. In classification by description, a language model supplies the descriptions and there is no automatic factual check. We must review them.

This is the general trade-off. Each rung up the ladder removes a human labour cost, and with it a human check. The labour saved on labelling has to be spent later, on checking that the concepts are really in the image.

How should we judge a concept bottleneck? We need to check the concepts, the predictions and the effect of interventions.

Keep four questions in mind. Task accuracy: is the final answer right on held-out images? Concept quality: are the named values correct, and based on the intended image evidence? Faithfulness: does the prediction actually use those concepts? Intervention utility: do corrections improve decisions, and at what cost in human time and mistakes?

Here is one measured example from a synthetic-shape experiment. Colour, shape and size determine the class. This image is a small red square. Before correction, large scores 0.78, and the head predicts large red square. Set only large to 0. The same trained head now predicts small red square. No retraining is needed.

Compare two models on the same test images. The multi-task model predicts concepts on a separate branch. Its class head never reads them, so a concept correction leaves accuracy at 82.0 percent. In the strict CBM, the head reads only concepts. One correction per image raises accuracy from 81.6 to 97.3 percent. Here, an oracle uses the true labels to correct the largest concept error first. These results use one training seed and 512 synthetic test images, with perfect corrections. They do not measure human effort or mistakes. Similar accuracy can hide very different intervention utility.

The bottleneck idea also applies to other kinds of input, provided the intermediate concepts have a useful meaning.

So far our inputs were photographs of birds. Concept bottlenecks fit wherever experts already reason with named findings, the things they check and write down by name.

In medicine, Koh and colleagues also trained a CBM on knee X-rays from the Osteoarthritis Initiative. The model first scores ten clinical findings, such as joint space narrowing and bone spurs, each on a small scale of its own. From those, the head predicts how severe the osteoarthritis is, on four levels from 0 to 3.

A radiologist can correct a finding, and the grade is recomputed. Here, joint space narrowing goes from 1 to 2, and the grade goes from 2 to 3. Radiologists already write their reports in these terms.

In language, the CEBaB benchmark annotates restaurant reviews with four concepts: food, service, ambiance and noise. In this review the food is praised and the service is criticised, while ambiance and noise are not mentioned. Text bottleneck models predict the star rating from those concept values, so you can see which aspect drove it. Reviewers already write this way.

In intensive care, Wu and colleagues built concept bottlenecks on patients' time series, measurements recorded over time. The task is vasopressor onset: when a patient will need vasopressors, drugs that support blood pressure. The concepts are summaries learned without concept labels, close to how doctors already sum up a patient, in terms like illness severity and kidney function.

In wearable sensing, X-CHAR explains a complex activity through the simpler activities it is made of, which a person can recognise in the sensor signal. In its nurse data, the simple activities include check vitals, collect blood and oral care. People already describe a complex activity as the simple steps it is made of.

And in sleep staging, the concepts are the patterns that scorers already use, as in Part 6: spindles, K-complexes and slow waves. The head turns them into a sleep stage, such as stage two. Scorers already mark these patterns by hand, so each concept matches a mark they would make.

The recipe is the same each time. Name the findings an expert checks, predict them from the raw signal, and let the decision depend on them alone.

An editable concept interface is useful, but we still need to check how it behaves and what its scores mean.

Readable concepts in the middle do not guarantee a good model. Let us follow the path from the photo x, through the concept scores c hat, to the answer y hat, and ask what can fail at each of five points.

First, distribution shift: the encoder meets data unlike its training data. In symbols, new inputs come from a different distribution than the training inputs. Recall the chest X-ray model from Part 1.

Its AUC, a score where 1 means perfect ranking and 0.5 means guessing, was about 0.93 on X-rays from the hospitals it learned from, and about 0.82 at a new hospital. In practice, the model looks fine where it was built and does worse at the new site.

Second, wrong grounding: the concept detector responds to the wrong evidence, such as the water behind a gull instead of its wingtip. In practice, the score looks right for the wrong reason, until a gull turns up away from water.

Third, missing information: the vocabulary leaves out what the task needs. In Part 8, a blue circle and a blue square shared one concept vector, so the best any head could do was guess, right half the time. In practice, two cases an expert would tell apart look identical to the head.

Fourth, the task head can learn the wrong relationship. Recall the billing head from Part 7: it takes away 2 for each unit of duplicate charge, the wrong sign. A correct edit sets duplicate charge to 1, and billing falls from 0.80 to 0.45. In practice, the expert's fix is right, yet the decision moves the wrong way.

Fifth, the intervention can be wrong, or too costly. On this chart from Part 11, each step to the right is one more correction per image, and each correction takes an expert's time. Ours came from true labels, an expert who never errs. In practice, a busy expert can misjudge a concept too, and every check uses time they could spend elsewhere.

So ask five questions of any concept bottleneck. Are the concepts meaningful? Are they grounded correctly? Are they sufficient for the task? Does the model actually use them? And do corrections improve decisions?

Knab and colleagues' 2026 survey organises the field around the same parts: the input, the concepts, the output and the training. It lists these issues among its open challenges.

We finish by following the whole path again, from an image through concepts to a prediction and an intervention.

One last time, the gull. A black box maps the photograph straight to a species.

Open it up. Input. Encoder. Concept predictor. Named concepts. Task head. Prediction.

A person sees that the wingtips are black and corrects that concept. The head now says American herring gull, at 0.91.

But the interface is only as good as its parts: the concepts it names, where they look, and the head that reads them. A concept may be missing, a detector may look at the water, and a head may hold a wrong rule.

In short, concept bottleneck models put named values that people can read in the middle of a prediction. Those values can be inspected, evaluated, and sometimes corrected.

The challenge is to make sure the interface faithfully represents the information on which the prediction depends. Put simply, whatever the answer uses should be visible in the concepts.

Four things to remember. Post-hoc methods explain a model after the fact, while a concept bottleneck makes the answer pass through named concepts. Fixing a concept helps only if the head reads it the right way. A soft score can carry more than its name says, and that extra information leaks to the head. And newer variants find concepts with fewer labels, so check that each one looks at the right evidence.

The appendix introduces testing with concept activation vectors, a way to ask whether a black-box model responds to a named concept.

This appendix adds one more post-hoc method, one that asks about concepts directly. Testing with concept activation vectors, or TCAV, asks a question like this one: does the model use grey wings when it says herring gull? Here the model is CLIP, the black box from Part 1.

The first step is to collect example photos: some birds with grey wings, and some without, here from a labelled bird dataset. None of them is a herring gull.

Pass every photo through the model and look at its internal activations. Fit a line that separates the two groups. The direction across that line is the concept activation vector: moving along it means more grey wing.

Now take a herring gull photo and ask: if its activation moved a little in the grey wing direction, would the herring gull score go up? The gradient of the score, times the direction, gives the answer as a sign.

The TCAV score is the share of herring gull photos where the score goes up. For grey wings, it went up for almost all 60 photos, in each of 10 repeats with different example photos.

A direction that separates two random sets of photos gives scores that jump between zero and one. So a concept counts only when its score is steady and different from random ones. And even then, TCAV says what the model responds to. It gives no way to correct it.

This final appendix is optional further reading, outside the main lecture. A concept name and a high score do not establish that the concept is present in an image. We revisit this issue, explain the random-layer control, and explore training from concepts located in image regions. This is an extension of rung two, using detector targets and class labels.

This optional appendix explores grounding the concepts. Recall the weakness of using CLIP similarities as concept values: a phrase gets a score, but nothing locates the named concept in the image. We need a check that the concept is actually visible. The following slides show how a detector can supply this spatial check during training.

The fix rests on a simple intuition. Naming a concept is not the same as pointing at it. So ask a detector to point at each concept in the image, and count only the concepts it can find.

There are three ways a concept can fail. Some are not visual, such as a gull's call. Some describe the setting, not the bird. And some score highly on images that do not contain them, such as a black head on a gull whose head is white.

Here is a control experiment. A pretrained image encoder already extracts useful features. Pass them through a layer with randomly chosen weights, keep that layer fixed, and train only the final class head using class labels. The layer was never taught to detect yellow bills or other named concepts. When the head can use enough of these mixed features, reported experiments show that class accuracy can approach concept-based models. So good predictions do not establish what each intermediate value means. Random describes how the weights are chosen; fixed describes whether they change.

Zoom in on the random layer. Each weight says how much of an image feature to add or subtract. Random means drawing these coefficients by chance, without learning them from labels. In this illustrative example, the encoder gives features 2 and 1. Suppose the weights are 0.4 and minus 0.2. Multiply and add: 0.8 minus 0.2 gives 0.6. This is a mixed image feature, not an estimate of yellow bill. In this baseline, we draw the weights once and leave them untrained.

Fixed, or frozen, means keeping the weights unchanged throughout training. The encoder keeps its pretrained weights, and the random layer keeps its one draw. Only the class head adjusts its weights using class labels. Fixed weights do not mean fixed outputs. Photo A has features 2 and 1, which give 0.6. Photo B has features 1 and 2; the same weights give 0. The head can learn from these differences, even though neither value has a guaranteed concept meaning. Check the concepts themselves as well as class accuracy.

Their method is the Vision-Language-Guided Concept Bottleneck Model, or VLG-CBM. It grounds the concepts first. It uses an open-vocabulary object detector: a detector that can search for any phrase you give it, not only a fixed list of objects. It looks for each concept in each training image, and draws a box where it finds it.

A concept counts as present only where a box was found. For our gull, the bill, head and wingtips each get a one, and the call gets a zero. We call this vector c det: the concept labels from the detector.

These grounded labels train the concept layer: a linear layer that turns the backbone's features into one score per concept. The loss is a binary cross-entropy, as in the original CBM, but no person draws the labels.

One term, as in Part 5. The wingtips have a box, so their label is one; if the layer predicts 0.8, the term is minus log 0.8, about 0.22. The call has no box, so its label is zero; a prediction of 0.2 costs the same, minus log of one minus 0.2.

They also report the number of effective concepts, NEC for short. Each row of the head holds one class's weights, one per concept. Count the non-zero weights in each row, here three, two and four, and average over the classes: the NEC is three.

Why does this count matter? A head that puts weight on every concept can fit almost anything. The paper proves that even a random, untrained concept layer can approximate any linear classifier once there are enough concepts. So a dense head can be accurate and still explain nothing.

So the fair comparison gives every model the same budget: the same number of concepts per class. Only then does higher accuracy say something about the concepts themselves.

In symbols, the count for class k is the zero norm of its row of weights: the number of entries that are not zero. The NEC averages that count over the K classes.

The paper reports accuracy with the NEC held at five. A head like that can actually be read: each class rests on a handful of named concepts.

Compared with direct CLIP scoring, the concept values now come from a learned layer. Detector boxes supply its training labels. At prediction time, this layer produces the concept values instead of comparing the image with text. The class head is still learned from class labels. We also judge it at a fixed number of concepts per class, so the explanation stays short.

In symbols: g is a backbone with a concept layer, W c, trained on the detector labels c det. f is sparse and linear, with the number of effective concepts held fixed. The data: images and class labels, plus a detector.