Where We Are in the Course

Week 7-8:   Build and tune the model      → Is it good?
Week 9:     Reproducibility                → Can someone re-run it?
Week 10:    Data drift                     → Is it STILL good?
Week 11:    Profiling & Quantization       → Is it FAST and SMALL?
Week 12:    Agents from Scratch            → Can it DO things?   ← today

Weeks 7-11 were about building and shipping a model that answers questions.

Today we build something that takes actions.

A Simple Question, A Hard Answer

You ask an LLM: "What's the weather in Gandhinagar right now?"

The LLM replies:

"I don't have access to real-time data, but Gandhinagar typically
experiences temperatures around 35-40°C in April..."

It guessed. It didn't check. Why?

An LLM is just a text predictor. It can only generate the next word.
It can't open a browser, call an API, run code, or check a database.

What If It Could Call an API?

Imagine the LLM could say:

"I need to check the weather. Let me call the weather API."

Then it produces this:

{"tool": "get_weather", "arguments": {"city": "Gandhinagar"}}

Your code executes that, gets back {"temp": 38, "condition": "Sunny"},
and feeds it to the LLM, which then says:

"It's currently 38°C and sunny in Gandhinagar."

That's an agent. An LLM that can use tools.

What Is an Agent?

An agent = LLM + tools + a loop.

The LLM decides whether to use a tool. You execute the tool.
The result goes back to the LLM. Repeat until done.

You Already Use Agents Every Day

In every case: an LLM decides what action to take, your device
executes it, and the result goes back to the LLM for the next step.

Deep Dive: What Does Claude Code Actually Do?

You type: "Fix the failing test in test_api.py"

Claude Code doesn't magically know the answer. Here's what the agent loop does:

Seven steps, four different tools, zero human intervention. The LLM
decided what to do at every step based on what it saw.

Exercise: Trace the Agent Loop (2 minutes)

Pick one of these prompts and write down what tools an agent would need
and what steps it would take. Share with your neighbour.

There's no single right answer — the fun part is thinking about which
tools and in what order. This is what AI engineers design every day.

Three Things That Make an Agent

Without tools, the LLM can only talk. With tools, it can do.

Without a loop, it can do one thing. With a loop, it can do
multi-step tasks — like "find the cheapest flight, then book it."

The "Restaurant" Analogy

Without tools (plain LLM):

You ask the chef: "What's a good recipe for biryani?"
The chef tells you the recipe from memory. Helpful, but you still need
to cook it yourself.

With tools (agent):

You ask the chef: "Make me biryani."
The chef checks the fridge (tool: inventory lookup), orders missing
spices (tool: delivery API), sets the oven (tool: oven control),
and serves you the biryani.

The chef (LLM) still has the knowledge. The tools let it act on that knowledge.

The Key Insight

Function calling is NOT the model running code. The model never executes
anything. Here's what happens:

The model is a decision maker. You are the executor.

Defining a Tool = Writing a Menu Item

Every tool has two parts: a Python function and a description (JSON schema).

# The function (what actually runs)
def get_weather(city: str) -> str:
    data = {"Gandhinagar": {"temp_c": 38, "condition": "Sunny"}, ...}
    return json.dumps(data.get(city, {"error": "unknown city"}))

# The description (what the model reads to decide)
weather_tool = {
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"]
        }
    }
}

The model never sees the function code — only the description.

Use Case 1: Calculator — LLMs Can't Do Math

Ask an LLM "What is 4729 times 8314?" and it will confidently give the
wrong answer. Token prediction is not arithmetic.

Fix: a calculate(expression) tool that calls Python's eval().

Every time an LLM needs exact arithmetic, it should call a tool instead
of computing in its "head."

Use Case 2: Weather — LLMs Have No Real-Time Data

An LLM's training data has a cutoff date. It can't tell you today's
temperature any more than a textbook can.

Fix: a get_weather(city) tool backed by mock data (or a real API).

Ask: "Should I carry an umbrella in Bangalore today?"

Model calls get_weather("Bangalore") → {"condition": "Rainy", "humidity": 65}
→ "Yes — it's rainy with 65% humidity."

The model used actual data instead of guessing from training statistics.

Use Case 3: Unit Converter — Precision Matters

Is 100°F hot or cold? Most people need to convert to answer that.

Fix: a convert_units(value, from_unit, to_unit) tool with 10 conversion pairs.

Use Case 4: Course Notes Search — Your Own Knowledge Base

"What week did we cover data drift?" — the LLM doesn't know your course.

Fix: a search_notes(query) tool backed by a dict of CS 203 topics.

topics = {
    "data drift":   "Week 10: KS test, PSI, distribution shift",
    "profiling":    "Week 11: cProfile, timeit, bottlenecks",
    "quantization": "Week 11: INT8, ONNX, model compression",
    "docker":       "Week 10: containerization, Dockerfiles",
    "agents":       "Week 12: tool calling, Gemma 4, agent loop",
    ...
}

This is a tiny RAG system: retrieval (search the dict) + generation
(model writes a natural-language answer from the result).

All Four Tools Together — The Model Picks the Right One

The model reads the descriptions and chooses which tool fits the question,
or answers directly if no tool is needed. You'll try all of these in the notebook.

The Agent Loop in Pseudocode

messages = [user's question]

repeat (up to max_steps):
    response = model.generate(messages, tools=...)

    if response has NO tool call:
        return response as final answer     ← done!

    for each tool_call in response:
        result = execute(tool_call)         ← YOUR code runs
        append result to messages

    go back to top of loop                  ← model sees the result

That's it. ~10 lines of real Python. You'll write it yourself in the notebook.

Multi-Step Example

"What's the square root of 144 plus the temperature in Mumbai in Fahrenheit?"

Four tool calls, three different tools, one coherent answer. The model
planned what to do — you just executed each step.

Agents in the Wild (2025-2026)

Every one of these is the same pattern you just learned:
LLM + tools + loop. The tools are more powerful, the models are bigger,
but the architecture is identical.

MCP — The Universal Tool Standard

Today you defined tools as Python dicts. But what if you want to share tools
across different LLMs and applications?

MCP (Model Context Protocol) is an open standard from Anthropic that
lets anyone publish tools that any agent can use:

┌──────────┐     ┌──────────┐     ┌──────────┐
│  Claude  │     │ Gemma 4  │     │  GPT-5   │
└────┬─────┘     └────┬─────┘     └────┬─────┘
     │                │                │
     └────────┬───────┘────────┬───────┘
              │                │
         ┌────┴────┐     ┌────┴────┐
         │ Weather │     │  Slack  │
         │  MCP    │     │  MCP    │
         │ Server  │     │ Server  │
         └─────────┘     └─────────┘

One tool definition, many LLMs. Like USB-C for AI tools.

The Agent Stack — A Mental Model

Layer 4:  Application      Claude Code, Cursor, Devin, your app
                           ↑
Layer 3:  Agent framework  The loop + memory + planning
                           ↑
Layer 2:  Tools            Calculator, search, file I/O, APIs
                           ↑
Layer 1:  LLM              Gemma 4, Claude, GPT, Llama
                           ↑
Layer 0:  Hardware          Your laptop, Colab T4, cloud GPU

Today you built layers 1-3 from scratch. Layer 4 is what you build
when you put it all together into a product.

The entire field of "AI engineering" in 2025-2026 is about making this
stack reliable, fast, and safe.

Guardrails: When Agents Go Wrong

Agents can be powerful but also unpredictable. Real-world agents need
guardrails:

Our max_steps=5 parameter is the simplest guardrail. Production agents
add human-in-the-loop confirmation, sandboxing, and cost limits.

Key Takeaways

1. An agent = LLM + tools + loop. The LLM decides, you execute.

2. Function calling is how the model requests tool use — it produces
   structured JSON, not free-form text.

3. Tools are just Python functions with a description. The model
   reads the description and decides when to use each one.

4. The agent loop is ~10 lines: generate → check for tool call →
   execute → feed result back → repeat.

5. Gemma 4 E2B runs on a free Colab T4 with 4-bit quantization and
   has native tool-calling support.

6. This is the architecture behind Claude Code, Cursor, Devin,
   Perplexity, and every other AI agent you use. Same pattern, bigger tools.

What to Try in the Notebook

Open the notebook in Colab