Where We Are

Today: The foundation everything else builds on!

Today's Big Questions

1. What makes ML different from regular programming?
2. What are the different ways machines can learn?
3. How do we represent data for computers?
4. How do we know if our model is good?

Traditional Programming

You provide data and rules → Computer gives answers

Example: Spam Detection Rules

Problem: Determine if an email is spam

Your Rules:
  IF email contains "FREE MONEY" → Spam
  IF email contains "You've won" → Spam
  IF sender is unknown AND has attachment → Spam
  ...

You must think of every possible pattern!

The Problem with Rules

What about these?

Spammers adapt. Rules break.

The ML Approach

You provide data and answers → Computer learns the rules!

ML in Code

# You provide examples (data + answers)
emails = ["FREE MONEY!!!", "Meeting at 3pm", ...]
labels = ["spam", "not spam", ...]

# Computer learns patterns
model.learn(emails, labels)

# Now it can classify NEW emails
model.predict("FR33 M0N3Y")  # → "spam"

The model learned to handle variations it never saw!

The Key Difference

When to Use ML?

ML is great when:

ML is NOT needed when:

• Rules are simple and fixed (calculator)
• Data is scarce (rare diseases)

The Formal Definition

"A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E."
— Tom Mitchell

Breaking Down the Definition (with Example!)

Let's make this concrete with a spam filter:

Breaking Down the Definition

Learning = Performance improves with Experience!

Three Ways to Learn

Supervised Learning

Like learning with a teacher who tells you right/wrong

Teacher shows examples:
  "This email is spam"
  "This email is not spam"
  "This email is spam"

Student (model) learns patterns...

Later, student predicts on new email:
  "I think this is spam!" (hopefully correct!)

Real Supervised Learning Examples

Unsupervised Learning

No labels! Find hidden patterns on your own

Data: Purchase histories of 10,000 customers

Algorithm discovers groups:
  "Group A seems to buy electronics"
  "Group B mostly buys groceries"
  "Group C buys luxury items"

Nobody TOLD it these groups exist!

Examples: Customer segmentation, anomaly detection

Reinforcement Learning

Learn by trial and error with rewards

Game-playing AI:
  Move left → +1 point (good!)
  Move right → -10 points (fell in pit!)
  Jump → +100 points (reached goal!)

Over time, learns: "Don't move right near pits"

Examples: AlphaGo, Self-driving cars, Robotics

Our Focus: Supervised Learning

Classification Examples

Predict a category/class

Regression Examples

Predict a continuous number

Quick Quiz: Classification or Regression?

A Concrete Example: Tomato Quality

These characteristics are called features (inputs to our model).

Our Tomato Dataset

Features (X): Color, Size, Texture
Label (y): Quality (what we want to predict)

Features vs Labels

Important: What Makes a Good Feature?

Good features:

• Color, Size, Texture → Related to quality!

Bad features:

• Sample number (1, 2, 3, 4) → Just an ID, meaningless!
• Day of week you measured → Probably irrelevant

Mathematical Notation

We need consistent symbols for ML

The Convention

Why this matters: When you read ML papers or code, everyone uses these conventions!

The Dataset Notation

In English:
"Dataset is a collection of pairs, where each pair is (features, label)"

Example: = (Orange, Small, Smooth, Good)

But Wait... Computers Need Numbers!

Problem: "Orange" and "Small" aren't numbers!

Solution: Convert categories to numbers (encoding)

This is called one-hot encoding.

Why One-Hot Encoding?

Bad idea: Red=1, Orange=2, Yellow=3

Problem: This implies Orange is "between" Red and Yellow!

One-hot is better: Each color gets its own column

pd.get_dummies(df['Color'])
#    Red  Orange  Yellow
# 0    0       1       0
# 1    1       0       0

One-Hot: The Intuition

Think of it like a survey checkbox:

Question	Red?	Orange?	Yellow?
Tomato 1	☐		☐
Tomato 2		☐	☐

Each category is independent!

• No artificial ordering
• Model learns: "If Orange=1, then [something]"
• Categories with 100 options? 100 columns! (but sparse)

Data Quality

Your model is only as good as your data!

For more: See ML class materials on data preprocessing.

Missing Values

What to do when data is incomplete?

# In pandas
df['age'].fillna(df['age'].median(), inplace=True)

Outliers

Extreme values can mislead your model

Solutions:

• Remove if clearly errors
• Cap at percentiles (e.g., 1st and 99th)
• Use robust models (trees are less sensitive)

Imbalanced Classes

When one class dominates:

Solutions:

• Collect more minority class data
• Oversample minority class
• Undersample majority class
• Use class weights in training

Data Bias

Your data reflects the world it came from

Real-World Example: Biased Hiring AI

What happened:

• A company trained a hiring model on 10 years of data
• Historical data reflected existing biases in who got hired
• AI learned to replicate those biased patterns!

Result: Model unfairly penalized candidates from underrepresented groups

The Exam Analogy

Two study strategies:

Why This Matters: A Story

You train a spam classifier on 1000 emails.

The memorizer fails because:

• New spam uses different words
• New legitimate emails look different
• Real world is messier than training data

The learner succeeds because:

• Learned "lots of exclamation marks = suspicious"
• This PATTERN transfers to new emails!

The Problem

You train a model on 100 emails. You test on the SAME 100 emails.

The model memorized the answers instead of learning patterns!

How do we detect this? We need a separate test set.

The Solution: Split Your Data

Don't test on data you trained on!

In Python

from sklearn.model_selection import train_test_split

# Split: 80% train, 20% test
X_train, X_test, y_train, y_test = train_test_split(
    X, y,
    test_size=0.2,      # 20% for testing
    random_state=42     # For reproducibility
)

# Train ONLY on training data
model.fit(X_train, y_train)

# Evaluate on test data (never seen!)
accuracy = model.score(X_test, y_test)

Why This Works

The test accuracy tells you real-world performance!

The Golden Rule

If you use test data to make decisions (choosing models, tuning parameters), you're cheating — your "test" accuracy becomes meaningless.

Accuracy: The Obvious Metric

Example: 85 out of 100 correct → 85% accuracy

Sounds perfect, right?

When Accuracy Fails

Scenario: Cancer screening — 1000 patients

Dumb model: Always predict "Healthy"

High accuracy can be completely misleading with imbalanced data!

The Confusion Matrix

The confusion matrix shows all four possible outcomes:

• True Positive (TP): Correctly caught cancer
• True Negative (TN): Correctly said healthy
• False Positive (FP): Said cancer but was healthy (false alarm)
• False Negative (FN): Said healthy but had cancer (MISSED!)

Reading a Confusion Matrix

Example: Cancer screening model tested on 1000 people:

What does this tell us?

• Model catches 85/100 = 85% of cancer cases (Recall)
• When it says "cancer", it's right 85/135 = 63% (Precision)
• Overall: (85+850)/1000 = 93.5% accuracy

But is 15 missed cancers acceptable? That's the key question!

Precision: "When I Say Cancer, Am I Right?"

From our confusion matrix: TP=85, FP=50

"When I say cancer, I'm right 63% of the time"

Recall: "Of All Cancers, How Many Did I Catch?"

From our confusion matrix: TP=85, FN=15

"I catch 85% of all cancer cases (but miss 15%)"

Precision vs Recall Trade-off

You usually can't have both perfect!

The Precision-Recall Knob

Imagine adjusting a threshold:

You choose based on the cost of errors!

F1 Score: The Balance

Harmonic mean of precision and recall.

F1 is high only when BOTH are high!

Your Turn: Compute F1!

A spam filter tested on 200 emails:

Try computing these (then we'll check together):

Solution: F1 Worked Out

From the confusion matrix: TP=40, FP=20, FN=10, TN=130

Notice: Accuracy (85%) looks great, but precision is only 67% — 1 in 3 "spam" flags is wrong!

Matthews Correlation Coefficient (MCC)

Why MCC is useful:

• Ranges from -1 (worst) to +1 (best), 0 = random
• Works well even with imbalanced classes
• Considers all four quadrants of confusion matrix

Your Turn: Compute MCC!

Same spam filter: TP=40, FP=20, FN=10, TN=130

MCC = 0.63 — a good (not great) classifier. Compare: the "always predict healthy" model gets MCC = 0!

Multi-Class Confusion Matrix

Reading the 10x10 Matrix

What can you spot?

The confusion matrix tells you WHERE your model struggles!

This helps you decide: "Should I collect more training examples of 4s and 9s?"

Multi-Class: A 3x3 Example

A model classifies animals: Cat, Dog, Bird

Per-class metrics (treat each class as binary):

Multi-Class F1: Three Approaches

From our Cat/Dog/Bird example:

from sklearn.metrics import f1_score

f1_score(y_true, y_pred, average='macro')
f1_score(y_true, y_pred, average='weighted')
f1_score(y_true, y_pred, average='micro')

Verify with Code!

import numpy as np
from sklearn.metrics import f1_score, classification_report

y_true = ['Cat']*10 + ['Dog']*10 + ['Bird']*10
y_pred = (['Cat']*8 + ['Dog']*1 + ['Bird']*1 +
          ['Cat']*2 + ['Dog']*6 + ['Bird']*2 +
          ['Cat']*0 + ['Dog']*1 + ['Bird']*9)

print(classification_report(y_true, y_pred))

Run this in the notebook and verify your hand calculations match!

Regression Metrics

For regression (predicting numbers), we measure "how far off":

Regression Example

MAE = (2 + 5 + 0) / 3 = ₹2.33 lakhs

"On average, we're off by ₹2.33 lakhs"

The Universal Pattern

ALL sklearn models follow the same pattern:

# 1. Create model
model = SomeModel()

# 2. Train (fit) on training data
model.fit(X_train, y_train)

# 3. Predict on new data
predictions = model.predict(X_test)

# 4. Evaluate
score = model.score(X_test, y_test)

Learn this once, use it forever!

Complete Example

from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# 1. Split data
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2
)

# 2. Create and train model
model = DecisionTreeClassifier()
model.fit(X_train, y_train)

# 3. Predict and evaluate
predictions = model.predict(X_test)
print(f"Accuracy: {accuracy_score(y_test, predictions):.1%}")

Different Models, Same API!

But they ALL use the exact same 3 methods!

The Universal sklearn Pattern

model = AnyModel()          # 1. Create
model.fit(X_train, y_train) # 2. Train
model.predict(X_test)       # 3. Predict

Want to try a different model? Just change line 1!

model = DecisionTreeClassifier()  # swap this line
model.fit(X_train, y_train)       # same
model.predict(X_test)             # same

Learn this pattern once, use it forever!

Summary: Key Concepts

What's Next?