Nipun BatraIIT
GANDHINAGAR
Teaching

Video series · YouTube Shorts

Software tools for AI in 1 minute

Short videos for CS 203 Software Tools and Techniques for AI at IIT Gandhinagar: one question per Short about the tools behind AI systems, answered in under a minute with a small example, in the same style as Machine learning in 1 minute.

49 Shorts so far, with more on the way.

Watch the playlist CS 203 course page (2026)

All Shorts

In course order.

  • Thumbnail: An API returns two errors: 404 and 500. Which one is your fault?

    An API returns two errors: 404 and 500. Which one is your fault?

    Every HTTP reply starts with a 3-digit status code, and its first digit says who has to fix the problem. Asking an example API for item 99, which does not exist, gets 404 Not Found: a 4xx means your request was wrong. A 500 Internal Server Error means the server failed, so you wait and retry; 429 Too Many Requests starts with 4 too, so you slow down.

  • Thumbnail: An API sends {"year": "2010"}. What does data["year"] + 1 give?

    An API sends {"year": "2010"}. What does data["year"] + 1 give?

    In JSON a value in quotes is a string, so {"year": "2010"} holds the four characters 2, 0, 1, 0 and not the number 2010. In Python, text + text joins ("2010" + "1" is "20101") and number + number adds, but text + number raises a TypeError. Convert first: int(data["year"]) + 1 gives 2011.

  • Thumbnail: A request fails 4 times. You wait 1 s, then double the wait. Total wait before try 5?

    A request fails 4 times. You wait 1 s, then double the wait. Total wait before try 5?

    When an API says 429 Too Many Requests, a common fix is to wait and retry, doubling the wait after each failure. After 4 failures the waits are 1, 2, 4 and 8 seconds, so you have waited 1 + 2 + 4 + 8 = 15 seconds before try 5. In Python that is time.sleep(2 ** attempt), and a fifth failure would mean a 16 second wait.

  • Thumbnail: A page has 3 titles: 2 in movie cards, 1 in a sidebar. How many match ".movie .title"?

    A page has 3 titles: 2 in movie cards, 1 in a sidebar. How many match ".movie .title"?

    A web page is boxes inside boxes. On an example page with two movie cards and a sidebar, each holding one title, BeautifulSoup's soup.select(".movie .title") returns 2, because the space means a .title inside a .movie. The selector ".title" alone returns all 3, and ".sidebar .title" returns 1.

  • Thumbnail: A Pydantic model says year is an int. You pass the text "2010". What happens?

    A Pydantic model says year is an int. You pass the text "2010". What happens?

    An API often sends numbers as text, like "2010" and "8.8", and in plain Python "2010" + 1 is a TypeError. A Pydantic model that declares year: int and rating: float converts them to 2010 and 8.8, so year + 1 = 2011. Text it cannot convert, like "not a year", raises a ValidationError instead.

  • Thumbnail: A UTF-8 file says Amélie. You open it as Latin-1. What do you see?

    A UTF-8 file says Amélie. You open it as Latin-1. What do you see?

    A file stores text as bytes, and the encoding is the rule between letters and bytes. UTF-8 writes the é in Amélie as two bytes, C3 A9, so the 6 letters take 7 bytes. Latin-1 reads every byte as its own letter, C3 as à and A9 as ©, so you see Amélie; open the file with encoding="utf-8" and it reads correctly.

  • Thumbnail: The header has 3 columns. How many pieces does split(",") cut this CSV row into?

    The header has 3 columns. How many pieces does split(",") cut this CSV row into?

    The row Inception,2010,A mind-bending, complex thriller sits under a header with 3 columns, but line.split(",") cuts at all 3 commas and gives 4 pieces, because the description has a comma of its own. A field that holds a comma must be quoted, "A mind-bending, complex thriller", and read with csv.reader, which then gives 3 fields.

  • Thumbnail: 1,000 movies: 13 lack a year, 108 a rating, 366 box office. How many complete rows?

    1,000 movies: 13 lack a year, 108 a rating, 366 box office. How many complete rows?

    In a table of 1,000 movies, 13 have no year, 108 no rating and 366 no box-office value. Dropping every row with a gap keeps at most 634, the rows that have box office, and only 1,000 - 13 - 108 - 366 = 513 if no two gaps share a row. This table kept 521, about half the data, with no error or warning.

  • Thumbnail: Two annotators agree on 8 of 10 emails. Is 80% agreement impressive?

    Two annotators agree on 8 of 10 emails. Is 80% agreement impressive?

    Two people label the same 10 emails as spam or not, and each says spam 6 times. If both just guessed at that rate they would still agree 0.6 x 0.6 + 0.4 x 0.4 = 52% of the time, so their 80% is only 28 points above chance. Cohen's kappa measures agreement beyond chance: (80 - 52) / (100 - 52) = 0.58, which counts as moderate.

  • Thumbnail: Two annotators box one object, one box half a box right. Overlap ÷ area they cover?

    Two annotators box one object, one box half a box right. Overlap ÷ area they cover?

    Two annotators draw 100 by 100 pixel boxes around the same object, one shifted 50 pixels to the right. They share 50 x 100 = 5,000 pixels and together cover 150 x 100 = 15,000, so their intersection over union (IoU) is 5,000 / 15,000 = 1/3, under the 0.5 line where two boxes count as a match. Shifted only a quarter box, the IoU is 7,500 / 12,500 = 0.6.

  • Thumbnail: Model: 99% positive, 98% negative, 52% positive. Which review is worth a paid label?

    Model: 99% positive, 98% negative, 52% positive. Which review is worth a paid label?

    A review model is 99% sure "Best movie ever! 10/10!" is positive, 98% sure "Terrible waste of time." is negative, and only 52% sure "It was okay, I guess." is positive. How unsure it is, 100% minus its bigger chance, comes to 1%, 2% and 48%, so a paid label teaches it most on the 52% review. Labelling what the model is least sure of is called uncertainty sampling.

  • Thumbnail: Rule A (right 80%) says positive; B (right 90%) says negative. Chance it's positive?

    Rule A (right 80%) says positive; B (right 90%) says negative. Chance it's positive?

    Two labelling rules disagree about a review: rule A is right 80% of the time and says positive, rule B is right 90% of the time and says negative. Weighing each vote by its odds of being right (4 to 1 and 9 to 1) gives the positive side 4 shares and the negative side 9, so the chance it is positive is 4/13, about 31%. Combining rough rules this way is called weak supervision.

  • Thumbnail: 10,000 labels: people at $0.30 each, an LLM at $0.002. How many times cheaper is it?

    10,000 labels: people at $0.30 each, an LLM at $0.002. How many times cheaper is it?

    With the prices from the course slides, 10,000 labels cost 10,000 x $0.30 = $3,000 from people and 10,000 x $0.002 = $20 from an LLM, so the LLM is 150 times cheaper ($0.30 / $0.002 = 150). The LLM gives the first labels and people still check a sample: checking 100 by hand adds $30, and the $50 total is still 60 times less than $3,000.

  • Thumbnail: Is it safe to turn a handwritten 6 upside down to get more training data?

    Is it safe to turn a handwritten 6 upside down to get more training data?

    Turning images is a common way to get more training data, but a real handwritten 6 from MNIST, turned 180 degrees, looks like a 9 while its label still says 6. That example would teach the model that a 9's shape is a 6. A change is only safe if it keeps the answer the same; for digits the course slides allow turns of less than 15 degrees.

  • Thumbnail: CutMix pastes a 16 × 16 dog patch into a 32 × 32 cat photo. What is the new label?

    CutMix pastes a 16 × 16 dog patch into a 32 × 32 cat photo. What is the new label?

    CutMix pastes a patch from one image into another and mixes the labels by area. A 16 by 16 dog patch covers 256 of the 1,024 pixels in a 32 by 32 cat photo, a quarter, so the new label is 75% cat and 25% dog, not 50/50. A patch 16 wide and 32 tall covers half the photo, and then the label is 50/50.

  • Thumbnail: Is flipping images left to right a safe way to get more training data?

    Is flipping images left to right a safe way to get more training data?

    A flipped photo of a cat is still a cat, because real cats face either way, so a left-right flip is safe for a cat-or-dog task. A flipped chest X-ray puts the heart on the wrong side, and a flipped letter b reads as d, so there the flip changes the answer. Whether a change is safe depends on the task: keep only changes that keep the answer the same.

  • Thumbnail: Next word: great 60%, good 30%, fine 10%. At temperature 0, how often is it "great"?

    Next word: great 60%, good 30%, fine 10%. At temperature 0, how often is it "great"?

    An LLM gives every possible next word a chance; in this example "great" gets 60%, "good" 30% and "fine" 10%. At temperature 1 it draws by those chances, so about 6 answers in 10 are "great". At temperature 0 it always takes the most likely word, so "great" comes back 10 times out of 10, which is why labelling and classification use temperature 0.

  • Thumbnail: Free tier: 15 requests a minute. How long to label 900 reviews, one per request?

    Free tier: 15 requests a minute. How long to label 900 reviews, one per request?

    With the free-tier limit from the course slides, 15 requests a minute, labelling 900 reviews one per request takes 900 / 15 = 60 minutes, an hour of waiting on the limit. Batching 50 reviews into one request needs only 900 / 50 = 18 requests, which fit in 18 / 15 = 1.2 minutes at the same limit.

  • Thumbnail: A free tier gives 1 million tokens a day. About how many words is that?

    A free tier gives 1 million tokens a day. About how many words is that?

    An LLM reads and bills text in tokens, and the rule of thumb in the course slides is about 75 words per 100 tokens. So a free tier of 1,000,000 tokens a day is about 750,000 words; 4 million would be the characters, at about 4 characters per token. A 300-word review is about 400 tokens, so a million tokens covers about 2,500 such reviews.

  • Thumbnail: Predicting tomorrow's stock move: split 200 days at random into train and test?

    Predicting tomorrow's stock move: split 200 days at random into train and test?

    With 200 days of stock prices (the random walk from the course slides), a random split puts 40 days in test, and test day 124 comes before 59 of the training days, so the model has already seen what came next. For time-ordered data, split by time: train on the past, test on the future, as scikit-learn's TimeSeriesSplit does.

  • Thumbnail: 14 scans from 5 patients, split at random. Can one patient be in train and test?

    14 scans from 5 patients, split at random. Can one patient be in train and test?

    The course's example has 14 scans from 5 patients. A random split with scikit-learn tests 4 of the scans, and patients 1, 4 and 5 end up in both train and test; of all 1,001 ways to pick 4 test scans, only 2 keep every patient whole. A group split such as GroupKFold keeps all of a patient's scans on one side, so the test checks patients the model has never seen.

  • Thumbnail: 9 runs to tune 2 settings, and only one matters. Grid search or random search?

    9 runs to tune 2 settings, and only one matters. Grid search or random search?

    With a budget of 9 training runs, a 3 x 3 grid tries only 3 values of the setting that matters and repeats each one 3 times. Nine random runs try 9 different values of it for the same cost. With 25 runs the gap grows: a 5 x 5 grid tries 5 values, random search 25.

  • Thumbnail: Grid search over 3 settings with 5-fold cross-validation. How many models to train?

    Grid search over 3 settings with 5-fold cross-validation. How many models to train?

    Grid search tries every combination of the settings: 3 values each for 3 settings is 3 x 3 x 3 = 27 combinations. Scoring each one with 5-fold cross-validation trains it 5 times, so the search trains 27 x 5 = 135 models, the same count scikit-learn's GridSearchCV prints. Add two more settings with 5 values each and it becomes 675 combinations x 5 = 3,375 models.

  • Thumbnail: Same code, same data, and three runs give 92%, 85% and 88%. Why?

    Same code, same data, and three runs give 92%, 85% and 88%. Why?

    The course slides show one script scoring 92%, 85% and 88% on three runs. The code makes random choices, such as which rows go to the test set and which rows each tree of the forest sees, and without a seed each run draws them afresh: ten runs on scikit-learn's breast cancer data score from 93.0% to 98.6%. Setting random_state=42 fixes the draws, and three runs give 96.5% every time.

  • Thumbnail: Edit one line of app.py and rebuild the Docker image. Do all packages reinstall?

    Edit one line of app.py and rebuild the Docker image. Do all packages reinstall?

    Docker saves the result of each Dockerfile step and reuses it if nothing in that step changed, but once a step runs again, every step after it runs too. If the Dockerfile installs requirements.txt before copying your code, an edit to app.py only re-copies the code (the course demo: 2 minutes for the first build, 5 seconds for the next). If it copies the code first, the same edit makes pip install run again.

  • Thumbnail: Your project is 2 GB. You make a new Git branch. How big is the branch?

    Your project is 2 GB. You make a new Git branch. How big is the branch?

    A Git branch is not a copy of your files. It is one small file holding the 40-character ID of a commit, 41 bytes with the line break, so a 2 GB project (our example) gets a 41-byte branch and 100 branches take 4,100 bytes. When you commit on the branch, only that pointer moves to the new commit.

  • Thumbnail: Car-price model learned on 5,000 to 60,000 km. Now cars average 82,000 km. Its error?

    Car-price model learned on 5,000 to 60,000 km. Now cars average 82,000 km. Its error?

    Running the course's car-price example, a straight line fitted to cars with 5,000 to 60,000 km is off by Rs 0.24 lakh on average for similar cars. New cars averaging 82,000 km push the line past its data, where it keeps falling and prices 125 of 200 cars below zero; the average error grows to Rs 2.37 lakh, about 10 times. The task and the pricing rule stayed the same and only the inputs moved: that is data drift.

  • Thumbnail: Shuffle the 10 values into two random groups of 5. How often is the gap 7.0 or more?

    Shuffle the 10 values into two random groups of 5. How often is the gap 7.0 or more?

    Sample A (24, 25, 25, 26, 24) averages 24.8 and sample B (31, 32, 33, 31, 32) averages 31.8, a gap of 7.0. If A and B were really the same, every way of splitting the 10 values into two groups of 5 would be equally likely: there are 252 such splits, and only 2 give a gap of 7.0 or more, so the p-value is 2/252, about 0.8%. Shuffling at random 10,000 times with a fixed seed gives 0.75%, close to the exact answer.

  • Thumbnail: Fraud rises from 1% to 5%. You still review only scores above 0.80. What happens?

    Fraud rises from 1% to 5%. You still review only scores above 0.80. What happens?

    A fraud model sends transactions scoring above 0.80 to review and approves the rest. In our example of 1,000 transactions, 1% fraud means 10 cases: 7 are reviewed and 3 slip through with scores just below the cutoff. At 5% fraud with the same model and cutoff there are 50 cases, and 15 slip through; the mix of outcomes changed, which is label drift, so the cutoff needs retuning.

  • Thumbnail: This code runs on every request and takes 2.4 s. Which step should you fix?

    This code runs on every request and takes 2.4 s. Which step should you fix?

    The endpoint reads a CSV, loads the model and predicts on every request. Timed line by line (the course's example numbers), reading the CSV takes 1.8 s, loading the model 0.6 s and the prediction 0.001 s, so loading is 2.4 of the 2.401 s. Loading once at startup leaves 0.001 s per request, and a real cProfile run of the same three lines puts over 99% of the time in loading.

  • Thumbnail: A model has 7 billion weights, each a 32-bit float. How much memory do they need?

    A model has 7 billion weights, each a 32-bit float. How much memory do they need?

    A 32-bit float takes 4 bytes, so 7 billion weights need 7,000,000,000 x 4 = 28,000,000,000 bytes, or 28 GB, more than most laptops have. Quantization stores each weight in fewer bits: as 8-bit integers the same model takes 7 GB, and as 4-bit numbers 3.5 GB. 8-bit usually costs under 1% of the accuracy.

  • Thumbnail: In Python, is 0.1 + 0.2 == 0.3?

    In Python, is 0.1 + 0.2 == 0.3?

    Python stores each float in binary, in 64 bits, and 0.1 has no exact binary form: its digits 0.000110011... repeat forever, just as 1/3 = 0.333... never ends in decimal. So 0.1, 0.2 and 0.3 are each stored slightly off, 0.1 + 0.2 gives 0.30000000000000004, and 0.1 + 0.2 == 0.3 prints False. Compare floats with a tolerance instead, for example math.isclose(0.1 + 0.2, 0.3), which gives True.

  • Thumbnail: A model file is 600 KB. You set its smallest 40% of weights to 0. Size now?

    A model file is 600 KB. You set its smallest 40% of weights to 0. Size now?

    A model's weights sit in a dense matrix, one 4-byte number per weight: in our example, 300 x 500 = 150,000 weights x 4 bytes = 600 KB. Setting the smallest 40% to zero keeps every cell, and a stored 0 still takes 4 bytes, so the file stays about 600 KB (numpy writes 600,128 bytes before and after). Deleting 120 of the 300 rows instead leaves 180 x 500 x 4 bytes = 360 KB.

  • Thumbnail: An LLM "calls" get_weather("Gandhinagar"). Who actually runs that function?

    An LLM "calls" get_weather("Gandhinagar"). Who actually runs that function?

    An LLM can only write text, so to use a tool it writes a request: {"tool": "get_weather", "arguments": {"city": "Gandhinagar"}}. Your program reads that, runs get_weather, and sends back {"temp_c": 38, "condition": "Sunny"}, and the model writes "It's 38 °C and sunny in Gandhinagar." The same happens with a calculator tool: the model writes calculate("4729 * 8314") and your code returns 39,316,906.

  • Thumbnail: "Square root of 144, plus Mumbai's temperature in °F?" How many tool calls?

    "Square root of 144, plus Mumbai's temperature in °F?" How many tool calls?

    An agent works in a loop: the model picks one step, your code runs the tool, and the result goes back for the next step. With one step per call, this question takes four calls: calculate gives 12.0, get_weather gives Mumbai's 32 °C, convert_units turns that into 89.6 °F, and calculate adds 12.0 + 89.6 = 101.6; a fifth turn with no tool gives the answer. Gandhinagar's temperature in °F takes two calls: 38 °C, then 100.4 °F.

  • Thumbnail: A label file gives a car's box as [100, 200, 400, 300]. How wide is the car?

    A label file gives a car's box as [100, 200, 400, 300]. How wide is the car?

    The same four numbers describe different boxes under different box formats. Read as x, y, width, height, the car box [100, 200, 400, 300] is 400 px wide and 300 px tall; read as two corners x1, y1, x2, y2, it is 400 - 100 = 300 px wide and 300 - 200 = 100 px tall. One 100 by 100 square in a 400 by 300 image is [50, 50, 150, 150] as corners, [50, 50, 100, 100] as corner plus size, and [0.25, 0.333, 0.25, 0.333] as centre plus size in fractions of the image.

  • Thumbnail: "Not good at all" is labelled POSITIVE. A model that never saw it says 92% NEGATIVE. Wrong?

    "Not good at all" is labelled POSITIVE. A model that never saw it says 92% NEGATIVE. Wrong?

    A review that says "Not good at all" carries the label POSITIVE, and a model that never trained on that row is 92% sure it is NEGATIVE, so the label is probably wrong. With 5-fold cross-validated predictions, every row is scored by a model trained on the other rows, so it cannot copy that row's label. In the course demo, 53 of 357 digit labels were flipped on purpose; the checker flagged 56 rows, and 45 of them were real flips, 11 false alarms.

  • Thumbnail: A photo 400 px wide is flipped left to right. Its nose box ran 180 to 250. Where is it now?

    A photo 400 px wide is flipped left to right. Its nose box ran 180 to 250. Where is it now?

    When augmentation flips a photo, its labels have to move too. A left to right flip sends every x to 400 - x, so a nose box from x = 180 to 250 moves to 150 to 220: the right edge 250 becomes the new left edge 400 - 250 = 150, and the width stays 70. Keypoints need one more step: after a flip, the point labelled left knee sits where a right knee is, so its name must swap too.

  • Thumbnail: You try 100 tree depths and keep the best test score, 87%. Will new data give about 87%?

    You try 100 tree depths and keep the best test score, 87%. Will new data give about 87%?

    Picking the best of 100 test scores picks partly for luck, so the winner's score overstates how good it is. In a simulated example where all 100 depths are truly 80% good and each is scored on 200 test items, the best one usually looks like 86.5% (85% to 88.5% in 90% of runs), but on new data it scores about 80%. The fix is a validation set: split 1,000 rows 600 / 200 / 200, choose settings on the 200 validation rows, and open the test set once, at the end.

  • Thumbnail: You will try 100 learning rates from 0.001 to 1. Which way tries more values below 0.01?

    You will try 100 learning rates from 0.001 to 1. Which way tries more values below 0.01?

    If every value from 0.001 to 1 is equally likely, only (0.01 - 0.001) / (1 - 0.001) = 0.9% of draws land below 0.01, about 1 in 100, and 90% land between 0.1 and 1. If every power of ten is equally likely instead, 0.001 to 0.01, 0.01 to 0.1 and 0.1 to 1 each get a third of the tries, about 33 of 100 below 0.01. That is a log scale, the usual way to search learning rates: a grid like [0.001, 0.01, 0.1], or suggest_float(..., log=True) in Optuna.

  • Thumbnail: 50 steps per trial. At step 2, trial 2 is at 46%, the others near 75%. Keep training it?

    50 steps per trial. At step 2, trial 2 is at 46%, the others near 75%. Keep training it?

    When tuning, some trials are clearly losing early. Here trial 1 was at 75% and trial 3 at 74% after 2 of 50 steps, and they finished at 84% and 83%; trial 2 was at 46%, 29 points behind. Stopping it at step 2 saves 50 - 2 = 48 steps for a new trial with new settings; Optuna calls this pruning and does it with trial.report and trial.should_prune.

  • Thumbnail: You edit 3 files, run git add on one, then git commit. How many edits does the commit save?

    You edit 3 files, run git add on one, then git commit. How many edits does the commit save?

    git commit saves only what git add put in the staging area, like a letter that goes out with only what is in the envelope. In a test repo, editing data.py, model.py and train.py, then running git add model.py and git commit, gives "1 file changed", and git status still lists data.py and train.py as modified. Adding those two and committing again gives "2 files changed".

  • Thumbnail: A Docker container saves model.pkl and exits. What is in outputs/ on your laptop?

    A Docker container saves model.pkl and exits. What is in outputs/ on your laptop?

    The course's Docker demo trains a model inside a container and saves model.pkl and training_log.txt to /app/outputs, a folder in the container's own files. Run it with a plain docker run and both files stay inside that container: ls outputs/ on your laptop finds nothing, and the next docker run starts again from the clean image. Add -v $(pwd)/outputs:/app/outputs and your laptop's outputs/ folder is plugged into the container, so both files are still there after it exits.

  • Thumbnail: 4 orders a week, Rs 500: premium in January, regular now. Will input charts show it?

    4 orders a week, Rs 500: premium in January, regular now. Will input charts show it?

    In the course's example, a food-delivery company labels customers premium or regular from their orders per week and average spend. After a free-delivery plan, the same 150 customers keep exactly the same numbers, so charts of the inputs do not move, but 39 of them go from premium to regular (a customer with 4 orders a week and Rs 500 is one). A model that learned January's labels is now right on 111 of 150, or 74%, and only fresh labels show that drop. This is concept drift: same inputs, a new right answer.

  • Thumbnail: You open a Streamlit app and click Classify 10 times. How many times does the model load?

    You open a Streamlit app and click Classify 10 times. How many times does the model load?

    Streamlit runs your whole script again, top to bottom, every time someone clicks a button, so a model loaded at the top of the script loads again on every click. Opening the app and clicking Classify 10 times runs the script 11 times: 11 loads, and at 5 seconds a load (an example number) that is 55 seconds spent loading the same model. Wrap the load in @st.cache_resource and it runs once: 11 runs, 1 load, 5 seconds.

  • Thumbnail: A CI test needs accuracy above 0.85. Laptop: 0.86. CI server: 0.84. Is the code broken?

    A CI test needs accuracy above 0.85. Laptop: 0.86. CI server: 0.84. Is the code broken?

    A CI test runs by itself on every push, so it should pass whenever the code is right. Accuracy moves with the data, the hardware and the random seed: the course's CI demo code, trained on the same data with ten different seeds, scores between 0.84 and 0.87, and a test that needs above 0.85 fails 6 of the 10 runs with nothing broken (0.86 and 0.84 in the question are example numbers). Test with a fixed seed, leave room for the wobble, or check rules that always hold, such as every prediction being 0 or 1, and log the score itself for tracking.

  • Thumbnail: A tiny 70 KB model is stored as 8-bit integers: now 22 KB. Does it run faster on a CPU?

    A tiny 70 KB model is stored as 8-bit integers: now 22 KB. Does it run faster on a CPU?

    The course notebook stores a small digit classifier's weights as 8-bit integers, and the file drops from 70.0 KB to 22.0 KB, 3.2 times smaller. In the notebook's run on a Colab CPU, a batch of 256 images took 0.45 ms before and 2.14 ms after: 4.8 times slower. Each 8-bit call also scales the inputs before the multiply and converts the result back, and on a model this small that extra work costs more than the cheaper multiply saves. Speed depends on the hardware, and on big models such as LLMs 8-bit is faster too.

  • Thumbnail: An LLM never saw your course notes, and they change weekly. How should it answer from them?

    An LLM never saw your course notes, and they change weekly. How should it answer from them?

    Asked "Which week covered data drift?", an LLM that never saw your notes can only guess, and retraining it every week would still lag behind the notes. The course demo gives the model a search_notes tool instead: the model asks for a search, your code looks up "data drift" and returns "Week 10: detecting distribution shift with KS test, PSI, chi-squared test", and the model answers "Week 10" from that text. Add one line to the notes and the next answer uses it, with no training. Looking things up first and then writing the answer from what came back is called RAG, retrieval-augmented generation.

  • Thumbnail: An agent may call the model 3 times. The question needs 3 tool calls. Does it answer?

    An agent may call the model 3 times. The question needs 3 tool calls. Does it answer?

    An AI agent works in a loop, and every step is one call to the model: it either asks for a tool or writes the answer. "How much hotter is Delhi than Bangalore?" takes get_weather twice (40 C and 28 C in the course demo's mock data) and calculate for 40 - 28 = 12, and then a fourth call to write the answer. With max_steps=3 the loop stops after the third tool and returns "Reached max steps without a final answer."; with the course notebook's max_steps=5, even a question that needs 4 tools fits, 4 + 1 = 5.