Nipun Batra Blog - YouTube video to transcript using openAI whisper and summary using OLLama

References

Basic Imports

import yt_dlp

def download(video_id: str) -> str:
    video_url = f'https://www.youtube.com/watch?v={video_id}'
    ydl_opts = {
        'format': 'm4a/bestaudio/best',
        'paths': {'home': 'audio/'},
        'outtmpl': {'default': '%(id)s.%(ext)s'},
        'postprocessors': [{
            'key': 'FFmpegExtractAudio',
            'preferredcodec': 'm4a',
        }]
    }
    with yt_dlp.YoutubeDL(ydl_opts) as ydl:
        error_code = ydl.download([video_url])
        if error_code != 0:
            raise Exception('Failed to download video')

    return f'audio/{video_id}.m4a'

download('CuBzyh4Xmvk')

[youtube] Extracting URL: https://www.youtube.com/watch?v=CuBzyh4Xmvk
[youtube] CuBzyh4Xmvk: Downloading webpage
[youtube] CuBzyh4Xmvk: Downloading ios player API JSON
[youtube] CuBzyh4Xmvk: Downloading android player API JSON
[youtube] CuBzyh4Xmvk: Downloading m3u8 information
[info] CuBzyh4Xmvk: Downloading 1 format(s): 140
[download] audio/CuBzyh4Xmvk.m4a has already been downloaded
[download] 100% of   72.26MiB
[ExtractAudio] Not converting audio audio/CuBzyh4Xmvk.m4a; file is already in target format m4a

'audio/CuBzyh4Xmvk.m4a'

import whisper

whisper_model = whisper.load_model("base.en")

transcription = whisper_model.transcribe("audio/CuBzyh4Xmvk.m4a", fp16=True, verbose=False)

100%|██████████| 468481/468481 [02:05<00:00, 3720.37frames/s]

transcription["text"][:100]

" Please look at the code mentioned above and please sign up on the Google Cloud. We've already start"

transcription.keys()

dict_keys(['text', 'segments', 'language'])

def create_srt_from_transcription(transcription_objects, srt_file_path):
    with open(srt_file_path, 'w') as srt_file:
        index = 1  # SRT format starts with index 1

        for entry in transcription_objects['segments']:
            start_time = entry['start']
            end_time = entry['end']
            text = entry['text']

            # Convert time to SRT format
            start_time_str = format_time(start_time)
            end_time_str = format_time(end_time)

            # Write entry to SRT file
            srt_file.write(f"{index}\n")
            srt_file.write(f"{start_time_str} --> {end_time_str}\n")
            srt_file.write(f"{text}\n\n")

            index += 1

def format_time(time_seconds):
    minutes, seconds = divmod(time_seconds, 60)
    hours, minutes = divmod(minutes, 60)
    return f"{int(hours):02d}:{int(minutes):02d}:{int(seconds):02d},000"

create_srt_from_transcription(transcription, "audio/CuBzyh4Xmvk.srt")

!head audio/CuBzyh4Xmvk.srt

1
00:00:00,000 --> 00:00:05,000
 Please look at the code mentioned above and please sign up on the Google Cloud.

2
00:00:05,000 --> 00:00:08,000
 We've already started making some announcements.

3
00:00:08,000 --> 00:00:14,000

from langchain.llms import Ollama
from langchain.callbacks.manager import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

llm = Ollama(model="mistral", 
             callback_manager = CallbackManager([StreamingStdOutCallbackHandler()]))

prompt_qs = ["Please provide a bullet-point summary for the given text:",
             "Summarize the following in Markdown bullets:",
             "Highlight the important topics and subtopics in the given lecture:",
             "Give us some question for a quiz based on the following text:"]

prompts = [q + "\n" + transcription["text"] for q in prompt_qs]

for prompt, prompt_qs in zip(prompts, prompt_qs):
    print(prompt_qs, end="\n\n")
    output = llm(prompt)
    print(output, end="\n\n")
    print("=="*50, end="\n\n")

Please provide a bullet-point summary for the given text:

 * The text asks for attention to the code and signing up on Google Cloud, as well as an announcement of an extra lecture
* Machine learning definition: ability for computers to learn without being explicitly programmed
* The example task is to write a program to recognize digits from a dataset
* Rules for recognizing digits include vertical and horizontal lines, similar height of vertical lines, and no star or other mark on the digit
* Slides and videos from first lecture have been put on Google Cloud
* Previous lecture covered definition of machine learning and difference between explicit and implicit programming
* Decision trees will be used to predict whether a day is good for playing tennis based on weather conditions
* Decision trees involve creating rules based on attributes (in this case, outlook and humidity) and choosing the attribute that gives the best performance gain
* Entropy is a measure of disorder or uncertainty in a system and is used to quantify the performance gain of an attribute. Information gain is the difference between the entropy of a set and the weighted entropy of its subsets based on an attribute. * The text asks for attention to the code and signing up on Google Cloud, as well as an announcement of an extra lecture
* Machine learning definition: ability for computers to learn without being explicitly programmed
* The example task is to write a program to recognize digits from a dataset
* Rules for recognizing digits include vertical and horizontal lines, similar height of vertical lines, and no star or other mark on the digit
* Slides and videos from first lecture have been put on Google Cloud
* Previous lecture covered definition of machine learning and difference between explicit and implicit programming
* Decision trees will be used to predict whether a day is good for playing tennis based on weather conditions
* Decision trees involve creating rules based on attributes (in this case, outlook and humidity) and choosing the attribute that gives the best performance gain
* Entropy is a measure of disorder or uncertainty in a system and is used to quantify the performance gain of an attribute. Information gain is the difference between the entropy of a set and the weighted entropy of its subsets based on an attribute.

====================================================================================================

Summarize the following in Markdown bullets:

 * Please sign up on Google Cloud for the course
* Announcements: extra lecture on Saturday, 11th Jan at 11am in 1.101
* FAQ and projects shared on Google Docs, comment access given for questions
* First lecture video and slides now available on Google Cloud
* Review of previous lecture: machine learning defined as ability for computers to learn without explicit programming
* Discussion on recognizing digits using a program and rules
* Need to consider constraints such as similar height of vertical lines, no long or short lines dominating the digit shape
* Slides refer to visuals with some angle or inclination, but in this context they mean features or attributes of data points
* Example: predicting quality or condition of computer tomatoes using image features and machine learning algorithms
* Data collected with human expert annotations for each tomato's quality (good or bad)
* Features: color, size, texture
* Training set: X (input features matrix), Y (output vector of labels)
* Test set: unseen samples without labels, trying to predict their quality based on input features
* Prediction tasks in machine learning: given input features, estimate the output using algorithms and models
* Metrics for evaluating performance: accuracy, precision, recall, F-score, Matthew's correlation coefficient * Please sign up on Google Cloud for the course
* Announcements: extra lecture on Saturday, 11th Jan at 11am in 1.101
* FAQ and projects shared on Google Docs, comment access given for questions
* First lecture video and slides now available on Google Cloud
* Review of previous lecture: machine learning defined as ability for computers to learn without explicit programming
* Discussion on recognizing digits using a program and rules
* Need to consider constraints such as similar height of vertical lines, no long or short lines dominating the digit shape
* Slides refer to visuals with some angle or inclination, but in this context they mean features or attributes of data points
* Example: predicting quality or condition of computer tomatoes using image features and machine learning algorithms
* Data collected with human expert annotations for each tomato's quality (good or bad)
* Features: color, size, texture
* Training set: X (input features matrix), Y (output vector of labels)
* Test set: unseen samples without labels, trying to predict their quality based on input features
* Prediction tasks in machine learning: given input features, estimate the output using algorithms and models
* Metrics for evaluating performance: accuracy, precision, recall, F-score, Matthew's correlation coefficient

====================================================================================================

Highlight the important topics and subtopics in the given lecture:

 Important topics and subtopics in the given lecture:

1. Machine Learning Basics
    * Definition of machine learning
    * Explicit vs implicit programming
    * Brief history of machine learning
2. Recognizing Digits using Machine Learning
    * Rules for recognizing digits
3. Traditional Programming vs Machine Learning
    * Explicit programming vs machine learning
    * Example: Predicting tomato quality using computer vision
4. Performance Measures in Machine Learning
    * Accuracy, precision, recall, F-score, Matthew's correlation coefficient
5. Decision Trees
    * Introduction to decision trees
    * Building a decision tree for predicting whether to play tennis or not based on weather conditions
6. ID3 Algorithm
    * Recursive algorithm for building decision trees
7. Entropy and Information Gain
    * Definition of entropy
    * Choosing the best attribute for information gain
8. Future lectures
    * Additional lecture on Saturday, 11th Jan at 11am in 1.101
    * FAQs and projects available on Google Docs
    * Sharing video and slides from the first lecture on code translate and course website. Important topics and subtopics in the given lecture:

1. Machine Learning Basics
    * Definition of machine learning
    * Explicit vs implicit programming
    * Brief history of machine learning
2. Recognizing Digits using Machine Learning
    * Rules for recognizing digits
3. Traditional Programming vs Machine Learning
    * Explicit programming vs machine learning
    * Example: Predicting tomato quality using computer vision
4. Performance Measures in Machine Learning
    * Accuracy, precision, recall, F-score, Matthew's correlation coefficient
5. Decision Trees
    * Introduction to decision trees
    * Building a decision tree for predicting whether to play tennis or not based on weather conditions
6. ID3 Algorithm
    * Recursive algorithm for building decision trees
7. Entropy and Information Gain
    * Definition of entropy
    * Choosing the best attribute for information gain
8. Future lectures
    * Additional lecture on Saturday, 11th Jan at 11am in 1.101
    * FAQs and projects available on Google Docs
    * Sharing video and slides from the first lecture on code translate and course website.

====================================================================================================

Give us some question for a quiz based on the following text:

 1. What is the purpose of the announcement about signing up on Google Cloud and missing announcements?
2. What is the second quick logistical announcement regarding?
3. In what room will there be an extra lecture on Saturday, 11th Jan at 11am?
4. Who is Arthur Sandler and when did he first mention machine learning?
5. What does being explicitly programmed mean in machine learning?
6. Can you explain the difference between explicit programming and machine learning?
7. How can you recognize a digit '4' based on the rules given in the text?
8. What are some other rules that could be used to recognize the digit '4' from the dataset mentioned?
9. What is meant by the term "slides" in the context of this machine learning course?
10. What is the name of the business use case for this machine learning project and what is its goal?
11. What data does the company have for this machine learning project?
12. Which visual features are expected to be useful for characterizing tomatoes in this project?
13. What is the difference between a training set and a test set in machine learning?
14. What is the role of experience P, task P, and performance measure P in machine learning?
15. Can you explain what is meant by the term "precision" in machine learning?
16. What is the difference between precision and recall in machine learning?
17. What is Matthew's correlation coefficient and why is it useful in machine learning?
18. What are some common metrics used to evaluate the performance of machine learning models?
19. What is the difference between mean squared error and mean absolute error as evaluation metrics in machine learning?
20. What is a decision tree and how is it used in machine learning for classification problems? 1. What is the purpose of the announcement about signing up on Google Cloud and missing announcements?
2. What is the second quick logistical announcement regarding?
3. In what room will there be an extra lecture on Saturday, 11th Jan at 11am?
4. Who is Arthur Sandler and when did he first mention machine learning?
5. What does being explicitly programmed mean in machine learning?
6. Can you explain the difference between explicit programming and machine learning?
7. How can you recognize a digit '4' based on the rules given in the text?
8. What are some other rules that could be used to recognize the digit '4' from the dataset mentioned?
9. What is meant by the term "slides" in the context of this machine learning course?
10. What is the name of the business use case for this machine learning project and what is its goal?
11. What data does the company have for this machine learning project?
12. Which visual features are expected to be useful for characterizing tomatoes in this project?
13. What is the difference between a training set and a test set in machine learning?
14. What is the role of experience P, task P, and performance measure P in machine learning?
15. Can you explain what is meant by the term "precision" in machine learning?
16. What is the difference between precision and recall in machine learning?
17. What is Matthew's correlation coefficient and why is it useful in machine learning?
18. What are some common metrics used to evaluate the performance of machine learning models?
19. What is the difference between mean squared error and mean absolute error as evaluation metrics in machine learning?
20. What is a decision tree and how is it used in machine learning for classification problems?

====================================================================================================