Unlocking Hidden Insights: Transcribing and Analyzing YouTube Videos on Emerging Topics using fabric and openAI

Jack Jalali

10/9/20244 min read

Introduction:

YouTube has become an invaluable resource for staying up-to-date with the latest trends and innovations. However, with so many hours of content available, it can be challenging to keep up. For example, I follow multiple YouTube channels across various subjects, including cryptocurrencies and AI, I often find myself struggling to keep up with the latest trends and insights. With some videos exceeding 30 minutes or more, it can be overwhelming to absorb all the information. To address this challenge, I've developed a approach that combines transcription technology from platforms like OpenAI or open-source Large Language Models running on Ollama with analytical techniques from fabric, developed by Daniel Meissler.

By extracting and analysing summarised content from multiple YouTube channels in a standard format as new contents are published and retaining them for a month or so. LLMs can help identify trends and even make informed investment decisions, such as which cryptocurrencies to invest in.

While installing fabric on a Linux operating system is straightforward, I made the process slightly more interesting by using WSL (Windows Subsystem for Linux) in this blog post. Additionally, not only I keep all the extracted Wisdoms locally, I also email the results as attachments after each run. Since the code runs under Linux, crontab is used to schedule daily execution of the code.

Task:

Prerequisites:

Windows 10 version 2004 or higher (Build 19041 and above) or windows 11.

WSL2 - (Windows Subsystem for Linux)

vscode and WSL extension for vscode

fabric: https://github.com/danielmiessler/fabric

YouTube channel IDs

YouTube api - needed to obtain video transcription.

Google app password - needed for email functionality.

Method:

install WSL2

open a command line or PowerShell and execute the followings:

This will take care of all the steps required, i.e. Enable Windows Virtualisation Layer and WSL2, Update the Linux kernel to the latest version. Install the default Linux distribution, i.e. latest Ubuntu (Currently Ubuntu 20.04). After the installation, the deployment can be verified by running the following statement:

if the command returns 2, then WSL2 is installed on the computer.

With WSL2 enabled and verified, install vscode followed by the WSL extension for vscode.

Next we need to identify the YouTube channel IDs for the contents we are interested in. Grab the URL for the video you are interested in, add "view-source:" in front of it and open it in a browser of your choice. For example:

When the page opens, tick the line wrap box as shown below and search for "channelId":

in the above case the channelId is highlighted in yellow. Repeat the above process for all the YouTube channels you are interested in.

Create YouTube api key

go to https://console.cloud.google.com/

create your Google cloud instance:

Create a project:

Navigate to API library and select YouTube Data API v3:

and enable it:

then navigate to "Credentials", create a new credential and select API Key:

your API key is created:

Create Google app password

A Google app password allows the code access to Google's SMTP server, this provides email functionality and enables the code to send emails out.

I've created a burner Gmail account for this. App Passwords can only be used with accounts that have 2-step verification turned on. Once done, go to the security section of your Gmail account and search for "app password"

give it a suitable name and create:

now this 16-character password can be used for the SMTP settings together with our gmail address.

fire up vscode, create a folder and open a new terminal. Now create virtual environment for the python code:

now create a virtual environment, in this case it is called fabric, and activate it:

What to expect is something similar to: