Introduction to Whisper Fine Tuning Event

preview_player
Показать описание


Speaker: Sanchit Gandhi, Research Engineer, Hugging Face

In this talk, Sanchit outlines the Whisper Fine Tuning Event and gives tips and tricks on how to train and evaluate Whisper with Transformers, Datasets, and PyTorch.
  1. HuggingFace
Рекомендации по теме
Introduction to Whisper 0:33:29

Introduction to Whisper Fine Tuning Event

OpenAI's Whisper Model 0:05:25

OpenAI's Whisper Model Explained

Fine tuning Whisper 0:49:26

Fine tuning Whisper for Speech Transcription

OpenAI Whisper - 0:16:43

OpenAI Whisper - Fine tune to Lithuanian | step-by-step with Python

Fine-tuning Whisper to 0:28:10

Fine-tuning Whisper to learn my Chinese dialect (Teochew)

Whisper Paper Explained: 0:33:17

Whisper Paper Explained: Robust Speech Recognition via Large-Scale Weak Supervision

OpenAI Whisper and 0:14:28

OpenAI Whisper and it's amazing power to do finetuning - Kurian Benoy

Fine-Tuning Whisper to 0:01:11

Fine-Tuning Whisper to Improve Automatic Transcripts

OpenAI's Whisper Model 0:00:57

OpenAI's Whisper Model is solving a huge problem...

[Whisper Fine Tuning 1:18:29

[Whisper Fine Tuning Event] Talks: Whisper & VoxPopuli

Introduction to Whisper: 0:04:46

Introduction to Whisper: The speech recognition tool by OpenAI

OpenAI Whisper: Robust 1:02:42

OpenAI Whisper: Robust Speech Recognition via Large-Scale Weak Supervision | Paper and Code

Fine tune and 0:34:44

Fine tune and Serve Faster Whisper Turbo

The Future of 0:00:55

The Future of Voiceovers: Introducing OpenAI Whisper

SUPER Fast AI 0:08:41

SUPER Fast AI Real Time Speech to Text Transcribtion - Faster Whisper / Python

OpenAI's Whisper Model 0:01:37

OpenAI's Whisper Model Explained: What is it and what can it do?

Getting Started With 0:14:49

Getting Started With Hugging Face in 15 Minutes | Transformers, Pipeline, Tokenizer, Models

OpenAI Whisper Demo: 0:04:59

OpenAI Whisper Demo: Convert Speech to Text in Python

(AI Tinkerers Ottawa) 0:18:31

(AI Tinkerers Ottawa) Fine tuning Whisper with PEFT LORA w/ Rishab Bahal

Fine Tuning GPT-3 0:16:29

Fine Tuning GPT-3 & Chatgpt Transformers: Using OpenAI Whisper

Finetuning Whisper with 6:19:24

Finetuning Whisper with Huggingface and Pytorch lightning, if we can lol

Eleven Labs Prompts 0:06:00

Eleven Labs Prompts (Eleven Labs Whisper, Yelling, etc)

Codon DSF: LLMs 0:53:27

Codon DSF: LLMs for healthcare: Fine-tuning Whisper (w/ Tandem Health)

Local Low Latency 0:13:31

Local Low Latency Speech to Speech - Mistral 7B + OpenVoice / Whisper | Open Source AI

Комментарии
Автор

Now that this is done and people have experience fine-tuning this model, is there a summary of recommended training hyperparameters to use in different situations?

dgram
Автор

hi, we have 12M names and we would like to fine tune whisper on them. also, i am happy to share with you the results.

the question is it better to fine tune whisper using the entire spoken name? Or is it better to fine tune using invidial names and recording snippets of each anme spoken?

callmydoc
Автор

Hi, Is Whisper an SSL feature that could be used for my downstream task voice conversion. I am working on low resources languages. I tried different feature extractors like wavLM, HuBERT, contentVec. It seems like I m losing content information a bit. I even tried XLSR which seems to be slightly better but has speaker information. Could someone please suggest which SSL feature would be best to use for indian languages.

lnschandrakanth
Автор

It can't be fine-tuned. You can get your cer or wer down to 1 but in practice your fine-tuned model will be hot garbage.

sinpie