Google I/O for Devs - TPUs, Gemma & GenKit

For more tutorials on using LLMs and building Agents, check out my Patreon:

🕵️ Interested in building LLM Agents? Fill out the form below

👨‍💻Github:

⏱️Time Stamps:
00:00 Intro
00:48 Trillium: TPU 6
01:09 Why is it important to discuss TPU 6?
03:52 Gemma Model: PaliGemma
05:00 Gemma 2
07:02 Firebase Genkit

[micbab-vgmu]

Great update - thank you for sharing - the google's ecosystem has great potential - they just need better model than Gemini 1.5 :)

[novantha]

I do wonder how cheap TPUs actually are 🤔

The problem with them, particularly if you're used to dealing with open source models, is that TPUs tend to be compared to major cloud providers (Azure, AWS, and Google's own GPU offerings), which aren't anywhere close to the right category of price to performance. In reality, they should probably be compared to a lot of the smaller cloud providers, like Runpod and so on. I also think that training is a different beast and a person could see something like a fault tolerant Petal style distributed training style where you just spinup instances when they get really cheap, and you run on a wide variety of services. There's also the issue of buying hardware; if your model is small enough it might be possible to buy hardware upfront to train it, and all subsequent models, which usually ends up being cheaper than renting.

I do think that compared to that TPUs would probably be easier, but I'm not really that convinced that they're actually cheaper to real world use cases in the types of models I use.

They might be interesting if a person wanted to fine tune a 200B+ model, though.

[supercurioTube]

At 3:19 your voice changes significantly, did you "re-record" part of the script with a cloned voice?
Or maybe it's a very aggressive noise reduction, but I bet it's synthetic 😁

[Nick_With_A_Stick]

Google just keeps flexing their tpu’s on us and selling us a usb accelerator with half a gb of memory called coral.