Groq API - 500+ Tokens/s - First Impression and Tests - WOW!

Groq API - 500+ Tokens/s - First Impression and Tests - WOW!

👊 Become a member and get access to GitHub:

🤖 AI Engineer Course:

📧 Join the newsletter:

🌐 My website:

Groq:

In this video I give my first impression and to multiple test on the Groq API. Like real time speech to speech and comparing Groq to ChatGPT. A new chip design for running inference on AI apps like LLMs. Thanks to Groq for giving me early access!

00:00 Groq API Intro
00:45 Groq LPU
01:45 Groq Real Time Speech to Speech Test
06:07 Groq vs ChatGPT Test
09:33 Groq Chain Prompting Test
11:15 Conclusion

[GroqInc]

Thanks for the demos. We love what you're doing.

[kate-ptny]

I have joined your membership. Now I just need to get the groq API application approved. Thank you again Kris for sharing.

[khalifarmili]

yes i did enjoy it, thanks for the video, keep it up ❤

[ATLJB]

This guy is too smart for my brain to process

[pauluk]

Been testing this out online and been getting around 400 to 550 tokens on it, it's crazy fast.

There are only two models it lets you select, but both are big models and run lightning fast compared to any other A.I. model I've seen online and locally.

[limebulls]

A video from you about Avatar AI would be awesome! Haven’t found one yet

[theflipbit]

You wouldn't believe, I was experimenting with Siri and groq api, and I asked the same question on summarizing the "attention is all you need" paper even before coming across this video. I mean I am spooked here, what are the odds of that happening? We humans obviously do behave in patterns. lol

[THE-AI_INSIDER]

Groq api I have been using to use the mistral 8x7 and it is currently letting me use it for free, but until when it is free any idea ?

[hqcart]

What matters most is the first token latency, question is does grow has edje on that?

[KodandocomFaria]

I was wondering if you know anything about airllm? I read that this inference is capable of loading a 70B model on gpus as small as 4gb, but I don't saw no one speaking about that

[indikom]

How expensive is that kind of voice conversation using Groq API?

[sirrobinofloxley]

It's amazingly, stupendously and miraculously FAST, isn't it, haha

[JNET_Reloaded]

you didnt show how to set this up i see the site now what?

[JaredWoodruff]

I need this kind of inference speed in Skyrim with GPT :D

[IvanBialotski]

Where the membership link? You just provide the like to your YouTube Chanel.

[ScottWinterringer]

Its actually depressing that hardware made for int math wasnt made already.

[orksbsp]

so 5x faster than openAI but the card cannot be used to train your own models...

[teebu]

nvidia killer. im sure they're also working on a training dedicated card... and if not them, someone else. a lot of companies are going to try to eat nvidias 2T dollar lunch.

[thierry-le-frippon]

Not too late for Meta to cancel its order from Nvidia 😅😅😅

[HectorDiabolucus]

We should start using AI for something that is more useful than playing games and generating p0rnographic images. Its not a toy.