Claude's 100K Token Context Window is INSANE!

OpenAI's competitor Anthropic just released a new version of their Claude LLM with a 100K token context window. We will explore this powerful LLM in this video.

LINKS:

▬▬▬▬▬▬▬▬▬▬▬▬▬▬ CONNECT ▬▬▬▬▬▬▬▬▬▬▬▬▬▬
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
All Interesting Videos:

[googacct]

I suspect that at least for a while there will be a big competition as to which company or open source project can offer the largest context size.

[XoLucyna]

I recently got access to Claude's 100k model and it is insanely fast considering the fact i posted 5 research papers as mixture without any title and thats over 29k words. And it successfully summarized and could name each paper correctly in just 2.1 seconds. Sometimes the time got higher but it always stayed around 0.1-4.5 seconds.

[interspacer]

The problem with context windows is they scale exponentially and are inclusive of the reply no? If I put in Gatsby, I can't have it generate another version of it.

Also, the longer a conversation goes, the more context bloat happens unless there is no token penalty for repeated context. This however only applies to explicit context. The moment the LLM infers or implies, you eat up the window.

I dont thing embeddings are going anywhere, and I think novel systems like llamaIndex are going to take over. Chunking is inevitable.

[GothicDragonX]

All I want is an AI that can view your documents. It doesn't need all that memory. 50k would be more than enough, and continue to scan relevant info in your record for answering the person's question or use whatever information it has to produce the user's desired result.

For example, as a writer with world-building and multiple characters, building, and location, it would be amazing to ask the AI to pull in various information about a character's interaction with other characters, items, events, or locations. Most writers are often okay with around 80k tokens for a complete book unless you are going for a thousand pages long, which is rare. Most people would split their books into multiple series like J. R. R. Tolkien's Lord of the Ring, JK Rowling's harry potter, and Terry Pratchett's Discworld series.

A personal LLM that can view user-generated content offline would be more worthwhile. Let Google, Microsoft/ OpenAI, and others fight the AGI space.

[Rafael-rnhn]

Until its available for use with a consumers grade GPU (up to 4090), it's essentially not worth it. I'd much rather a significantly lower context size that I can confortably run on my machine, and which does not require me to share my data, than all the computing power in the world. Besides, none of these cases showcase something so groundbreaking you absolutely _must_ get these services. Open source will soon catch up to whatever is available in the commercial space.

I still believe an easy to use off-line assistant with access to tools and a memory database will be the killer app in this space.

[Viewable]

Embeddings will never become obsolete because there will always be users who want to feed LLMs with more data than the context can hold.

[greendsnow]

100K tokens = 75K words = 150 pages.
If we can do 1M tokens in the near future, this will make a revolution in the Academics.
The problem is PRICE.
Claude's Input price is 16 cents per 100K tokens and output for 51 cents for. basic questions..
and for complex questions 110 cents per 100K tokens.... That's expensive.
Use embeddings for your pocket.

[naniwho]

You forgot to leave a link to your discord

[Zolipants]

All i really like to see that is someone using this fuckin AI for something useful apart from getting rich!

[picklenickil]

The real question is, whats the point if it requires 1000gb of VRAM

[griffinquarles]

ChatGPT with gpt4 only allows inputs of 2048 tokens, it still has the context window of 8k for the conversation though

[muhammadsaqifhasbullah]

Huh, now the claude 2 only have 5 time use and 1 if outside the US. Note: I'm using VPN and is disappointed that Claude 2 is now only 5 instead of the usual 30.

[kaynkayn]

Sure, you can do 100k tokens but I doubt it works well. OpenAI has a large token window and it still struggle as context gets larger. Less token with concise and clear prompt will always be better than larger tokens.

[squeezyDUB]

This has nothing to do with sam altman pleaaaase

[mirkakonest]

Premieres suck! Couldn't you just post your videos without a premiere, because that leads to a useless distraction when the videos aren't ready, and then missing them when they finally show up but are already lost in the subscription feed. 😐

[piotrjee]

It's good to see progress, but no thank you this is API only model. I am not going to feed it with data, will wait for open-source 13B+ LLMs. Good content BTW

[aamira]

Tried using your discord link, message I got was unable to accept invite.

[AnthonioPettit]

Thanks for the hot lead. It's amazing how quickly 8000 tokens gets filled up you're using GPT-4 this is going to make a lot possible!!

[pizzaiq]

Per 1000 tokens they're about the same as openai. Gpt 3.5 is still the cheapest

[ozzymr]

buen video.. me sumo al discord, porfavor pone un canal en español