Building AGI in Real Time (OpenAI Dev Day 2024)

As Ben Thompson has noted, this year’s DevDay took a quieter, more intimate tone. No Satya, no livestream, (slightly fewer people?). Instead of putting ChatGPT announcements in DevDay as in 2023, o1 was announced 2 weeks prior, and DevDay 2024 was reserved purely for developer-facing API announcements, primarily the Realtime API, Vision Finetuning, Prompt Caching, and Model Distillation. We recap all releases and interview some of the product leaders from OpenAI who are behind them.

Chapters:
01:23 NotebookLM Recap of DevDay
09:25 Ilan's Strawberry Demo with Realtime Voice Function Calling
19:16 Olivier Godement, Head of Product, OpenAI
36:57 Romain Huet, Head of DX, OpenAI
47:08 Michelle Pokrass, API Tech Lead at OpenAI ft. Simon Willison
1:04:45 Alistair Pullen, CEO, Cosine (Genie)
1:18:31 Sam Altman + Kevin Weill Q&A
2:03:07 Notebook LM Recap of Podcast

[claudioagmfilho]

🇧🇷🇧🇷🇧🇷🇧🇷👏🏻, I'm eagerly awaiting the release of real-time video for Plus users from OpenAI, as it was originally mentioned as part of the ChatGPT Omni update, which sadly never reached us. This feature will be revolutionary, enabling us to tackle a wide range of daily tasks more efficiently. Real-time video integration within ChatGPT would greatly enhance productivity by allowing for interactive, dynamic assistance and more streamlined workflows. It would be especially useful for tasks like desktop sharing—being able to visually assist and collaborate on real-time activities is just phenomenal. I hope this feature rolls out soon, as it could drastically improve how we approach everyday challenges.

[CurtCox]

Asking Claude:

Read the provided transcript. Provide lists for each of the following that are mentioned:

APIs
Tools
Techniques
People
Organizations
Problem Domains



Here are the lists based on the provided transcript:

APIs:

1. Real-time API
2. WebSocket API
3. Chat Completions API
4. Assistants API
5. Function calling API
6. Structured output API

Tools:

1. ChatGPT
2. GPT-4
3. GPT-3.5
4. O1 (OpenAI's new model)
5. Cursor (coding tool)
6. Genie (coding AI)
7. Cognition (AI tool)
8. Harvey (AI tool)
9. Case Text (AI tool)
10. Notebook LM (Google's tool)
11. Claude (Anthropic's AI)
12. Gemini (Google's AI)
13. Waymo (self-driving car technology)
14. Code Interpreter

Techniques:

1. Fine-tuning
2. Distillation
3. Prompt caching
4. Vision fine-tuning
5. Model evaluation
6. Iterative deployment
7. Chain of thought reasoning
8. Function calling
9. Structured outputs
10. System prompts

People:

1. Sam Altman
2. Kevin Weil
3. Roman Hewitt
4. Michelle Pokras
5. Simon Willison
6. Alistair Pullen
7. Olivier Godemont
8. Ilya Sutskever (implied)
9. George Hotz
10. Jon Frankle
11. Chris Lattner
12. Jeremy Howard

Organizations:

1. OpenAI
2. Anthropic
3. Google
4. Microsoft
5. Cosine
6. Cognition
7. Harvey
8. Case Text
9. Twilio
10. LifeKit
11. Agora
12. Waymo
13. Twitter (former)
14. Coinbase (former)
15. Y Combinator

Problem Domains:

1. Natural language processing
2. Computer vision
3. Speech recognition and synthesis
4. Code generation
5. Language translation
6. Autonomous systems
5. Artificial General Intelligence (AGI)
6. AI safety and alignment
7. AI ethics and responsible deployment
8. Developer tools and platforms
9. AI-assisted software engineering
10. Real-time AI interactions
11. Multimodal AI (text, voice, vision)
12. AI agents and automation
13. AI in government and public services
14. AI for scientific discovery
15. AI user interfaces and experiences