100+ Insane ChatGPT Vision Use Cases

Today we look at 100+ ChatGPT use cases as detailed in the Microsoft paper The Dawn of LMMs:
Preliminary Explorations with GPT-4V(ision).

Links:

---------------------------------------------------------------------------------

#aivision #chatgptvision #gpt4v

---------------------------------------------------------------------------------

[Killerbader]

Its insane how good this all got in just 1 year. where are we in 10 years with this technoglogy? it hast the potencial to change our world forever.

[Onkeldoc]

4:57 incorrect deduction. The bottle of water is full so the content in the glass can’t be from that bottle. Still amazing that gpt-4 points out the possibility.

[TrevorFosterTheFosterDojo]

Dude thank you so much for taking the time to curate all of this and break it down

[gcmisc.collection]

Thank you.. Human for a brilliant podcast..

[ziadnahdi]

Great. Thanks very much. We want more examples like this please

[IndigoGPT]

I don’t have access to gpt vision or gpt voice on my iPhone, and I’m a plus subscriber
Edit: I have everything now. Vision is AMAZING

[FrankysChilloutTime]

love your videos and find them really informative

Amazing. Truly amazing. Thanks for covering this for us!

[augustuslxiii]

I have a hope.

I have a hope that, one day, I - a Plus subscriber from day one - will have access to any of this stuff.

[Tonceitoys]

I just used Dalle 3 to make some album art for a future single I'd like to release and after making some small adjustments to the image I showed the image to this tool asking about what did it transmit to it both artistically and musically and the answer was very interesting and kinda accurate with what I was trying to convey, this is great!

[xuan]

Is it just me or I still don’t have the ChatGPT vision??? I literally check everyday 😢😢😢

[CoderN]

🎯 Key Takeaways for quick navigation:

00:00 🌐 Overview of GPT-4 Vision Capabilities
- GPT-4 Vision extends language models to understand and analyze images.
- The model can process images with contextual understanding, going beyond simple recognition.
01:09 🖼️ Image Recognition for Accounting
- GPT-4 Vision can analyze receipts and invoices, recognizing specific details like tax amounts.
- Provides time-saving benefits for tasks involving large volumes of receipts.
02:35 📄 Template Filling with ID Recognition
- Demonstrates the use of templates for filling out identification details.
- Highlights the potential for structured data extraction from various documents.
03:31 👉 Pointing in Image Recognition
- Introduces pointing as a method for specifying regions of interest in images.
- GPT-4 Vision can understand pointing gestures for precise identification in images.
04:56 🧠 Deep Understanding of Image Relations
- GPT-4 Vision can comprehend relationships between objects in an image.
- Analyzes an image with arrows and circles, showcasing nuanced understanding.
05:38 🎯 Few-Shot Prompting for Improved Accuracy
- Introduces the concept of few-shot prompting for accurate image recognition.
- Demonstrates the importance of providing multiple examples for effective training.
06:21 🌐 Recognition of Celebrities and Landmarks
- GPT-4 Vision excels at identifying celebrities, landmarks, and their associated contexts.
- Understands not only who they are but also their potential actions or attributes.
07:32 🍜 Food Recognition Beyond Visuals
- Impressive capability to recognize and describe specific dishes, even from low-quality images.
- Goes beyond basic food identification, providing detailed information about cuisine.
07:46 🩹 Medical Diagnosis with X-rays
- GPT-4 Vision can identify medical conditions in X-rays, such as fractures or infections.
- Highlights potential implications for medical professionals.
08:14 🏥 Diagnostic Capabilities in Medical Images
- Discusses the ability to infer potential health issues from medical images.
- Raises ethical considerations regarding self-diagnosis through AI.
09:25 😄 Understanding and Describing Memes
- GPT-4 Vision can interpret and describe memes, capturing humor and context.
- Demonstrates the model's ability to understand visual jokes and cultural references.
10:06 🌐 Ecological Understanding in Illustrations
- GPT-4 Vision can interpret complex ecological illustrations, identifying roles in a food web.
- Showcases the model's ability to comprehend intricate visual representations.
10:34 🕵️‍♂️ Analyzing Scenes for Detective Work
- Discusses the potential use of AI in surveillance by analyzing visual clues in a room.
- Raises ethical concerns about the depth of information AI can derive from visual data.
11:30 🏠 Practical Applications for Everyday Tasks
- Demonstrates practical uses, such as understanding floor plans and locating specific features.
- Envisions how GPT-4 Vision could assist in real estate or interior design tasks.
11:58 📑 Summarizing Academic Papers with Visuals
- Discusses the potential for GPT-4 Vision to summarize academic papers with text and diagrams.
- Highlights the model's current limitations in handling complex academic content.
13:25 🌍 Multilingual Translation and Cultural Context
- GPT-4 Vision seamlessly integrates language translation with image recognition.
- Recognizes cultural context, providing translations that align with local norms.
14:07 📰 Reformatting Images for Productivity
- GPT-4 Vision can reformat images into various layouts and formats.
- Offers practical utility for professionals needing quick formatting tasks.
15:02 🕹️ Interacting with Software Icons
- Envisions a future where GPT-4 Vision can assist in understanding software icons.
- Discusses the potential for AI guidance in navigating unfamiliar interfaces.
16:26 🎭 Emotional Analysis in Images
- GPT-4 Vision not only recognizes emotions in images but also understands and describes them.
- Raises considerations about the societal impact of emotion analysis technology.
17:09 🏖️ Image Description and Persuasion
- Describes a rocky beach at sunset with seaweed and algae.
- Discusses the persuasive impact of image descriptions on emotions.
- Mentions imperfections in the model, like inaccuracies in spotting differences between images.
18:06 🛠️ Identifying Irregularities in Images
- Highlights the model's ability to identify irregularities in objects.
- Uses examples like damaged screws and evaluates potential applications.
- Discusses simplifying processes previously requiring human intervention.
19:16 🛒 Analyzing Shopping Carts and Business Applications
- Discusses training models on specific business products for accurate analysis.
- Demonstrates the model's capability to analyze shopping carts from low-resolution images.
- Talks about potential applications in simplifying various business processes.
19:30 🩺 Impressive Medical Image Analysis
- Briefly explores medical examples and the model's accurate analysis.
- Acknowledges some skepticism about cherry-picked examples.
- Notes the model's overall impressive performance in medical image recognition.
19:57 🎨 AI Art and Visual Capabilities
- Discusses the model's ability to rate images accurately.
- Mentions the synergy between language and vision models in creating powerful AI art.
- Envisions the potential of iterative improvements through critique by autog GPTs.
20:41 🤖 Autonomous Agent and Navigation Abilities
- Explores the idea of autonomous agents with visual capabilities.
- Mentions scenarios like home robots navigating environments using visual input.
- Discusses the model's potential in complex real-world situations.
21:39 🌐 Browsing the Web and Internet Search
- Contrasts model capabilities in navigating the internet with Bing's search capabilities.
- Highlights the model's success in a scenario involving buying a keyboard on Amazon.
- Suggests the browsing model with Bing is a simplified version of the true capabilities.
22:36 📱 Analyzing Short Form Video Content
- Demonstrates the model's ability to analyze short-form video content from TikTok.
- Discusses the possibility of perfect transcription using image prompts.
- Explores the pairing of visual capabilities with Bing search.
23:34 🔄 Integration of Plugins and Multimodal Capabilities
- Discusses the integration of plugins for advanced capabilities.
- Envisions a future where all plugins merge into a single multimodal model.
- Talks about the potential of combining text and vision capabilities seamlessly.
24:16 🔄 Self-Reflection and Self-Correction
- Explores the model's ability to self-reflect and self-correct under certain circumstances.
- Describes a scenario where multiple instances of GPT-4 communicate and improve a prompt.
- Highlights the potential for continuous improvement in thinking processes.

Made with HARPA AI

[CM-zljw]

Not everyone is ready?

Oh I was born ready! Do people have access no?

Thanks for the long, in-depth format video Igor. These are really useful and helpful. 🙏🔥👏

[agnesslovehealz]

Love and esp that Watching at 111 lol its amazing cause we are moving at super fast pace and use cases so many love the breakdown

[map]

Whoaaa Czech language!!! You are much closer to me now :)

[extrememiami]

When you provide it training examples does it keep the info forever in its core, available for everyone, only for the remainder of the chat window, or forever but only on your account level? If anyone knows please tell us. I traines it on solving some puzzles and I wonder how long and where it stores that info.

[BartvandenDonk]

Most of it I have seen weeks ago. The end part was new to me. AI improving itself by generating images and reusing the original prompt and interpretation of the dall-e image generated and coming to a correct interpretation is coming soon, I hope. 😁

[ValentineandLance]

When do you think Chatgpt will be able to create accurate illustrations for educational purposes? I know that DALL-E is being built natively within Chatgpt but it seems that the integration seems to be more about refining text prompts to guide DALL-E 3 in generating images that adhere closely to the text, rather than equipping DALL-E with logic and reasoning capabilities (e.g. it would be amazing for Chatgpt to create illustrations for me to enhance my understanding of exercise mechanics and have it create illustrations showing forces).

[galaktuss]

Nice video . only problem i see with travel part mostly have low or no net and gpt not work without. :D

[policani]

I frequently have problems with ChatGPT 4 hallucinating after I give it more than 3 pages of content to analyze. I can only assume this has to do with the small 4000 token context window, so I can see how the vision use case for analyzing more than 5 pages of text and visual content failed. One quick related thought - Stable Diffusion has a context window size of 75 tokens... Not sure how many tokens Dall-E 3 will support, but if it's a similar small number I wouldn't expect too much from a text + vision use case analyzing more than a single page.