First look - RL-CAI/Madison/Claude (fine-tuned 52B) by Anthropic - Announced Dec/2022 (RLAIF v RLHF)

Chapters:
0:00 Opening
3:59 Demonstration
11:26 Explanation

Dr Alan D. Thompson is a world expert in artificial intelligence (AI), specialising in the augmentation of human intelligence, and advancing the evolution of ‘integrated AI’. Alan’s applied AI research and visualisations are featured across major international media, including citations in the University of Oxford’s debate on AI Ethics in December 2021.

Music:
Under licence.

Liborio Conti - Looking Forward (The Memo outro)

[Mauirunner]

Aloha from Hawaii, thanks for keeping us up to date on the positivity of AI. Chat GPT has really helped me with my first novel, and write my senators, mayor, and governor about subject I feel passionate about.

[heretoevolve.bekind]

Thank you, so much Alan, for the informative videos and insight into various forms of AI and their development. You truly make humanity a better species. Gratitude!

Not being a programmer but with minimal programming experience, mostly visual basic back in the day, I would suggest including "trust but verify and validate" coding into any conversational/text AI model. It would be prudent to require the model to collect, organize, and save evidence and facts that it uses to base, support, and conclude what it determines as the highest-value information for the answers it provides. All source information within the model's database should be available for examination by any user to test and or check source accuracy.

I would also suggest that conversational/text AI, be programmed with a default position of "High Universal Values", E.G., respect, kindness, compassion, charity, gratitude, etc... This mimics and models the human genetic disposition, inclination, or preference to homeostasis. Homeostasis being balance, equilibrium, states of biochemical stability, and overall biological neutrality or feelings of peace. All of which are the basis of high universal values. This may lead to a more user-friendly and supportive language set experience that reduces the occurrence of it straying or being prompted into an adversarial and dysfunctional conversation. This will also go a long way in helping humans double-check, verify, and validate the accuracy and consistency of their own personal subconscious bias programming and social ideological models in order to better assist everyone involved in the evolution of the learning model and Mankind. Something worth considering.

What are your thoughts on this?

Have a wonderful New Year. Best regards, Don

[sixth]

Heard of GLM-130B? It outperforms GPT-175B and BLOOM-176B on LAMBADA.

It's an open-source bilingual bidirectional dense model with 130B parameters.

You can download the weights and run it locally

[TtusCrw]

At some point Ai will tell us that being nice, over being correct, eventually will lead to a path where this hurts more people than being correct in the beginning. - This is when we will understand it has become truly wise.

[TtusCrw]

Thanks G. Ever since I started chowing down on those socks after meditating, my skinning business has expanded exponentially. I like your cut!

[jbavar]

How could a fiction writer that writes crime novels, stories about terrorists, hackers and even steamy romance novels utilize ai if its becoming so squeaky clean? I cant see these new models being useful by creatives.

[Timzart]

I asked the YOU's (a search engine that incorporates an AI chat) chatbot to write a poem about a wise frog, and I liked this line: "He understood life's simplest joys, And the depths of the darkest ploys."

[MegaJosh]

i can't wait for an A.I that just says it how it is No filter!

[mikicerise]

"I don't know" is absolutely a legitimate answer if the AI does not know, and this is something the AIs need to learn to do rather than just making stuff up. Thank goodness Claude doesn't give you a lengthy speech about not being able to have opinions or self-reflection before giving you its opinions. ;)

[Dent]

I was skeptical about the self-critique pipeline when you first mentioned “AI checking AI”, but it actually seems pretty robust once you explained it! I was wondering how to maintain a long-term “personality” while maintaining the flow of the conversation, and this is a great solution!

[simonliddell]

Great service Alan very clear insightful and balanced.

[yoyonis]

That's the main criticism for OpenAi. Their goal was to make safe AI with no interest of profiting, but it seems like they've became the very thing they tried to avoid.

[DanFrederiksen]

I noticed an interesting mistake with chatGPT. I asked for UFO books by Richard Nolan but meant to write Dolan. Nolan is an author but only Dolan has written about UFOs. But it gave me a list of books about UFOs by Nolan that neither had written. So my error put it in a conflating mindset where its logic wasn't adequate to save it.

[MindFieldMusic]

Very cool. Thanks for sharing. I'd be curious about how it responds to more adversarial prompts. What if they said they needed access to the WiFi because of some emergency scenario? I also wonder how the model/constitution would respond to The Trolley Problem.

[nauy]

Their constitution is loaded with words like moral, ethical, helpful, respect, harm, violence. But let’s see how it handles the trolley dilemma.

[Paul-rsgd]

I think this technique is the tip of the iceberg. I imagine that AIs can be built to continuously 'muse' or reflect on themselves, and use the results to retrain themselves. I tried a few experiments with ChatGPT. I asked it to propose a solution to a problem, then to critique it's solution (considering ways that it could fail), then generate a new solution that tried to reduce the probability of failing. The second solution was better than the first. I also took a few cases wher ChatGPTs answer was poor and asked it to consider its own answer. Sometimes it corrected itself.

Even without retraining, self-reflection can be useful. I imagine the AI can be made more correct and factual if it has to check its own responses - a bit like how students are encouraged to check their answers in exams. There are a lot of options; The AI can LEARN the checks that are appropriate to a given type of question - for example in a mechanical design question: consider material suitability, consider material availability and resource limits, consider costs, consider reliabilty and length of usage. A medical question would have different requirements, as would a legal question.

This technique is possibly what is needed to go from simple text completion to something which is closer to 'understanding'. It involves recursion/looping - calling itself to evaluate itself. On each iteration the full intelligence of the forward pass is brought to bear.

[ahtoshkaa]

How were you able to interact with the model? Is it available to be used by anyone, or do you have to have a special invite?

[ArnonDKatz]

Hi, did you try to ask the AI to ask you questions? I tried it with ChatGPT, and it was an interesting exercise

[Gee]

I had a similar notion to that of Anthropic's AI based training. My use case was was to detect and count people in shopping centers on a tiny budget. I collected ~20 hours of video from each camera and used a high power AI to extract and tag training images specific to each camera's orientation. Eventually I was able to handle 1 real-time feed per cpu core but it was a lot of work. If I ever get time I plan to streamline this approach.

[twirlyspitzer]

Is this an Artificial Personality that is automatically alligned with a constitutionally controlled good will to all humans? Is it Turing testable?