'Principle Driven Self-Alignment' and 'Preference Ranking Optimization' [Best Alignment Papers!]

🚀 Welcome to the New Era Pathfinders Community! 🌟

Are you feeling overwhelmed by the AI revolution? You're not alone.
But what if you could transform that anxiety into your greatest superpower?

Join us on an exhilarating journey into the future of humanity in the age of AI! 🤖💫

🔥 What is New Era Pathfinders? 🔥

We are a vibrant community of forward-thinkers, innovators, and lifelong learners who are passionate about mastering the AI revolution. From college students to retirees, tech enthusiasts to creative souls - we're all here to navigate this exciting new era together!

🌈 Our Mission 🌈

To empower YOU to thrive in a world transformed by AI. We turn AI anxiety into opportunity, confusion into clarity, and uncertainty into unshakeable confidence.

🧭 The Five-Pillar Pathfinder's Framework 🧭

Our unique approach covers every aspect of life in the AI age:

1. 💻 Become an AI Power-User
Master cutting-edge AI tools and amplify your productivity!

2. 📊 Understand Economic Changes
Navigate the shifting job market with confidence and foresight!

3. 🌿 Back to Basics Lifestyles
Reconnect with your human essence in a digital world!

4. 🧑‍🤝‍🧑 Master People Skills
Enhance the abilities that make us irreplaceably human!

5. 🎯 Radical Alignment
Discover your true purpose in this new era!

🔓 What You'll Unlock 🔓

✅ Weekly Live Webinars: Deep-dive into each pillar with expert guidance
✅ On-Demand Courses: Learn at your own pace, anytime, anywhere
✅ Vibrant Community Forum: Connect, share, and grow with like-minded pathfinders
✅ Exclusive Resources: Cutting-edge tools, frameworks, and insights
✅ Personal Growth: Transform your mindset and skillset for the AI age

🚀 As You Progress 🚀

Unlock even more benefits:
🌟 One-on-One Mentoring Sessions
🌟 Exclusive Masterclasses
🌟 Advanced AI Implementation Strategies

💎 Why Join New Era Pathfinders? 💎

🔹 Expert-Led: Founded by a leading AI thought leader, connected with top researchers and innovators
🔹 Holistic Approach: We don't just teach tech - we prepare you for life in an AI-driven world
🔹 Action-Oriented: Real skills, real strategies, real results
🔹 Community-Driven: Join 300+ members already navigating this new era
🔹 Cutting-Edge Content: Stay ahead of the curve with the latest AI developments and strategies

🔥 Don't just survive the AI revolution - lead it! 🔥

[Randomuser]

David just want say how much i appreciate the hard work you are putting, Im constantly looking for content about AI and future prediction on how it will impact our society .
Im waiting for every video like it some episode from my favorite series . Keep the good work 🙏

[RecruitingMaven]

Love the explanations as they help to understand this topic more profoundly. Thank you!

[Wingularity]

Fantastic vids all around. Thanks for the work you put into these AI videos.

[angelo]

Thanks David Shapiro! Your work does not go unnoticed! Your videos are my go to when eating breakfast. My favorite form of "news or content" that I want to be updated on. I read through the papers you suggest for research, consider your opinions, and always look up concepts, words, or frameworks I hear for the first time.

[krissnoe]

Expert system could be expanded to Pantheon in form of structure, a set of large AI with specialties that generally force them to work together in terms of simple data reflection. Imagine a Pantheon of super intelligence discussing these topics as if sitting at some table together representing the good-faith of their creator's.

[vincentcaudo-engelmann]

Nothing useful to say but - this is so freaking awesome.

[Dan-ojiq]

The TLDR here for me here is the statement "human values are unreliable as a signal". I actually had to play back that piece of the entire video multiple times. Not sure how playing Dave's videos at more than 1x speed would be workable. Mt age may have a lot to do with that.

[emile]

The social media analogy was eye opening!

[DD-jtyn]

I listened to the presentation again. It was referring to considering commercial driving as being outlawed and unethical. Not recreational. It makes sense.

Perhaps in the not so distant future will invent new forms of jobs where only enhanced humans will be able to drive for commercial purposes. That might also be a solution to the problem of safety.

It would be a terrible loss if humans would lose the ability to drive a vehicle. It is exciting to imagine a future where anyone could operate a land or flying vehicle, for either commercial or recreational purposes.

Coming from a layman’s perspective, drawing timelines would be helpful to help more people be involved in these important conversations and appease some of the fear AI and automation is bringing.

Most people have limited time for the in-depth research necessary to gain full clarity and understanding. And many times, me included cannot tell the difference between a serious response or an intelligent insider’s joke. 😅

Once again, thank you for the excellent content you are making!

[nicholasmordin]

Please discuss the practicality of offering all users of GPT-4 and Claude access to their data analysis capabilities. Code Interpreter has been very glitchy in the last week and unable to answer complex questions. Claude's ability to analyse files has just been taken offline. If you look at the processing power you would need to run software like Code Interpreter on your own PC, you'd need a high end PC with lots of RAM. It is surely not economic for Open AI or Anthropic to offer all their users virtual PC's worth thousands when they're only charging them 20 bucks a month or nothing at all. Does this not suggest a move away from online AI to personal AI similar to the move from mainframe to personal computers that happened in the 1980's? Why is nobody talking about this?

[jamespercy]

David seems to be proposing a core value that makes the conversion of adversarial processing into opponent processing wherever possible an imperative, the difference between the two being that opponent promising is premised on a shared narrative and a shared future.

[isajoha]

Great video. I wonder what might be the correlation between a lot of Alignment "code" and Attention that seems to be a kind of serious problem in reaching Utopia. How much data can a LLM keep track of? Is there some physical limit? 🤔

[jamespercy]

One could suggest that an algorithmic imperative to convert adversarial processing into opponent processing wherever and as far as possible without surrender leads to cognitive attunement and resonant cognition(optimal fluidity of instantiation of cognition balanced with optimal anticipation and correction for unintended consequences). Just a thought,

[JorgetePanete]

Do we, as humans, even have axioms for morality? Is every right axiomatic?

[hobo]

Hey dave you should make a video on how ai can become autonomous people are kinda confused how it can gain autonomy

[christian-schubert]

Hm... Everytime I listen/read about AI Alignment, it always leaves that sour taste of potentially stifling/lobotomizing AI's full potential in unwanted ways before we even have a chance to arrive at AGI in my mouth.

I also don't buy all that "AI is already sentient/self-aware!" crap. The best AI models are still narrow AI models. Of course they appear AGI-like, they've virtually been trained on all of the Internet (and whatever other sources we don't know about)! Claiming that one of the next GPT iteration will become "self aware" / "AGI" (or even "ASI" mind you) if none of the underlying core principles change in fundamental ways (not denying that that could and most probably will happen with all the research going into it) is like building a car and then planning a trip to the bottom of the ocean.

A lot of "Safeguard mechanisms" also seem to be mere excuses for simply not exposing intellectual property. A fair concern, but let's call a spade a spade and keep it real. It's problematic at least in a sense that it is also hindering progress.

I'm not keen on a Terminator-like future either, all I'm asking is for some awareness of the potentially dark side of over-alignment. This could ultimately prove detrimental as society's current norms and policies are clearly not the optimal way of running things. Or in the words of Winston Churchill: "Democracy is the worst form of Government except for all those other forms that have been tried from time to time".

Well, what if there WERE better alternatives to democracy that we inadvertently "safeguarded" against? Also, I 100% guarantee you, the moment an AGI comes up with reasons why infringing on some big Tech's intellectual property might be the only sustainable way to move forward, it is going to be realigned due to "safety concerns" not due to Big Tech Business Interests.

[asi_karel]

Look what I have found:

Model: Code Interpreter
What version are you?
I am an instance of OpenAl's GPT-3 model, specifically the chat model known as ChatGPT.
The training data I was trained on includes information up until September 2021. Since the model itself does not have version numbers like software might, it's not possible to give a specific version number. Any updates or improvements made to the model since my training data cut-off wouldn't be reflected in my responses.

Model: Plugins • Enabled plugins:
What version are you?
As an Al developed by OpenAl, I don't have a specific version number like software applications do. However, as of my last training data, I'm based on the GPT-3 model. Please note that my training data only includes information up until September 2021, and I may not have the most recent updates or information beyond that date.

Model: GPT-4
What version are you?
lam based on the GPT-4 architecture, which is an iteration of OpenAl's language model. The 4 Q information I am providing is accurate up to September 2021, which is when I last received an update in my training data. After this date, I can't provide real-time data or updates.

[sebstr]

You say you had discovered some of these techniques before there was made papers on them. so wouldn't it be great if you tried collecting your own thinkings in to research papers so the people that only read papers also has a chance to see your great ideas.
If you was able to do that then it would probably accelerate the research in alignment.

[Diabloto]

What is concerning still is all these techniques just fine-tune there is no true deep/axiomatic embedding, you can still jailbreak all these systems and get the complete opposite of what is was fine-tuned for.

[zacharyaustin]

Can you do a follow up on how AI can balance conflicting principles? We live in a world of paradox.. (you went on to say human values are inconsistent) I think there is more work to be done here. even if we had perfect alignment capability we need to know what we are aligning with. In any framework the definitions matter greatly such as with the constitution of the US, second amendment has a lot of support that civilians should be able to use guns but it really matters how the right to bear arms is defined. (militia, hunting, self defense, suicide and killing people are all use cases). I am trying to say that if we do not want the AI to be its own Supreme Court interpreting the principles we need to spell out the definitions in a way that logically balances them. This is where I believe we are not ready for AGI because we dont have a robust framework that has balanced principles well. Just because frameworks and codes of ethics exist doesnt mean this is a solved problem and we as a species need to figure this out, together. I believe this if anything could lead us towards a hegemony with one framework rising above all others even if it is not a good framework (any sci-fi empire). I agree with your conclusion that experts need to be included on developing framework for AI and I personally believe the church has the most experience creating frameworks for humanity with systematic theology. It is fascinating to view the pentateuch as such a framework for society, if it is indeed easy to use principle driven self-alignment I wonder how far we could get with the pentateuch. we have a lot of more recent philosophy such as with moral uncertainty but I seriously want to experiment with promised land 2.0 using ai agents equipped with pentateuch and seeing what civilization and governance would look like. It doesnt feel far off that we will have a digital twin of earth to run experiments with. obviously you could do this with any framework but I dont feel like using the Hammurabi code would be functional.