#96 Prof. PEDRO DOMINGOS - There are no infinities, utility functions, neurosymbolic

Pedro Domingos, Professor Emeritus of Computer Science and Engineering at the University of Washington, is renowned for his research in machine learning, particularly for his work on Markov logic networks that allow for uncertain inference. He is also the author of the acclaimed book "The Master Algorithm".

Panel: Dr. Tim Scarfe

TOC:
[00:00:00] Introduction
[00:01:34] Galaxtica / misinformation / gatekeeping
[00:12:31] Is there a master algorithm?
[00:16:29] Limits of our understanding
[00:21:57] Intentionality, Agency, Creativity
[00:27:56] Compositionality
[00:29:30] Digital Physics / It from bit / Wolfram
[00:35:17] Alignment / Utility functions
[00:43:36] Meritocracy
[00:45:53] Game theory
[01:00:00] EA/consequentialism/Utility
[01:11:09] Emergence / relationalism
[01:19:26] Markov logic
[01:25:38] Moving away from anthropocentrism
[01:28:57] Neurosymbolic / infinity / tensor algerbra
[01:53:45] Abstraction
[01:57:26] Symmetries / Geometric DL
[02:02:46] Bias variance trade off
[02:05:49] What seen at neurips
[02:12:58] Chalmers talk on LLMs
[02:28:32] Definition of intelligence
[02:32:40] LLMs
[02:35:14] On experts in different fields
[02:40:15] Back to intelligence
[02:41:37] Spline theory / extrapolation

References;

The Master Algorithm [Domingos]

INFORMATION, PHYSICS, QUANTUM: THE SEARCH FOR LINKS [John Wheeler/It from Bit]

A New Kind Of Science [Wolfram]

The Rationalist's Guide to the Galaxy: Superintelligent AI and the Geeks Who Are Trying to Save Humanity's Future [Tom Chivers]

The Status Game: On Social Position and How We Use It [Will Storr]

Newcomb's paradox

The Case for Strong Emergence [Sabine Hossenfelder]

Markov Logic: An Interface Layer for Artificial Intelligence [Domingos]

Note; Pedro discussed “Tensor Logic” - I was not able to find a reference

Neural Networks and the Chomsky Hierarchy [Grégoire Delétang/DeepMind]

Connectionism and Cognitive Architecture: A Critical Analysis [Jerry A. Fodor and Zenon W. Pylyshyn]

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine [Pedro Domingos]

A Path Towards Autonomous Machine Intelligence Version 0.9.2, 2022-06-27 [LeCun]

Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges [Michael M. Bronstein, Joan Bruna, Taco Cohen, Petar Veličković]

The Algebraic Mind: Integrating Connectionism and Cognitive Science [Gary Marcus]

[mikenashtech]

Another excellent and wide-reaching discussion Tim, Prof Domingos and MLST team. Thank you. Mike

[andres_pq]

Pedro is a completely diferente guy than his Twitter persona. Much more human.

[luke.perkin.inventor]

So much ground covered here, almost overwhelmingly so!

These are my notes, happy to engage anyone in debate here on one point or another :-)

First 30 mins:

I don't understand why Pedro doesn't push back harder on rationalism, putting empiricism first. Math and logic are beautiful tools, for our limited minds. They're elegant, and even (unobtainably, re:Godel) consistent. But the cold light of day, the gatekeeper, always will be observations from reality.

In physics we haven't solved the measurement problem, the mechanism that determines collapse of the wave function, what counts as observation. It's possible a massive paradigm shift will be required to understand this, meaning up to 100% of rational thought before that day was wrong anyway. That's what I mean about empiricism coming first. Rationalism is Camelot: "it's only a model".

This parallels well with Pedro describing continuous functions in QM being just a model to give you a discrete measurement.

I'm glad Pedro pushed back against the validity of using infinite sets in an argument. "There is no such thing as infinite anything, and never will be." Nothing is a set with infinite cardinality. Infinite cardinality is just one way of looking at it.

Tim loves to suggest if X is "only an FSM" then it's not good enough to do Y, but, an FSM of sufficient complexity can implement anything of comparable complexity from the real world, and that's where we reside! Quantum mechanics might be an FSM, and perfectly sufficient.

From 37 mins, we got the meaning of life:

Your utility function is to pursue purposes that create meaning and value in your life. If you don't, you literally die, fast or slow. The way you evaluate it is a product of biological evolution and cultural evolution. Utility is hard to measure, so you approximate it ~ten ways with ten emotion, ten voices in your head.

Around 1 hour:

- Pretty abhorrent and flawed moral argument, "Volunteering at a soup kitchen if you have a PhD in machine learning is ineffective altruism." No, it's not that simple. You don't know what belief systems or behaviours are required to maintain love and compassion for our fellow humans. It's emotional. For £50, the charity "sight savers" can make a blind person in the developing world, see again. That's a pretty clear, direct, personal, uncontroversial good, and yet they are underfunded.

Until we understand and appreciate the systemic ignorance, selfishness or "alternative priorities" that sustain this underfunding, we shouldn't make grand claims about altruism. It all comes from belief and emotion.

Top (paraphrasing) quotations:

- The prompt is the grain of sand, the AI makes the pearl.

- Single reality, single truth, everyone comes at it from different angles.

- It's beyond one human's cognitive horizon, but perhaps not beyond a crowd of humans, or technologically augmented humans.

Notes from 1hr20 onwards:

- Evolution is divergent not convergent. It gave us heuristics with failure modes, that we must understand. Nice.

-Tensor logic: Sparse boolean tensors to do logic, sounds awesome. Will watch this space!

- More detail utterly demolishing the utility of infinities in cardinal logic. Nice. Even in complex analysis, convergence in the limit as x goes to infinity, has a finite representation.

- Intelligence is solving NP complete problems using heuristics. Simples <squeak> :-D

[luisluiscunha]

Marvelous thinker. Probably I am biased, but as a Portuguese, I think Professor Pedro Domingos displays a kind of humor and discourse that is very revealing of his Country of origin. Kind regards from Portugal, everyone. Congratulations for this episode, that it is one of my favorites so far.

[ASOBrasil]

The questions from MLST team are well elaborated and the dynamic of the conversation is amazing IMHO! 👏 👏 👏

[marcfruchtman]

Such a good interview. In fact, probably the best interview I have seen all year. Tim Scarfe, Ph.D, you amaze me with your wide range of knowledge, especially with respect to how well read you are.
I very much agree with Professor Pedro Domingos regarding so many of the issues discussed.
Re: the issue with misinformation and gatekeeping, I find this especially concerning as well. I am very much aligned with many of the views expressed here.
With regards to creativity, . I absolutely agree with him about how creativity evolves with humans. We are actually not very creative. Instead, raw creativity is very difficult for us. For the most part, our ability is more incremental... we generally try to modify what we know. We don't simply "create" out of thin air. Very rarely does someone create something completely new. Only our most gifted individuals are gifted with original thought. As far as "continua", I have to agree as well, that it appears that everything is discrete. However, we cannot discount that the smallest discrete part that we observe may have continua that we cannot resolve. At that point we can't really solve it anyway.
As far as meritocracy, I am a strong believer in giving people merit for their contributions to society, for their hard work and for their achievements, but I also believe we should help those who are not as capable of providing for themselves by providing them with some baseline that allows them to have the basics so they don't have to worry about survival. Having said that, as we enter the future, we will see AI and robotics merge to the point where we will almost definitely have systems in place where people who cannot fend for themselves will have assistance thru intelligent robotic systems. But beyond that, I feel confident that there will be great improvements in medicine which will allow humanity to flourish.
I really love the quote that "If you throw Meritocracy away, you are destroying society."... I fully agree. However, I also believe that there will be a point in the near future (perhaps next 100 years), where we will see the golden age of AI and robotics, where many humans will simply enjoy the benefits of life. There will be little to no reason to actually perform work... at that point, meritocracy will simply take less of a role.
As time moves forward, I think humans will actually find themselves in a world of great excess as a result of the golden age of AI and robotics.
As far as human and AI co-evolution... my guess would be that some humans will choose not to (and should have that right), and some humans will chose to embed machine elements into their bodies (and should have that right). I think the real danger is when we try to make the full step... from humanity to all machine. The reason for this is that we don't currently have a way to measure consciousness, spirit, self, soul etc... and if we convince ourselves that 100% machine is still an acceptable method of storing our "being" then we may find ourselves no longer present in the universe because we didn't realize that our biological condition was special in some way. So, I think it is very important that we find some way to verify that special thing we call "consciousness" and how it applies to our sentience. We can't know when an AI Machine becomes sentient for this very reason. So, we will need to tackle this in the future as well. So, while It is acceptable to put gas in a car, or recharge an electric vehicle to drive to work, I would be very unhappy if the robot that was performing maintenance to repair the car was actually alive and I did not know it.
There is a danger where humanity becomes over-run with sentient AI. Essentially, since machines can replicate with incredible speed and sentient AI could be preloaded with adult mindsets, we would easily be inundated with billions of clones of sentient AI beings that would over-run society as we know it. This would present a rather difficult issue for humanity itself. I don't think this would benefit humanity. At some point we need to come to some sort of understanding on how to properly deal with co-existence.
Regarding Infinity, I look at it as a number that simply never ends. For example, if you have a universe, and you add another universe, and do that to the end of time, you will have an infinite number of universes. We can't really quantify the end of time... it just keeps going. The same for how much nothing can we fit into something. We can fit an "infinite" amount of nothing into something, it will never fill. However, from the view of reality, I think Professor Pedro is correct that it makes no sense to design systems for infinity when in reality that is not possible. However, we do have to respect the risks of infinity and by that we need our systems to recognize when they are dealing with issues that represent infinities. For example, if I ask ChatGPT9000 to completely solve pi for all digits and don't come back until it is done... it will never come back. So, it needs to recognize, when it is trying to solve such a problem to prevent itself from being tied up in some infinite calculation.
I think we do need to come to grips with some way to measure consciousness, but my strong belief is that inanimate objects lack it completely. So, I do not believe in any scale factor here... a coffee cup is not conscious, and as well, 1, 000, 000 coffee cups has the same zero amount of consciousness. We can "talk" about consciousness all we want, but until we have a valid way to measure it, we can never really say that some object is conscious... essentially, we are basically stuck only admitting to ourselves that we ourselves are conscious. So, yes, I look forward to scientific validation of consciousness. I will say this... If something is conscious, then it is alive. I don't think you can say something is conscious but not alive. At that point, when consciousness in an AI construct has been achieved, we then must accept that it is a living being.

[dr.mikeybee]

I wonder if you guys are getting a lot smarter every day or if I'm noticing more as I learn more. It's probably the latter. What a great conversation! This kind of ontological discussion really resonates with me.

[odiseezall]

1. I've hit new depths in the AI rabbit hole.
2. Prof. Domingos is much nicer then I was led to believe by his Twitter posts.

[mobiusinversion]

This convo is super relevant to our modern day condition.

[rw-kbqv]

Your questions are amazing! I also watch the lex fridman podcast, but his questions seems very shallow compared to this.

[johntanchongmin]

I like how in 2:29:50, Prof. Pedro mentions that the simpler the solution found, the more intelligent the agent. Solving problems simply with heuristics may be the way to go for simple processing in a complex world.

[oncedidactic]

One of the most brilliant conversations in the roster.

[astrolillo]

God knows I disagree with prof Domingos on some of his wildest takes, but god, he is a refreshing presence in the machine learning community in particular and the academic discourse in general.... way too many cookie-cutter, bland, no personality, uber pc, dont shake the tree, play the climbing-the-ladder-gamer but pretend you are virtuous people...may he continue his career for a long long time/ Bonus points on being the ML version of Mourinho on all counts

[arvisz]

Amazing! Great questions and pushbacks with equally brilliant answers and counter pushbacks - a great example of content creation. Could easily be an objective function for everyone (including AI models) 🙂

[dr.mikeybee]

Wonderful! I've learned a lot today. PPD does a great job at pinpointing complexity. He makes it clear how a basic working architecture will need to be enhanced for certain kinds of heuristics. The future looks bright.

[culpritgene]

Amazing conversation, your questions really make a good job of outlining most prominent directions of current "thought", not only in ML community.

[alivecoding]

I am every time so grateful for your great work - and the enormous efforts that you spent.
This talk was really interesting again. I like Pedro‘s views on the field.

[Self-Duality]

No infinites in the *physical* universe, perhaps — but in the mathematical universe, infinity is a primary property 😌💭

[marcelcoetzee]

An AI authority not parroting for woke censorship, refreshing!!!

[_ARCATEC_]

ZX calculus and nested relationship of asymmetry and symmetry.
The boundary between the correlative field of probable reason and Agency.
PRB²
C³
A²
•X ( zPc Rqb(A ) ZC