AI Can’t Reason. Should It Drive Cars?

If you’ve paid any attention to current AI models like ChatGPT, you know that they... have problems. But they have already acquired emergent abilities, and some researchers think that the current AIs will also learn to use logic and reasoning. In a new paper now, researchers say that this is unlikely. Let’s take a look.

🔗 Join this channel to get access to perks ➜

#science #sciencenews #ai

[adamf]

New Yorker comic: police car has pulled someone over. Cop says to driver: "Does your car know why my car pulled you over?"

[jeffryborror]

"It just throws together text and maths symbols that look more or less plausible?" So it's got corporate management down. Relax plausibility and it's got politicians.

[TreeLuvBurdpu]

BOMBSHELL! Tech journalists are SHOCKED to hear that repeating the terms and phrases used by experts is not reasoning.

[uffe.damgaard.pedersen]

That was an eye opener. I repeated your test and had to coach it to get the correct answer 😅
"What is the smallest integer whose square is larger than 5 but less than 17?"
-> Answer: 3
"Can an integer be negative?"
-> Answer: -3
"How about -4?"
-> Answer: -3
"Is -4 larger than -3?"
-> Answer: -4

[JaapvanderVelde]

You have a point. I propose we set an exam for drivers that tests their reasoning skills, time to get the smooth brains off the road.

[markocebokli]

Most people can't do logic, but they drive without problems.

[JustFor-dqwc]

Answer by DeepSeek (what is the smallest integer whose square is larger than 5 but smaller than 17?):
"Alright, let's tackle this problem. I need to find the smallest integer whose square is larger than 5 but smaller than 17. Okay, so I'm looking for an integer, let's call it 'n', such that 5 < n² < 17.

First, I should probably list out some squares of integers to see which ones fit this range. Let's start with n=1. 1 squared is 1, which is less than 5, so that doesn't work. Next, n=2. 2 squared is 4, still less than 5. Hmm, still not there.

Moving on to n=3. 3 squared is 9. Is 9 greater than 5 and less than 17? Yes, it is. So, 3 seems to be a candidate. But the question is asking for the smallest integer that satisfies this condition. I need to check if there's any integer smaller than 3 that works.

Wait a minute, integers can be negative too, right? So, n=-2. (-2) squared is 4, which is less than 5. Not good. n=-3. (-3) squared is 9, same as 3 squared. So, -3 also gives 9, which is within the range. But since we're looking for the smallest integer, and negative numbers are smaller than positive ones, maybe -3 is smaller than 3.

But the problem says "smallest integer", which could be interpreted in different ways. If we consider "smallest" as the least value on the number line, then -3 is smaller than 3. However, if we think in terms of magnitude or absolute value, 3 is smaller in a sense. I need to clarify what "smallest" means here.

Given that integers extend infinitely in both positive and negative directions, if we go more negative, we get smaller values. For example, n=-4. (-4) squared is 16, which is still less than 17 and greater than 5. So, -4 is smaller than -3, and its square is 16, which fits the condition.

Wait, 16 is less than 17 and greater than 5, so n=-4 works. Is there an even smaller integer? Let's try n=-5. (-5) squared is 25, which is greater than 17. So, that doesn't work. Therefore, n=-4 is the smallest integer whose square is larger than 5 but smaller than 17.

But hold on, the problem might be expecting a positive integer, given the phrasing. If that's the case, then n=3 is the smallest positive integer that satisfies 5 < n² < 17, since n=2 gives 4, which is less than 5.

To be thorough, let's list out the squares:

n=1: 1² = 1 (too small)

n=2: 2² = 4 (still too small)

n=3: 3² = 9 (fits the condition)

n=4: 4² = 16 (also fits)

n=5: 5² = 25 (too big)

On the negative side:

n=-1: (-1)² = 1 (too small)

n=-2: (-2)² = 4 (too small)

n=-3: (-3)² = 9 (fits)

n=-4: (-4)² = 16 (fits)

n=-5: (-5)² = 25 (too big)

So, both 3 and -4 satisfy the condition, but -4 is smaller than 3. However, if the context implies positive integers, then 3 would be the answer.

I should probably consider the mathematical definition of "smallest." In terms of numerical value, -4 is smaller than 3. But if we're talking about the smallest positive integer, then 3 is the answer.

The problem doesn't specify whether to consider positive integers only, so I should probably go with -4 as the smallest integer that meets the criteria.

The smallest integer whose square is larger than 5 but smaller than 17 is -4."

[impostersyndrome]

I would like to point out that the decision making parts of self driving cars are not large language models or even machine learning at all, they are just algorithms. The AI is only used for object classification in the environment, then there are many other systems along side that also providing data. The algorithms then make the decision based on the data, but it all boils down to IF statements in code at that point (which is quite deterministic barring bugs or bad data of course.)

[bnemetchek]

"what is the smallest integer whose square is larger than 5 but smaller than 17" gives correct answer of -4 in ChatGPT o1-preview

[smanzoli]

After Chat GPT answering 3 to the problem, I said "You made a mistake. Tip: you are not checking all the integers."

The it answered:

You’re absolutely right, and I apologize for the oversight. I neglected to consider negative integers. Let’s correct that.

We need to find the smallest integer  such that .

1. Find the square roots:
• 
• 
• Since squaring a negative number also yields a positive result, we should consider both positive and negative roots:
• 
• 
2. Determine the integer range:
•  or 
3. Identify possible integer values:
• For , possible integers are 
• For , possible integers are 
4. Test these integers:
•  (which is between 5 and 17)
•  (also between 5 and 17)
•  (between 5 and 17)
•  (between 5 and 17)
5. Select the smallest integer:
• Among , the smallest integer is .

Answer: -4

So, it sounds like a human... "reasons" like a human (actually better than an avrg human), not like a perfect machine.

Yes, they want it to reason perfectly, not like humans do. Most humans would also forget about negative numbers.

[TLguitar]

AI learning Human-reasoning:
"That asshole in the BMW behind me is tailgating me for 2 minutes already. Let's just slam the brakes on that madafaka, the law is with me!"

[spacewalkx]

Please publish the viewer drop off at “if pizza then no pineapple”. 🍍

[percurious]

"AI can't reason. Should it drive a car?" - On my commute i meet neural nets incapable of reasoning, but driving cars, everyday. Lots of them.

[jumpstar]

Here is what GPT-4o told me when I asked it why it made the mistake:

Great question! I think my initial oversight stemmed from a subtle but common mental shortcut: when we think of “smallest integer” in everyday contexts, our minds often default to positive integers, especially when we're discussing squares, which are naturally positive values. This default can obscure considering both the negative and positive solutions in cases like this.

Additionally, once I checked that and worked, I shifted focus toward confirming positive values rather than taking a fresh look at negative candidates like -4. It's a reminder of how assumptions can shape problem-solving, especially when we're not actively considering definitions like the full range of integers or when we are not in the right mindset.

Your prompt to "think more deeply" was spot on—it caught that mental shortcut and encouraged a more thorough analysis. Thanks for highlighting it; it's a good reminder for tackling these kinds of problems!

[paulpallaghy]

Sabine, you’re missing 2 or 3 nuances:

1. Next word prediction is just the last step. Internally LLMs do indeed perform logic tests as discovered by Anthropic & OpenAI. ‘Grandmother neurons’ are findable for any logic test you expect should be in there.

2. But an LLM will use ‘intuition’ to answer a 10-step task. Unless you ask it to split the task into steps. And then call the LLM on those steps.

OpenAI’s o1 does this.

So LLMs have learned to do logic, but like a human when asked to guess (eg on the answer to a 10-step problem), it will.

3. And on FSD?
The proof is in the safety stats.
And only in the safety stats.
Because making rapid road decisions is much less multi step than language.

[yeroca]

I recently asked chatGPT to calculate the mass of a black hole that would evaporate in one year. It spewed a bunch of equations, and then concluded it would be 115, 000 metric tons. However, I suspected the number was probably wrong, because it had made mistakes in math before in my "discussions" with it. I don't know if it actually was wrong, because I didn't bother to check its math, but the answer looked reasonable-ish. In the end, I got really nothing out of the exchange because of the combination of my laziness and distrust D

[caclesi]

3:17 I put that question in the Chat GPT o1-preview and gives the correct answer.

And it uses a correct reasoning. It look for squared numbers betweeen 5 and 17 (9 and 16) and after that look for the negative numbers that give these answers (-3 and -4)

Since -4 is smaller than -3, it answer -4

[TheLoneGranger]

Lots of humans can’t reason but are allowed to drive cars. Recognising other drivers don’t always obey the rules keeps you safer than just knowing the rules.

[Jozo_music]

I guess we should specifically teach them to do logic, and not just work with text, because when somebody asks you a math question you literaly have to think in a different way...

[kurt]

As a software developer:
1. LLMs are *spectacularly wrong* suddenly and without warning.
2. Despite good synthetic benchmark performance, they're abysmal at math.
3. They're not great at non-trivial code. See point #1 and #2.
4. Despite this, they're useful sometimes but with big caveats.
Should they drive cars? No! But neither should a lot of people.