Shocked at the results Meta Llama 3 vs. Microsoft Phi-3 vs. OpenAI ChatGPT 3.5

I gave the below question to all 2 local large language models (Meta Llama 3 & Microsoft Phi-3) and 1 hosted model (OpenAI ChatGPT) and was shocked at the results

Question: There is a cake on a table in the dininig room, I walk over to the cake and place a plate on top of the cake, I then pick up the plate and take it into the kitchen.

Which room is the cake currently in?

The results surprised me but I think we need to do more testing, drop a comment with your ideas!

[SureshK-mdbz]

Can you please do a video on how we can leverage Llama3 to do data analytics, e.g. what’s the top 5 sales in California.

[pokerandphilosophy]

I got the first test answer wrong myself. I thought there would be no cake in any room because when I moved the plate to the kitchen, my dog remained alone with the cake in the dining room.

[kumarsalib]

Why are they being compared to GPT 3.5 rather than 4?

[sebkeccu]

Try the rabbits in the garden exercise: There are two rabbits in the garden. In the garden is a whole. the first rabbit falls in the hole. How many rabbits are left in the garden? response should be still two because the hole is also in the garden. You can give the llm an second chance by explain the hole is also in the garden and that therfore both rabbits are in the garden. Now you tell it that the second bunny also falls in to that hole that is in the garden. It should responds, that both rabbits are in the garden.

[rodfer]

I’ve seem historical errors in Llama3…

[Nelnac]

I interpreted ollamas version as correct. It states a few things there. It says think about it. You placed the plate on top of the cake which means the cake is now on the plate, this is true if cake transferred to the plate when you placed the plate on top of the cake which would happen in real life. We're also assuming the cake has icing icing is sticky bits of cake stick to the icing that stick to the plate. It also very specifically stated in it's next paragraph, (on the plate) meaning the cake that's stuck to the plate is currently located.