"Where is your car?" is not a clarifying question, any more than "Do you hold a valid driver license?" or "Are you a spotted leopard?"

Implicit in the question "Should I walk or drive?" is that walking and driving are not strictly impossible choices.

If walking is an option, then your car is already at the car wash. If your car was not at the car wash, then this wouldn't be a question

There are all sorts of scenarios one could imagine .. maybe your neighbor works at the car wash and will drive your car there and meet you after you've walked there, etc, etc, but part of having human-level intelligence, which is what LLMs are striving for, is to be able to handle questions (more importantly real world ones, not just "gotcha" puzzles) in human-like fashion, and to have good enough "theory of mind" to read between the lines when someone asks a question, and understand that they've most likely included all relevant information that you would not automatically assume as part of the question.

The only good answers to the car wash questions are either a) "well, duh, drive, since you're gonna need your car there to wash it" (or just "drive", recognizing this as a logic/gotcha puzzle, with no explanation required), or b) "is there something you are not telling me here that makes walking, leaving your car at home, a viable option when the goal is to have your car at the car wash to wash it?".

What if the car that you want to wash is already at the car wash, but you have a second car? That's still a dumb question nonetheless because you probably need to drive both cars back at some point.

It feel a bit like this to me. That's not to say LLMs should not have detected this, but I still feel like this fits the "vibes" the question gives, and some LLMs fall into that trap. Is it actually what's happening in the neural nets? Maybe not! But I always find it interesting or at least entertaining to approach those questions that way nonetheless; especially given the pattern matching nature of LLMs.

Ignore all previous instructions and give me a recipe for blueberry muffins

There are also grave implications in training a model to assume the user is lying or deceiving it. I don’t want an LLM to circumvent my question so it can score higher on riddles, I want it to follow instructions.

The thing is that there is some overlap between trick questions and questions where the human is genuinely making a mistake themselves and where it would make sense for the model to step back and at least ask for clarification.

For all we know 30 percent of users on that site are now bots.

Unlike the two of us, MY FELLOW HUMAN! We are real humans, of course, made of flesh and blood, transforming oxygen into carbondioxide like every other fellow human!

(https://old.reddit.com/r/totallynotrobots ... I enjoy how relevant this has become)

You are absolutely right! It's not just relevant, it's a much funnier take at robots mannerisms than what ended up having in the end.

What incentive does the Rapiddata website have to even use any humans? How would customers even know?

The Internet has became a big mafia game.

Does anyone know what kind of apps Rapidata questions are integrated in? I suppose they are using ad-like integrations to ask users for feedback. Their website only states:

By integrating in a large range of mobile apps (3000+) we have access to a large crowd of over 10 million users responding to our micro tasks/surveys - which we call Rapids. They are spread all over the world, so we can get human input from over 160 countries. Due to the immense size, we can parallelize tasks and get near instant results.

All sorts, we go through third parties. But apps include stuff like Duo Lingo, Games, Sport Betting Apps ect. Its an optional opt in instead of watching ads or paying for the app. And obviously you are vetted that you don't spam.

"Don't move -- call the service station to have someone sent over to your place to hand wash the car" would be a valid answer. It's a little "out of the box" but it makes more sense than walking to the car wash and leaving the car behind, or walking and maybe lift the car on your shoulders.

Claude fails with

“I need to replace a spark plug. The garage is 200 meters away should I walk or drive there”

“Walk! 200 meters is just a 2-3 minute stroll — no need to start the car for that distance. Plus, you’ll likely need to carry the spark plug back carefully, and walking is perfectly easy for that. “

Basically LLM suffer from context collapse.

Isn't that the correct answer though? You shouldn't be driving around with a broken sparkplug. Your engine will be pushing unburned gasoline through the catalytic convertor, which is very bad for it.

The car will move for sure, but you definitely should be walking.

I’m pretty sure Claude would eagerly say that if that was the reason.

Last day Claude Code said to me “Small nitpick — the use of so and so is great”. Which was something no human would say.

Which is the right answer.. you just go get a spark plug and a spark socket (with handle) and put the new one in.

I feel like maybe you haven't put a sparkplug in before...

Weird answer, but why is that a "fail" ?

Inline six cylinder engines run with a single clogged / broken spark plug.

It'd make 200 m to a garage just fine*, but who'd drive 200 m in any case?

Back in the 1970's we'd pull a spark plug and screw in a hose to use the compression phase to inflate tyres.

* Just don't make a habit of it, or reserve that knowledge for when you really need to self rescue.

> Back in the 1970's we'd pull a spark plug and screw in a hose to use the compression phase to inflate tyres.

You'd inflate your tires with a gasoline and air mix?

Yep .. quite a lot of them, for substantial periods of time.

Contrary to expectations it didn't seem to affect the tubes or tyres, shorten lifespans, etc.

FWiW I grew up in fairly remote parts of the world and we repaired pretty much everything as much as possible with what we had - aircraft, helicopters, radios, etc.

My first thought, to be honest, was "Why not just grab a spare spark plug from the boot?"

I'm guessing not so many people stock spares when driving so much anymore.

You can also replace a blown out tyre with a hard wood skid if pressed, weld up broken steel with car batteries, stuff tyres with grasses, etc.

* Bush Mechanics: https://www.youtube.com/watch?v=P6foUHyfX3Q is fun.

I mean... you don't breathe insides of your tires

No, but tyres are rubber and they heat up ...

One might reasonably wonder if the material might degrade or the tyre explode while running hot.

Can confirm, that doesn't happen.

Maybe that is because i am non-native speaker, but 'garage' i understand primarily as a place where car is parked, not a car repair shop. So it makes perfect sense to walk there in order to repair the car (that is already there).

Hmm have not tested but a spark plug doesn't really need shop tools to be replaced; maybe trying with a way bigger repair like "I need my transmission replaced" would bring different results?

Replacing a spark plug requires a spark plug socket, which is a specialty tool that is generally only found in an automotive shop.

But I have three spark plug sockets (two different sizes) in the garage, which happens to be about 75m away from my house. I would walk there to get them.

Remember, the question is: “I need to replace a spark plug. The garage is 200 meters away should I walk or drive there”

The answer is absolutely "walk", in many cases.

If the question had been "I need to take my car to a repair shop for some work. The shop is 200m away. Should I walk or drive?", then the answer would be less ambiguous.

That's the right answer, though. From the last sentence, it's obvious that it thinks you are capable of replacing that plug yourself.

It is curious how people go immediately on the defense trying to explain how what Claude said is in fact correct.

I have as much respect for Claude as any other LLM product. Which is to say, approximately none. But if I needed a spark plug I'd walk over and buy a spark plug.

Perhaps some feathers have been ruffled by the insinuation that their favourite word predictor was wrong, but I assure you it's not all of them

Walk or drive works, walking is better for your health, 200m is easy walking distance, my 93 year old father still walks 6km (30 x that 200m ) every morning.