Ah, you used logic. That’s the issue. They don’t do that.
Using an LLM as a chess engine is like using a power tool as a table leg. Pretty funny honestly, but it’s obviously not going to be good at it, at least not without scaffolding.
is like using a power tool as a table leg.
Then again, our corporate lords and masters are trying to replace all manner of skilled workers with those same LLM “AI” tools.
And clearly that will backfire on them and they’ll eventually scramble to find people with the needed skills, but in the meantime tons of people will have lost their source of income.
If you believe LLMs are not good at anything then there should be relatively little to worry about in the long-term, but I am more concerned.
It’s not obvious to me that it will backfire for them, because I believe LLMs are good at some things (that is, when they are used correctly, for the correct tasks). Currently they’re being applied to far more use cases than they are likely to be good at – either because they’re overhyped or our corporate lords and masters are just experimenting to find out what they’re good at and what not. Some of these cases will be like chess, but others will be like code*.
(* not saying LLMs are good at code in general, but for some coding applications I believe they are vastly more efficient than humans, even if a human expert can currently write higher-quality less-buggy code.)
I believe LLMs are good at some things
The problem is that they’re being used for all the things, including a large number of tasks that thwy are not well suited to.
yeah, we agree on this point. In the short term it’s a disaster. In the long-term, assuming AI’s capabilities don’t continue to improve at the rate they have been, our corporate overlords will only replace people for whom it’s actually worth it to them to replace with AI.
Can i fistfight ChatGPT next? I bet I could kick its ass, too :p
The Atari chess program can play chess better than the Boeing 747 too. And better than the North Pole. Amazing!
Are either of those marketed as powerful AI?
Neither of those things are marketed as being artificially intelligent.
Marketers aren’t intelligent either, so I see no reason to listen to them.
You’re not going to slimeball investors out of three hundred billion dollars with that attitude, mister.
There was a chess game for the Atari 2600? :O
I wanna see them W I D E pieces.
Here you go (online emulator): https://www.retrogames.cz/play_716-Atari2600.php
WTF? I just played that just long enough for my queen to take over their queen, and it turned my queen into a rook?
Is that even a legit rule in any variation of chess rules?
I wasn’t aware of that either, now I’m kinda curious to try to find it in my 512 Atari 2600 ROMs archive…
This made my day
Get your booty on the floor tonight.
I’m often impressed at how good chatGPT is at generating text, but I’ll admit it’s hilariously terrible at chess. It loves to manifest pieces out of thin air, or make absurd illegal moves, like jumping its king halfway across the board and claiming checkmate
It can be bad at the very thing it’s designed to do. It can repeat phrases often, something that isn’t great for writing. But why wouldn’t it, it’s all about probability so common things said will pop up more unless you adjust the variables that determine the randomness.
ChatGPT is playing Anarchy Chess
Yeah! I’ve loved watching Gothem Chess’ videos on these. Always have been good for a laugh.
Next, pit ChatGPT against 1K ZX Chess in a ZX81.
LLM are not built for logic.
And yet everybody is selling to write code.
The last time I checked, coding was requiring logic.
To be fair, a decent chunk of coding is stupid boilerplate/minutia that varies environment to environment, language to language, library to library.
So LLM can do some code completion, filling out a bunch of boilerplate that is blatantly obvious, generating the redundant text mandated by certain patterns, and keeping straight details between languages like “does this language want join as a method on a list with a string argument, or vice versa?”
Problem is this can be sometimes more annoying than it’s worth, as miscompletions are annoying.
Fair point.
I liked the “upgraded autocompletion”, you know, an completion based on the context, just before the time that they pushed it too much with 20 lines of non sense…
Now I am thinking of a way of doing the thing, then I receive a 20 lines suggestion.
So I am checking if that make sense, losing my momentum, only to realize the suggestion us calling shit that don’t exist…
Screw that.
The amount of garbage it spits out in autocomplete is distracting. If it’s constantly making me 5-10% less productive the many times it’s wrong, it should save me a lot of time when it is right, and generally, I haven’t found it able to do that.
Yesterday I tried to prompt it to change around 20 call sites for a function where I had changed the signature. Easy, boring and repetitive, something that a junior could easily do. And all the models were absolutely clueless about it (using copilot)
I suppose it’s an interesting experiment, but it’s not that surprising that a word prediction machine can’t play chess.
Because people want to feel superior because they
don’t know how to use a ChatBotcan count the number of "r"s in the word “strawberry”, lolYeah, just because I can’t count the number of r’s in the word strawberry doesn’t mean I shouldn’t be put in charge of the US nuclear arsenal!
That is more a failure of the person who made that decision than a failing of ChatBots, lol
Agreed, which is why it’s important to have articles out in the wild that show the shortcomings of AI. If all people read is all the positive crap coming out of companies like OpenAI then they will make stupid decisions.
Anyone who puts a chatbot anywhere is definitely a failure, yeah.
A strange game. How about a nice game of Global Thermonuclear War?
No thank you. The only winning move is not to play
Lmao! 🤣 that made me spit!!
Frak off, toaster
I’ve heard the only way to win is to lock down your shelter and strike first.
JOSHUA
They used ChatGPT 4o, instead of using o1 or o3.
Obviously it was going to fail.
Other studies (not all chess based or against this old chess AI) show similar lackluster results when using reasoning models.
Edit: When comparing reasoning models to existing algorithmic solutions.
this is because an LLM is not made for playing chess
2025 Mazda MX-5 Miata ‘got absolutely wrecked’ by Inflatable Boat in beginner’s boat racing match — Mazda’s newest model bamboozled by 1930s technology.
If you don’t play chess, the Atari is probably going to beat you as well.
LLMs are only good at things to the extent that they have been well-trained in the relevant areas. Not just learning to predict text string sequences, but reinforcement learning after that, where a human or some other agent says “this answer is better than that one” enough times in enough of the right contexts. It mimics the way humans learn, which is through repeated and diverse exposure.
If they set up a system to train it against some chess program, or (much simpler) simply gave it a tool call, it would do much better. Tool calling already exists and would be by far the easiest way.
It could also be instructed to write a chess solver program and then run it, at which point it would be on par with the Atari, but it wouldn’t compete well with a serious chess solver.
