https://twipped.social/@twipped/114662771295312758
article they are referencing: https://futurism.com/atari-beats-chatgpt-chess
Hundreds of billions of dollars spent
No profitable product
No consistently usable product other than beginner code tasks
Massive environmental harms
Tens of thousands of (useful!) careers terminated
Destroyed Internet search, arguably the one necessary service on the Internet
No chance it’s going to get better
Atari 2600 beating it at chess is a perfect metaphor. People who want to complain about it can bite its plastic woodgrain printed ass.
Massive environmental harms
I find this questionable; people forget that a locally-hosted LLM is no more taxing than a video game.
No chance it’s going to get better
Why do you believe this? It has continued to get dramatically better over the past 5 years. Look at where GPT2 was in 2019.
No consistently usable product other than beginner code tasks
It is not consistently usable for coding. If you are hoping this slop-producing machine is consistently useful for anything then you are sorely mistaken. These things are most suitable for applications where unreliability is acceptable.
No profitable product […] Tens of thousands of (useful!) careers terminated
Do you not see the obvious contradiction here? If you are sure that this is not going to get better and it’s not profitable, then you have nothing to worry about in the long-term about careers being replaced by AIs.
Destroyed Internet search, arguably the one necessary service on the Internet
Google did this intentionally as part of enshittification.
No consistently usable product other than beginner code tasks
I mean, it’s pretty good as a productivity tool for programmers as it eliminates a bunch of chore.
Oh my god you ‘people’. Did you not read what you replied to?
‘people’ in scare quotes since coders aren’t people I guess.
Fine, we’ll stipulate to that. The conclusion is upheld.
it introduces more problems than it solves.
Such as?
hallucinations, bugs, security flaws,
I swear you have never actually used copilot while programming and are just making shit up
i swear you have never actually programmed anything
Why? Because I can use a smarter autocomplete and I am open to change and not a dogmatic closeminded person?
I bet the llm doesn’t even know what en passant is
Holy hell
It knows the definition though!
It literally doesn’t. It has a pointer to other pointers that often times are marked as correct.
“Knows”
I mean, you literally have whole videos on YouTube made by GothamChess who shows how LLMs play chess. They literally spawn pieces from air, play moves that are illegal etc.
Holy hell
New response just dropped
I don’t know if this is real, but AI for chess kinda has to be tailor made for chess, right?
Yes the point is that LLMs don’t reason.
These articles aren’t written for people who know how LLMs work or what they do, anyway.
It’s to prove to everyday people that the techbro marketing is bullshit and these are limited tools, not conscious beings. The populace is being sold a hammer that hallucinates and told everything is a nail.
I fuck around with AI chatbots every now and then. Sometimes I’ll type in one thing to it and it will spit out a respone that doesn’t have anything to do with what I said to it
maybe it’s the way I type and form sentences, but yeah, a lot of ai “generated” stuff won’t be what you typed in
That or AGI. But there’s no AGI yet.
as far as the general public knows at least
LLMs can’t beat anyone or anything at chess because they can’t play chess at all. Try it. They don’t get more than a few moves in without degrading into total nonsense.
By my best feelings, this shit is a bigger bust than the .com bubble, and I predate that latter shit by roughly twenty years.
I’m quite sure that the guy understood pretty well what LLMs can do. He just wanted to deinflate all the bullshit promises by Techbros
Deinflate? Is that like uninflating? Or more like making something inflateless?
Tap for spoiler
It’s just deflate, and yes I feel like a dickhead for pointing it out.
“Deinflate” feels like actively sucking all the air out instead of letting it out passively. Unrelated, I know, but I think words are so neat
Man that sucks
LLM sucks at maths, sucks at chess, sucks at remembering stuff and being consistent … They suck at everything a computer is usually good at.
Yes, LLMs are designed to emulate how a human would respond to a prompt by digesting a huge amount of human-generated content. They can do that fairly well except when they can’t.
It’s a very specialized program intended to get a computer to do something that computers are generally very, very bad at - write sensible language about a wide variety of topics. Trying to then get that one specialized program to turn around and do things that computers are good at, and expect to do it well, is very silly.
It also demonstrates how much AI companies mislead the public on what their products can do. If a guy is selling lawnmowers that actually just generate grass clippings without mowing the lawn, you’re not an idiot for thinking it was going to mow grass.
furthermore. companies mislead journalists, investors, philosphers, influencers etc. most of which dont have a technical background but a lot of reach. They then carry their misunderstanding into the general public.
All these public “academic” panel debates on conferences about AGI being the next nuclear weapon and singularity. They lead to Highbrow publications, opinion peaces, books and blog articles, which then lead to tweets, memes and pop cultural references
But once someone explains it to you and you insist the grass was mowed, they show you the unmowed grass, and you still insist it’s great for mowing lawns.
And also you’re in the desert where you shouldn’t even have a fucking lawn, and you plant more lawns because they’re so easy to mow now
What do you call that? Because it’s a bit past ‘idiot’.
Huh… wait… what if we make a box… generate electricity bills… Call it a crypto miner?
Cram a bunch of space heaters into a box. Convince investors that all the electricity it burns up means it’s basically printing money. The building will inevitably burn down before anyone can investigate our claims.
Chess engines don’t have real difficulties. Every level of the chess engine is designed to make more blunders as the elo gets smaller.
In other works it is programmed to make bad moves in regular intervals. What that means is even on beginner modes when the engine isn’t blundering it is playing perfect chess. This is why it isn’t good to play against chess bots. At best you will learn some pattern recognition but chess puzzles are better at that.
In CoD MW 2 (or maybe Black Ops) the multiplayer AI bots were like this. Obviously all bots are but the kill cams were illuminating. And they didn’t even try to make it look human. They’d even use a light machine gun. They’d walk around. Once they see you they’d turn towards you. The only thing the difficulty changed was how fast they turned. Then they’d shoot a single shot at your head. For things like a sniper rifle it looked mostly believable, but that’s not how people use machine guns lol. The single shot with the most inaccurate weapon is just dirty lmao.
I’m quite sure that you could use a LLM to play chess and probably even successful, but you need to train it on chess notation of games instead of a pile of fanfiction and other copyright infringements. I have considered trying that but was turned off by how inaccessible LLM training is and how difficult it would be to get a sufficient amount of games written in proper chess notation. Obviously this would not be a real LLM, as it does not “speak”, but I was curious how well this would work utilizing the same technique.
You shouldn’t train an LLM for that, just any other type of machine learning.
You don’t need text to play chess.
deleted by creator
Make the computer play 100,000,000 games against itself. Human games are useless as training data because humans are useless at chess compared to computers.
It’s called AlphaZero and is the best chess engine to date
Thank you, I couldn’t think of the name, but I knew there was a machine learning chess bot out there that made cheating at online chess really common.
I had a dedicated electronic chess game - a board with LEDs on it showing where the game wanted to move. You had to move physical pieces around and press membrane switches under the squares to tell it where you moved. I don’t remember if it was described as “AI” back then or not. I thought of it as a chess expert system on a chip. As a total novice player I could rarely beat it on its lowest skill level. Was never interested enough in chess to get the game for my 2600. But I still have both of those things in a box.
They could probably have done better by training a crow to play chess.
Nobody thought it would do very well. This was a software dev’s little diversion.
We should praise attempts to make the public aware of the limitations of LLMs, not laugh at the guy who did this.
Okay, i think there is quite a misunderstanding here.
Some older versions of LLMs (chatgpt3.5-turbo-instruct) can play chess relatively well (around 1750 Elo) : here is a link to an article studying that.
Some points :
- it is of course way worse than almost any algorithm designed for chess
- one of the reason we cannot get these result back (at least not that good, here is a link to a blog post of someone making recent LLMs chatbots better at chess) could be that we do not have access to pure completion mode on models trained on selected data (where they could purposefully choose only good chess matches), and those are now hidden behind a chatbot layer instead.
- it seems to reveal that models have a somehow accurate representation of the chess board when predicting chess moves
- it seems to have a quite unique feat that is : if you feed them a prompt that say they play as a very good player, and then the beginning of a game with a blatant bad move (giving away a queen for example), they sometimes play the entire game with moves that purposefully give away pieces, as if they guess that the only reason they would lose a piece that easily is by purposefully losing them. It has close to zero utility, but it’s interesting anyway.
