“There’s no way to get there without a breakthrough,” OpenAI CEO Sam Altman said, arguing that AI will soon need even more energy.
OH NO, AI IS THE BLOCKCHAIN.
So AI can’t exist without stealing people’s content and it can’t exist without using too much energy. Why does it exist then?
The models get more efficient and smaller very fast if you look just a year back. I bet we’ll run some small LLMs locally on our phones (I don’t really believe in the other form factors yet) sooner as we believe. I’d say prior 2030.
I can already locally host a pretty decent ai chatbot on my old M1 Macbook (llama v2 7B) which writes at the same speed I can read, its probably already possible with the top of the line phones.
Lol, “old M1 laptop” 3 to 4 years is not old, damn!
(I have running macbookpro5,3 (mid 2009) on Arch, lol)
But nice to hear that M1 (an thus theoretically even the iPad, if you are not talking about M1 pro / M1 max) can already run llamma v2 7B.
Have you tried the mistralAI already, should be a bit more powerful and a bit more efficient iirc. And it is Apache 2.0 licensed.
But nice to hear that M1 (a thus theoretically even the iPad, if you are not talking about M1 pro / M1 max) can already run llamma v2 7B.
An iPhone XR/XS can run Stable Diffusion, believe it or not.
3 to 4 years is not old
Huh, nice. I got the macbook air secondhand so I thought it was older. Thanks for the suggestion, I’ll try mistralAI next, perhaps on my phone as a test.
So AI can’t exist without stealing people’s content
Using the word “steal” in a way that implies misconduct here is “You wouldn’t download a car” level reasoning. It’s not stealing to use the work of some other artist to inform your own work. If you copy it precisely then it’s plagiarism or infringement, but if you take the style of another artist and learn to use it yourself, that’s…exactly how art has advanced over the course of human history. “Great artists steal,” said Picasso famously.
Training your model on pirated copies, that’s shady. But training your model on purchased or freely available content that’s out there for anyone else to learn from? That’s…just how learning works.
Obviously there are differences, in that generative AI is not actually doing structured “thinking” about the creation of a work. That is, of course, the job of the human writing and tweaking the prompts. But training an AI to be able to write like someone else or paint like someone else isn’t theft unless the AI is, without HEAVY manipulation, spitting out copies that infringe on the intellectual property of the original author/artist/musician.
Generative AI, in its current form, is nothing more than a tool. And you can use any tool nefariously, but that doesn’t mean the tool is inherently nefarious. You can use Microsoft Word to copy Eat, Pray, Love but Elizabeth Gilbert shouldn’t sue Microsoft, she should sue you.
Edit: fixed a typo
Because it’s a miracle technology. Both of those things are also engineering problems - ones that have been massively mitigated already. You can run models almost as good as gpt3.5 on a phone, and individuals are pushing the limits on how efficiently we can train every week
It’s not just making a chatbot or a new tool for art - it’s also protein folding, coming up with unexpected materials, and being another pair of eyes that will assist a person do anything.
They literally promise the fountain of youth, autonomous robots, better materials, better batteries, better everything. It’s a path for our species to break our limits, and become more.
The downside is we don’t know how to handle it. We’re making a mess of it, but it’s not like we could stop… The AI alignment problem is dwarfed by the corporation alignment problem
🙄 iTS nOt stEAliNg, iTS coPYiNg
By your definition everything is stealing content. Nearly everything in human history is derivative of others work.
Because the shareholders need more growth. They might create Ultron along the way, but think of the profits, man!
There’s no way these chatbots are capable of evolving into Ultron. That’s like saying a toaster is capable of nuclear fusion.
deleted by creator
Thats if you set the toaster to anything above 3
It’s the further research being done on top of the breakthrough tech enabling the chat bots applications people are worried about. It’s basically big tech’s mission now to build Ultron, and they aren’t slowing down.
What research? These bots aren’t that complicated beyond an optimisation algorithm. Regardless of the tasks you give it, it can’t evolve beyond what it is.
I think we’ve got a bit before we have to worry about another major jump in AI and way longer for an Ultron. The ones we have now are effectively parsers for google or other existing data. I personally still don’t see how we feel like we can get away with calling that AI.
Any AI that actually creates something ‘new’ that I’ve seen still requires a tremendous amount of oversight, tweaking and guidance to produce useful results. To me, they still feel like very fancy search engines.
This is the best summary I could come up with:
The process is ludicrously energy intensive, with experts estimating that the industry could soon suck up as much electricity as an entire country.
Unperturbed, billionaires including Jeff Bezos, Peter Thiel and Bill Gates have poured substantial amounts of money into the idea.
However, while the emergent crop of startups like Helion has repeatedly claimed that fusion energy is right around the corner, we have yet to see any concrete results.
Of course, if Altman’s rosy vision of the future of energy production were to turn into a reality, we’d have a considerably greener way to power these AI models.
According to an October paper published in the journal Joule, adding generative AI to Google Search alone balloons its energy uses by more than tenfold.
“Let’s not make a new model to improve only its accuracy and speed,” University of Florence assistant professor Roberto Verdecchia told the New York Times.
The original article contains 525 words, the summary contains 149 words. Saved 72%. I’m a bot and I’m open source!
The human brain uses about 20W. Maybe AI needs to be more efficient instead?
That would require a revolutionary discovery in material science and hardware.
And yet we have brains. This brute force approach to machine learning is quite effective but has problems scaling. So, new energy sources or new thinking?
We just run the AI for a gazillion epochs and then it’s
overfittedevolved intelligence. Thanks Darwin we did it again.
It’d be way easier to just grow brains instead
We invented computers to do things human brains either couldn’t do, or couldn’t do fast enough.
Perfect let’s use human brains as CPUs then. Not the whole brain just the unused bits.
We use all of our brain. Well, some of us try to anyway.
I’ve seen that film
Flubber!
It’s what matrix would’ve been if the studios didn’t think people would too dumb to get it, so we ended with the nonsense about batteries.
They also thought we wouldn’t understand how Switch could be a woman in the matrix but a man in the real world. So they just made the character a butch woman because apparently that’s easier somehow. So many little changes like this were made.
Holy fuck now her name makes so much more sense. God dammit, why are we so fucking stuck up as a society that we couldn’t even keep that
Tbf that society was a while ago now.
I don’t think it’s gotten better, and honestly they oversimplify even more today. For some reason
I would love it (if there exists a FOSS variant of that) imagine being able to run a LLM, or even LAM in your head,
wait…
🤔
FOSS Neuralink
Is the answer people? I think I’ve seen this movie before.
This is probably why he’s invested so much into Helion Energy, who are trying to make a small and cheap nuclear fusion reactor.
make a small and cheap nuclear fusion reactor
Aneutronic fusion isn’t happening on this planet. We don’t even have the fuel for it. It’s a dumb thing to market when we can’t even break even on D-T fusion and turning the neutrons into heat.
It blows my mind honestly. This is such a young technology that commercialization at this point seems ridiculous
While I’m too much of an optimist to say that we’ll never figure out viable fusion power, I do think you’re more right than wrong.
Fission power is essentially us discharging a fusion battery, where the battery was charged by a supernova. We don’t get any free help with fusion, and we have to replicate input energies only seen in nature with stellar amounts of gravitational mass. It is (IMO) an important area of research, but I don’t expect it to power our cities in my lifetime.
Yeah, but what they’re marketing specifically is aneutronic fusion. That’s helium fusion, which has never been demonstrated outside of a star. Hydrogen fusion, which we haven’t actually achieved much with beyond bombs is more managable. The difference is hydrogen fusion creates a big neutron flux, which needs to be isolated (the small part) and creates waste by neutron activating whatever it’s around (the cheap part, volume wise hydrogen fusion creates more radioactive waste than fission but it’s much easier to manage low level waste).
It doesn’t help that the helium is a primordial resource that has literally escaped the crust of our planet and floated out into space. Supposedly the moon has more.
Optimizing power consumption? Why?!
Some of the smartest people on the planet are working to make this profitable. It’s fucking hard.
You are dense and haven’t taking even a look at simple shit like hugging face. Power consumption is about the biggest topic you find with anyone in the know.
Some of the smartest people on the planet are working to make this profitable. It’s fucking hard.
[Take a look at] hugging face. Power consumption is about the biggest topic you find with anyone in the know.
^ fair comment
In fairness the computing world has seen unfathomable efficiency gains that are being pushed further with the sudden adoption of arm. We are doing our damnedest to make computers faster and more efficient, and we’re doing a really good job of it, but energy production hasn’t seen nearly those gains in the same amount of time. With the sudden widespread adoption of AI, a very power hungry tool (because it’s basically emulating a brain in a computer), it has caused a sudden spike in energy needed for computers that are already getting more efficient as fast as we can. Meanwhile energy production isn’t keeping up at the same rate of innovation.
It’s not so much the hardware as it is the software and utilisation, and by software I don’t necessarily mean any specific algorithm, because I know they give much thought to optimisation strategies when it comes to implementation and design of machine learning architectures. What I mean by software is the full stack considered as a whole, and by utilisation I mean the way services advertise and make use of ill-suited architectures.
The full stack consists of general purpose computing devices with an unreasonable number of layers of abstraction between the hardware and the languages used in implementations of machine learning. A lot of this stuff is written in Python! While algorithmic complexity is naturally a major factor, how it is compiled and executed matters a lot, too.
Once AI implementations stabilise, the theoretically most energy efficient way to run it would be on custom hardware made to only run that code, and that code would be written in the lowest possible level of abstraction. The closer we get to the metal (or the closer the metal gets to our program), the more efficient we can make it go. I don’t think we take bespoke hardware seriously enough; we’re stuck in this mindset of everything being general-purpose.
As for utilisation: LLMs are not fit or even capable of dealing with logical problems or anything involving reasoning based on knowledge; they can’t even reliably regurgitate knowledge. Yet, as far as I can tell, this constitutes a significant portion of its current use.
If the usage of LLMs was reserved for solving linguistic problems, then we wouldn’t be wasting so much energy generating text and expecting it to contain wisdom. A language model should serve as a surface layer – an interface – on top of bespoke tools, including other domain-specific types of models. I know we’re seeing this idea being iterated on, but I don’t see this being pushed nearly enough.[1]
When it comes to image generation models, I think it’s wrong to focus on generating derivative art/remixes of existing works instead of on tools to help artists express themselves. All these image generation sites we have now consume so much power just so that artistically wanting people can generate 20 versions (give or take an order of magnitude) of the same generic thing. I would like to see AI technology made specifically for integration into professional workflows and tools, enabling creative people to enhance and iterate on their work through specific instructions.[2] The AI we have now are made for people who can’t tell (or don’t care about) the difference between remixing and creating and just want to tell the computer to make something nice so they can use it to sell their products.
The end result in all these cases is that fewer people can live off of being creative and/or knowledgeable while energy consumption spikes as computers generate shitty substitutes. After all, capitalism is all about efficient allocation of resources. Just so happens that quality (of life; art; anything) is inefficient and exploiting the planet is cheap.
For example, why does OpenAI gate external tool integration behind a payment plan while offering simple text generation for free? That just encourages people to rely on text generation for all kinds of tasks it’s not suitable for. Other examples include companies offering AI “assistants” or even AI “teachers”(!), all of which are incapable of even remembering the topic being discussed 2 minutes into a conversation. ↩︎
I get incredibly frustrated when I try to use image generation tools because I go into it with a vision, but since the models are incapable of creating anything new based on actual concepts I only ever end up with something incredibly artistically compromised and derivative. I can generate hundreds of images based on various contortions of the same prompt, reference image, masking, etc and still not get what I want. THAT is inefficient use of resources, and it’s all because the tools are just not made to help me do art. ↩︎
It’s emulating a ridiculously simplified brain. Real brains have orders of magnitude more neurons, but beyond that they already have completely asynchronous evaluation of those neurons, as well as much more complicated connecting structure, as well as multiple methods of communicating with other neurons, some of which are incredibly subtle and hard to detect.
To really take AI to the next level I think you’d need a completely bespoke processor that can replicate those attributes in hardware, but it would be a very expensive gamble because you’d have no idea if it would work until you built it.
The problem there is the paradox of efficiency, making something more efficient ends up using more of it not less as the increase in use stimulated by the greater efficiency outweighs the reduced input used.
Unity developers be like.
This dude al is the new florida man, wonder if it’s the same al from married with children
dude think about this stuff before you open the floodgates bro
That requires someone in business to think beyond the next quarter’s profits.
That requires someone in business to think
I’m not convinced that Altman has cleared this beyond meaningless buzzwords
Man, if only there was some sort of energy source that is not only green and renewable, but also outputs a ton of energy rather efficiently…
Nuclear isn’t renewable, it’s just green. We don’t have supernovas on earth making new uranium or thorium.
yes, but extremely toxic and radioactive waste tho.
thorium could be an option
fusion could be an option
or… ya know, we just continue expanding solar and wind energy until we have one of the above.
How about we build the reactors that can consume the extremely toxic waste so that we can get rid of it?
i think we would’ve done that.
We can. You can blame the “Greens” for it not happening.
deleted by creator
for sure, but nuclear fission like we are doing it right now is definetely not sustainable. we can do it for like a hundred more years, then the waste really starts piling up.
Yeah, no reason to ever think of alternate options. Just push one or two things until they magically work for everything somehow.
Toxic/Radioactive waste is obviously toxic and radioactive, but how bad that really is is kind of overblown especially if you compare it to the harm caused by popular existing methods like coal/etc. When adjusted based on energy produced, there’s more than one study out there showing how Nuclear is significantly safer than coal by a very wide margin. Coal ash is also radioactive and coal plants have very limited requirements to prevent it from escaping to the environment.
Even ‘Radioactive Waste’ really only feels scary because all of the bad stuff is condensed into a much smaller package when you adjust based on energy produced again.
yes, absolutely. we germans should’ve shut down coal plants first. but there’s no point in reactivating the nuclear power plants, especially because we are literally producing more power than we can use. 60% of which is renewables. some providers will give you energy for basically free, as long as you use it at night when everything shuts down.
Thorium fuel cycle is nearly the same as the uranium fuel cycle in regards to downsides. It just requires breeders, which you could use with uranium too. The only real benefit of thorium is that it’s more plentiful, but the cost in nuclear power isn’t in Uranium.
Oh man, I clicked that link really hoping you were going to link to a Dyson Sphere.
The most efficient use of nuclear energy.
Elon will probably start taking preorders for Dyson spheres soon
“guys it’ll totally be a ready by 2030 i promise”
So will AI (or more accurately LLM) be the push needed to make limitless fusion energy a reality?
Obviously not. We’re being faced with an existential threat if we don’t secure alternative, sustainable forms of energy and even that threat isn’t enough to motivate our species.
Threat to existence is nothing compared to ability to hoard more money
The positive thing there is that it probably paces our development. If we can’t get to true AGI without way more energy than we can currently produce, then we don’t have true AGI risk right now.
There’s still risk because it might not be true or we might be able to get close enough to do damage. But slowing down AI is fine by me.
Subsidize retrofitting the entire nation with solar and invert it back into the grid
The sun gives us free energy. Is he aware of that?
Not efficiently and not as reliably as a nuclear reactor though. It would if they built a space station in an orbit with minimal other objects getting in the way of it and the sun. Teach the ai in between Sol and Venus and bring it back if it discovers anything useful rather than making revenge porn and plagiarizing artists
The sun is a fusion reactor.
That is way too hot and uncontrollable for our current tech to tap into. Solar panels are the best we can do for now until we figure out Dyson spheres
1 GW of solar is much cheaper than 1 GW of nuclear. Solar is both cheaper to build and cheaper to run. It’s the most efficient energy source e currently have.
It also turns on and off outside of any human control.
Nuclear produces that 1 GW 24/7 and all year though. My solar panels vastly overproduce on most summer days and are worthless at nights and from fall to spring.
If we’re still talking about AI, you can ramp up the AI training and batch workloads when the sun is shining and stop them overnight. It’s one of those things like aluminum smelters where you can adjust the load
They stop reactors all the time. It’ll probably be cheaper to get a massive battery pack+solar
Plus it doesn’t have the con of meltdown and nuclear fallout. Safer nuclear power is still potentially fucking dangerous for life for 20000 years. Nuclear fission energy is the height of human hubris.
I love when people invent something then complain about how dangerous it is. It really hits you in the feels.
In the end, as always, it will only benefit the companies. And all the people get is put out of a job because they have been replaced by some piece of software no one even understands anymore.
This is a silly take, people have benefitted hugely from all the big tech developments in the past and will do from ai also - just as you have a mobile phone that can save and improve your life in a myriad of ways so you’ll have access to various forms of ai which will do similar. GPS is a good example, functionally free and making navigation far safer, faster, and better.
Here’s a genuine already happened use case for ai benefitting you, an open source developer was able to add a whole load of useful features to their free software by using AI to help code - I know because it was me, among many many others.
I know people making open source ai tools too and they’re all using AI coding assistants - mostly the free ones. I’ve seen a lot of academic researchers using AI tools also generally built using open source tools like pytorch and with help from ai coding tools. Even if you don’t use ai yourself you’re already benefitting from it, even if you don’t use open source software the services you rely on do.
Imagine being able to implement the most advanced and newest methodologies in your design process or get answers to complex and niche questions about new technology instantly. You buy a printer for example and say to your computer ‘I’ve plugged in a printer make it work’ and it says ‘ok, there isn’t a driver available that’ll work with your pc but I’ve written one based on the spec in the datasheet, do you want me to print a test page?’
Imagine being able to say ‘talk me through diagnosing a fault on my washing machine’ and it guides you through locating and fixing the fault, possibly by designing a replacement part and giving you fabrication options.
Or being able to say ‘this website is annoying, change it so that I only see the video window’ or ‘make a playlist in release order of all abba songs that charted’ or ‘check on currently available archives to see if there’s a mirror of this deleted post’ or ‘check all the sites and see if anyone posted a sub version of the next episode of this anime’ or ‘Keep an eye on this lemmy community and add any popular memes involving fish to my feed but don’t bother with any meta stuff or aquatic mammals’ or ‘this advert says I can make free money, is it ligit?’
The use cases that will directly benefit your life are almost endless, natural language computing is a huge deal even without task based solvers and physical automation but we also have those too so the increased ability of people to make community projects and freely shared designs is huge.
wasn’t the same thing said about ATM’s? and then it created the need for banks to hire more employees?
iirc, technology/robots has only been able to create more jobs, right? or am I misinformed?
The difference is the type of the job. Do we want to make jobs available for the general population and requiring minimal training, or do we want to make jobs available only for those with very difficult-to-get engineering degrees?
