Why do people host LLMs at home when processing the same amount of data from the internet to train their LLM will never be even a little bit as efficient as sending a paid prompt to some high quality official model?
inb4 privacy concerns or a proof of concept
this is out of discussion, I want someone to prove his LLM can be as insightful and accurate as paid one. I don’t care about anything else than quality of generated answers
I want someone to prove his LLM can be as insightful and accurate as paid one.
Why would someone do that for you when they’re happy using their local LLM?
“Why do people do X, when in my opinion if you disregard the two top reasons for doing X, it’s pointless? Prove to me that it would be better!?”
I’m not saying in advance that it’s pointless, it’s just your far-fetched preconception. I want to know if there are any further arguments for hosting LLMs besides the two I ruled out in advance for being too obvious. Furthermore, I think that since you moved so quickly to mockery, there are no other reasons than privacy and tinkering
Why would you need a reason other than privacy? Or for that matter any reason you want or nobe at all.
To be fair, he is just asking for more reasons than privacy. No reason not to give him them, and also okay to say we don’t think there is any.
His assumption seems to be that there must be other reasons or that that reason is somehow not valid enough though.
No, they said they “ruled out” privacy for “obvious reasons”.
Obviously mockable statement.
The self hosted community is probably the worst place you could ask about something while ruling out “privacy and tinkering”. Those words might as well be the motto here.
Afaik most people who seriously want to have a local LLM start of off a pre-trained one from the internet…?
Because they want to. You can get your proofs yourself.
You can control the electricity source (renewable) at home and cool your system without wasting drinking water.
Poor internet connection/no internet at all, network latency too high for their needs, specific fine tuned LLM ?
Off course, main reason is privacy. My company host its own GPT4 chatbot, and forbid us to use public ones. But I suppose there are other legit use case to host its own LLM.
want someone to prove his LLM can be as insightful and accurate as paid one.
The full DeepSeek model is available for download, and should generate about the same quality answers as the official one, with the bonus of less censorship. I pretty trivially got it to talk about the Tiananmen Square, and they can’t even ban me for it.
That said, that’s rarely the point. It’s usually because you can, a cost saving measure, sometimes you plainly just don’t need a good model, sometimes you want privacy, sometimes you need privacy at the cost of quality.
If your business is shoving customer reviews into a model, you really don’t need the best model for it to tell you how angry the customer is.
Personally I just do it for fun and because I can. Sometimes you just do things for no other reason than because you can.
if you wish to get roasted with additional specifics, see [email protected]
I want someone to prove his LLM can be as insightful and accurate as paid one.
Are you using LLMs as search engines?
Bold.
I use Gemma, LLama 3.2, and Deepseek to either fix formatting, summarize documentation to give me commands for Linux software, and write simple code structure for me to refine into working code.
Sure it takes longer to generate than a cloud compute would, but
-
privacy obviously. I know you dismissed it but that’s really the biggest reason anyone will have.
-
this feels better environmentally. I actually don’t know if that’s true, but it objectively touches less computers for such simple tasks. It would be wasteful of infrastructure to do it over the web.
-
it’s just cooler to have a conversation with my computer. I’ve learned a lot about how the whole process works and that’s more valuable to me as a non dev than just getting the end results.
-
Heh, you shouldn’t be paying for LLMs. Gemini 2.5 Pro is free, and so are a bunch of great API models. ChatGPT kinda sucks these days (depending on the content).
I have technical reasons for running local models (instant cached responses, constrained grammar, logprob output, finetuning), and I can help you set that up if you want, but TBH I am not going into a long technical proof of why that’s advantageous unless you really want to try this all yourself.
100% you don’t have to train a thing ollama uses open availability models. They many are decent, the best use a lot of ram/vram.
Probably best asking this in https://discuss.online/c/[email protected]
@[email protected] if you are willing to dismiss all of the reasons someone would prefer self-hosting an LLM, then of course a paid one will fare better, you self-host LLMs for privacy, or to tweak things which you cant do with external services, plus, in some cases it can be cheaper if you dont need a giant model to do a few basic things.
Take it to https://lemmy.world/c/fuck_ai we don’t care about this kind of rhetoric here
What you ask for is impossible to prove, let alone test, without a rigorous definition of “insightful” or “accurate”.
