Slow SSD issue. RAM is for chumps.
I don’t want to see the EXPLAIN for that query. This person really needs to learn more about sql, I’d wager.
I dunno why I didn’t realize you can add more swap to a system while running. Nice trick for a dire emergency.
Even better, you can
swapoffswap too!It’s Linux it’s made by people with a brain.
Jokes on you I used to have a 128gb ssd just for swap in my laptop
WHAT FUCKING QUERY ARE YOU RUNNING TO USE UP THAT MUCH MEMORY DAMN
A very very badly written one no doubt…
Why stop at just one full table scan?
In a database course I took, the teacher told a story about a company that would take three days to insert a single order. Thing was, they were the sort of company that took in one or two orders every year. When it’s your whole revenue on the line, you want to make sure everything is correct. The relations in that database were checked to hell and back, and they didn’t care if it took a week.
Though that would have been in the 90s, so it’d go a lot faster now.
What did they produce? Cruiseships?
We have a company like that here somewhere. When they have one job a year, they have to reduce hours, if they have two, they are doing OK, and if they have three, they have to work overtime like mad. Don’t ask me what they are selling, though. It is big, runs on tracks, and fixes roads.
No idea, but I imagine it was something big like that, yes. I think it was in northern Wisconsin, so laker ships are a good guess.
Poor man’s Optane
I’ve actually done something similar with a 2GB ram machine… 2GB ram / 8GB zswap, actually ran way faster lol
Yeah it works surprisingly well. I installed Gentoo on a 2005 era laptop a few years ago and had to keep adding zswap until Rust could compile for Firefox. Iirc it took about 12G of zswap to get it working, but it wasn’t too bad overall.
I actually ended up doing something like this on my main desktop lol, 16GB ram/ 32GB swap… I hate closing programs
Does the OOM killer actually work for anyone? In every linux system I’ve used, if I run out of memory, the system simply freezes.
Yes, it takes surprisingly long for the OOM killer to take action, but the system unfreezes. Just wait a few minutes and see whether that does the trick.
Yes. If you have swap the system will crawl to a halt before the process is killed though, SSDs are like a thousand times slower than RAM. Swapoff and allocate a ton of memory to see it in action.
Nvme PCIe 4 SSDs are quite fast now tho, you can get between DDR1 and DDR2 speeds from a modern SSDs. This is why Apple are using their SSDs as swap quite aggressively. I’m using a MacBook Pro with 16 GBs of RAM and my swap usage regularly goes past 20 GBs and I didn’t experience any slowdown during work.
Depends if the allocated memory is actively used or not. Some apps do not require a large amount of random access memory, and are totally fine with a small part of random access memory and a large part of not so random access and not so often used memory.
Alternatively I can imagine that MacOS simply has a damn good algorithm to determine what can be moved to swap and what cannot be moved to swap. They may also be using the SSD in SLC mode so that could contribute to the speedup as well.
It never kicks in for me when it should, but I figured out I can force trigger it manually with the magic SysRq key (Alt+SysRq+F, needs to be enabled first), which instantly recovers my system when it starts freezing from memory pressure.
Alt+SysRq+F, needs to be enabled first
Do note that this opens up a security hole. Since this can kill any app at random and is not interceptable, if you leave your PC in a public place, someone could come up and press this combo a few times. Chances are, it’ll kill whatever the locking app you’re using.
it does for me, usually by killing my session and throwing me back to the login screen
Oh yes. I’ve had massive compiles (well linking) which failed because of the OOM killer, and I did exactly the same, massive swap so it will just keep going. So what if it’s using disk as RAM and unusable for a few hours in the middle of the night, at least it finishes!
Yeah, default Ubuntu LTS webserver kicked the mysqld on a stupid query (but it worked on dev - all developers, someday) not too long ago…
Absolutely can and will take action. Doesn’t always kill the right process (sometimes it kills big database engines for the crime of existing), but usually gives me enough headroom to SSH back in and fix it myself.
I have limited experience with Linux, but why is it that when my system locks up, SSH still tends to work and let me fix things remotely? Like, if the system isn’t locked up, let me fix it right here and now and give me back control, if it is locked up, how is SSH working to help me?
So that’s the nifty thing about Unix is that stuff like this works- when you say “locked up”, I’m assuming you refer to logging in to a graphical environment, like Gnome, KDE, XFCE, etc. To an extent, this can even apply to some heavy server processes: just replace most of the references to graphical with application access.
Even lightweight graphical environments can take a decent amount of muscle to run, or else they lag. Plus even at a low level, they have to constantly redraw the cursor as you move it around the screen.
SSH and plain terminals (Ctrl-Alt-F#, what number is which varies by distro) take almost no resources to run: SSH/Getty (which are already running), a quick process call to the password system, then a shell like bash or zsh. A singular GUI application may take more standing RAM at idle than this entire stack. Also, if you’re out of disk space, the graphical stack may not be able to alive
So when you’re limited on resources, be it either by low spec system or a resource exhaustion issue, it takes almost no overhead to have an extra shell running. So it can squeeze into a tiny corner of what’s leftover on your resource-starved computer.
Additionally, from a user experience perspective, if you press a key and it takes a beat to show up, it doesn’t feel as bad as if it had taken the same beat for your cursor redraw to occur (which also burns extra CPU cycles you may not be able to spare)
Thanks, great answer!
Swap thrashing goes brrrrrrrrrrrrr
Hopefully that swap is on an SSD, otherwise that query may not ever finish lol
Once you’re deep into swap, things can get so slow that there’s no recovering from it.Protip: Put swapfile on ramdisk for highest speed
Unironically that’s how zram works
Doesn’t it compress the contents that it’s storing to help kind of get the best of both worlds?
You get faster storage because it’s in ram still, but with it being compressed there’s also “more” available?
I could be completely mistaken though
You are correct, although zram uses more cpu power since it compresses things. It’s not really an issue if you’re not using a potato :=)
even if you are using a potato it probably doesn’t have much ram so slightly slowing it to make things run smoother is a very popular choice
Today I learned!
Don’t do boy zram dirty, it has a ton of utility when you have ample spare compute and limited RAM.
Is that not how it works though? Lol
Exactly how I plan to deploy LLMs on my desktop 😹
I need it just for the initial load on transformers based models to then run them in 8 bit. It is ideal for that situation
That does make a lot of sense
You should be able to fit a model like LLaMa2 in 64GB RAM, but output will be pretty slow if it’s CPU-only. GPUs are a lot faster but you’d need at least 48GB of VRAM, for example two 3090s.
*laughs in top of the line 2012 hardware 😭
Amazon had some promotion in the summer and they had a cheap 3060 so I grabbed that and for Stable Diffusion it was more than enough, so I thought oh… I’ll try out llama as well. After 2 days of dicking around, trying to load a whack of models, I spent a couple bucks and spooled up a runpod instance. It was more affordable then I thought, definitely cheaper than buying another video card.
As far as I know, Stable Diffusion is a far smaller model than Llama. The fact that a model as large as LLaMa can even run on consumer hardware is a big achievement.
Both SD 1.5 and SDXL run on 4g cards, you really want fp16 though.
In principle it should be possible to get decentish performance out of e.g. an RX480 by using the (forced) 32-bit precision to do bigger winograd convolutions (severely reducing the number of
fmas needed) but don’t expect AMD to write kernels for that, ROCm is barely working on mid range cards in the first place.Meanwhile, I actually ended up doubling my swap because 16G RAM are kinda borderline to merge SDXL models. OOM might kick in, it might not, and in any case your system is going to lock without earlyoom.
I had couple 13B models loaded in, it was ok. But I really wanted a 30B so I got a runpod. I’m using it for api, I did spot pricing and it’s like $0.70/hour
I didn’t know what to do with it at first, but when I found Simply Tavern I kinda got hooked.
Same. I’m patient
I’d be lying if I said I hadn’t done something similar before.
Wrote my master thesis this way - didn’t have enough ram or knowledge, but plenty of time on the lab machine, so I let it do its thing over night.
Sorry, lab machine ssd.
It gave its life for academic achievement, there is no finer death for hardware. o7
You really need to index your tables. This has all the hallways of a Cartesian cross product.
Shoot and I thought my 30 second SQL queries were a problem
