Edit: unexpectatly we are back online. Will share how we found a solution later. There might be still some federation issues though.
Alt account of [email protected] here.
Our instance is currently down and I can’t get remote access to the servers. It appears that there might have been a hardware failure of the main firewall, which is the one thing I can’t work around remotely.
I am still trying a few things, but I am not very optimistic that I can get access.
The really unfortunate part is that just now I am on one of my rare work deployments abroad, so I also can’t access it physically during the next few weeks and my usual back up that could restart it is not available either.
As something like that never happened in 3 years operating the servers, I thought I can risk it, but murphy’s law seems inescapable 😓
I will try to keep you posted here on any updates, but probably there will not be much I can do for a while. Really bad timing 😥
Edit: we might use this “opportunity” to migrate the instance to Piefed, which has been an idea for quite some time now. I will keep you posted on that.
Thanks for letting us know what is going on.
Sorry that it took so long to get the word out. I guess we relied a bit too much on the xmpp server for internal communication.
Even the big operations break in unexpected ways. Facebook accidentally broke the authentication for their door locks when they took the site down a few years back and had to fly people to a datacenter and break down a door to get to servers and restart things.
We managed to resolve the issue and I have many ideas how to avoid this in the future 🤓
Thanks for all that you do for us and this community! A little downtime won’t hurt anyone. I would also be in gavour of PieFed migration 💙
I totally agree with everything you said :)
I just created an account in PieFed and it looks very interesting because there is the option to create your own feeds! I had the wrong impression that you can only choose from the existing ones. This is so exciting!!!
That sounds pretty cool. I don’t know much about Piefed — would we still be able to access posts from Lemmy instances?
Yes, they’re 100% cross compatible. As an example, you can access [email protected] from your Lemmy instance seamlessly.
Thank you for letting us know, good luck!
Thanks for letting us know
Very very bad timing. I just moved to that server and now it is just gone. I have moved to another server. I guess I am glad that it happened now so I know how poorly Slrpnk.net is managed.
If you want a professionally managed instance, you are welcome to pay for a hosted one.
Which is exactly what I did 2 years ago. Turns out the uptime is about the same and the price is higher than 0. But on the upside, if there’s downtime I can (usually) immediately reboot the server.
The server got a little more accounts on there than I initially expected though. But I’m fine with that.
Out of 3 years of reliable service, this is the one time the server has been down for any length of time, and only due to the admin not being able to be physically present for a wildly unlikely and unexpected problem.
I signed up with them yesterday and then they go down for I don’t know how long. This is my only experience with them.
I know, I’m the one who approved your application.
You happened to sign up right before we’ve had our first particularly bad issue in 3 years. Unlucky, yes, but it’s a bit out of order (IMO) to say we’re a poorly managed instance based on that, especially seeing as you lost nothing besides the time it took to write your application. :\
Does Lemmy let you retroactively deny applications?
As a matter of fact it does, actually 😅
My loss of time is small true. I will just move on. But keep in mind new users may not be as knowledgeable as myself.
Knowledgeable? Lmao
One issue in three years is poor management now? Wat
I signed up with them yesterday and then they go down for I don’t know how long. This is my only experience with them.
Ok, and?
That doesn’t change the fact this is a one off issue and theyve never had that issue in three years.This is a very big issue for a server no mater when it happens. A sever you can’t reset and you go on a trip ridiculous.
You are acting like this is a life or death situation. I think you may need to take a break from the internet.
You’re going to want to try “my little Internet” by Fisher-Price before using this adult Internet we’re on.
It usually can be reset, it’s entirely unexpected that a component that enables that functionality would randomly break. You’re essentially implying that any admin who self hosts cannot ever leave their house for any length of time without being irresponsible because they cannot ensure with 100% certainly that a computer component won’t fail while their gone, or a lightning storm won’t fry something, or a flood won’t harm the servers.
Like, dude, c’mon. We’re volunteers, and poVoq is running this stuff for free. We’re not an enterprise business with multiple staff on hand to swap out failed components 24/7 so that our ‘customers’ don’t lose millions of dollars.
If anything, this only proves the strength of decentralized media, as we’re able to swap to a functioning node and continue on our merry way while the other is being repaired. Like what an incredible and resilient system considering none of this is making anyone any money.
Anything less than perfection is unacceptable! 😤 /s
Yeah, I know. Complete Murphy’s law situation.
people hate the big cloud solutions, but this is the kind of thing their HA infrastructure prevents against… hardware failures
i dont enjoy using (or paying for) aws, but i will never have a firewall or disk failure.
Their HA infrastructure is all built on open source projects. The thing they have that we don’t is teams of SREs on-call 24/7.
Ouch. Best of luck getting it back up and running ASAP.
Solarpunks can have a little downtime, as a treat
Seriously, I think a big part of solarpunk ethos is combating the notion that everything has to always be available 24/7. Society pays a lot to deliver every convenience like fruit out of season from the other side of the world.
And on the punk side of things, anybody familiar with volunteer community action, cooperative events, whatever, knows that sometimes a person doing shit for the community fails to do the shit they were supposed to do because some other shit happened in their life. It happens. We just move on to the backup shit-doer-person or find a workaround for the lack of shit.
And this is the cool thing about a distributed network like Lemmy. If a giant centralized database like FB goes down, its users can’t do anything. If our favorite Lemmy node goes down, we can log into other nodes until our home base is back up.
Sending positive vibes, and passing offer of technical assistance once you get access again.
Hopefully it’s an easy fix once you get into things again.
Hey fellow slrpnk.net refugees… if you’re looking to recreate your list of subscribed communities, here’s a good place to start:
click here for a list of communities that are NOT politics, tech, or meme -related.
Most are currently active (except for the ones with a * which were less active last I checked) Sometimes politics, tech or memes sneak in but they’re not the focus.
GENERAL DISCUSSION / QUESTIONS
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
ART / PHOTOS
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected] *
- [email protected]
- [email protected] *
- [email protected] *
- [email protected]
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
ANIMALS
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected]
- [email protected] *
- [email protected]
COMICS / GRAPHIC NOVELS
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
ENTERTAINMENT
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
GENRES / STYLES
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected]
- [email protected]
- [email protected] *
- [email protected]
HISTORY
- [email protected]
- [email protected]
- [email protected] *
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
INFORMATION / KNOWLEDGE
- [email protected]
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
OTHER
- [email protected]
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected] *
FEDIVERSE
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected]
FINDING NEW/GOOD COMMUNITIES ON LEMMY
click here for a list of (mostly non-political) meme communities
MEMES, SOCIAL MEDIA REPOSTS, AND HUMOR
Most of these are currently active. (except for the ones with a * which were less active last I checked). Sometimes these include politics but that’s not the sole focus.
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected]
- [email protected]
- [email protected] *
- [email protected]
- [email protected] *
- [email protected]
- [email protected] *
- [email protected]
- [email protected]
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected]
- [email protected]
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected] *
- [email protected]
- [email protected] *
- [email protected]
- [email protected]
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected]
- [email protected] *
- [email protected] *
- [email protected]
- [email protected] *
- [email protected] *
- [email protected]
- [email protected] *
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
- [email protected] *
- [email protected] *
- [email protected]
- [email protected]
- [email protected]
- [email protected]
- [email protected]
(luckily I was posting this to newcomers whenever I had the chance…)
Great list, as usual!
Good stuff, Segio! :D
Try not to stress too much about it while you’re away! If nothing can be done, nothing can be done. Solarpunks need to be resilient to things like this anyway. Community over convenience.
Thanks, I am trying, but it really is quite annoying to lose access like that.
This is actually great timing, i’d say. Just now most of the northern hemisphere is starting into summer. At least i deeply feel it on a personal level that summer is arriving. I take this as a gift from the universe that directs me towards touching grass, and going swimming. Enjoy your summer, you all! Connect with your local community, if you can.
thirty minutes later
Okay I touched grass and now I’m bored and itchy
Glad that at least someone sees a positive side of it, because I sure don’t… well at least it gives me a bunch of ideas how to avoid such situations in the future 🤷
This is making me want to create an alt on slrpnk.net , since you’re so chill and communicative about issues.
Yeah me too, I’d wanted to for a while but never got around to it. Now I probably never will (unless they get the servers back up at some point in the future).
it seems they will, just not too soon
You’re very welcome! [email protected] / @[email protected] is an incredible admin, and up until now has declined donations, funding the entire instance several years now from his own pocket.
This current downtime is the exception that proves the rule. I hope he’s not stressing out too much. We’ll all be ready to jump right back into it when he gets back.
Already thinking about how to make the best of it 🤷 Maybe we can use this opportunity to try and migrate to Piefed? I had this in the back of my mind for some time already and Rimu seems optimistic that it is possible.
Wouldn’t that mean the instance would completely restart? I mean I don’t know much about piefed but logically it seems like the DB structure would be incompatible. Thus “Migrating” to piefed means a complete reboot, all posts, all users, all content just bye bye.
Not necessarily, no. We aim to preserve users, communities and posts/comments. Image uploads might get lost though.
Such an in-place migration will need extensive database operations and likely some support by the Piefed developer (to add support for bcrypt hashed passwords), but we are hopeful to make it happen and maybe this will result in a database migration script other Lemmy instances could also use.
If this turns out to be infeasible, we will stay with Lemmy rather than reset everything.
Oh awesome, I mean I’ve heard of other platforms like sublinks offering database migration but I didn’t think it would be feasible with piefed since unlike sublinks it’s very different than lemmy.
communicative about issues
Aren’t most admins this way?
*glances at your instance name*
Ah…
I miss BBS.
you can still have it with NodeBB!
Are there modem sounds to play air guitar to?
beep beep boop boop boop boop beep… EEEEEEEEEEeeeeeeee awwwwwww ka-dong ka-dong da kshhhhhhhhhhhhhhHHHHHHHHHHHHHHHHHHHHHHHHHH
eerie silence
Sorry to hear that! My main account is on slrpnk.net. But don’t let it stress you out, “solarpunk” says “punk” which implies “DIY” which means this sort of thing happens. Much better to have an instance run by an admin who truly cares, rather than some soulless corporation.
Ty for the update! slrpnk is my favorite instance that isn’t my home instance