Edit: unexpectatly we are back online. Will share how we found a solution later. There might be still some federation issues though.

Alt account of [email protected] here.

Our instance is currently down and I can’t get remote access to the servers. It appears that there might have been a hardware failure of the main firewall, which is the one thing I can’t work around remotely.

I am still trying a few things, but I am not very optimistic that I can get access.

The really unfortunate part is that just now I am on one of my rare work deployments abroad, so I also can’t access it physically during the next few weeks and my usual back up that could restart it is not available either.

As something like that never happened in 3 years operating the servers, I thought I can risk it, but murphy’s law seems inescapable 😓

I will try to keep you posted here on any updates, but probably there will not be much I can do for a while. Really bad timing 😥

Edit: we might use this “opportunity” to migrate the instance to Piefed, which has been an idea for quite some time now. I will keep you posted on that.

    • Draconic NEO
      link
      fedilink
      English
      015 days ago

      Yeah me too, I’d wanted to for a while but never got around to it. Now I probably never will (unless they get the servers back up at some point in the future).

    • threelonmusketeers
      link
      fedilink
      English
      6
      edit-2
      17 days ago

      communicative about issues

      Aren’t most admins this way?

      *glances at your instance name*

      Ah…

      • @[email protected]
        link
        fedilink
        English
        2
        edit-2
        17 days ago

        Lol. Yeah, sdf.org feels pretty hands-off. They have longevity going for them–I’ve had an sdf account of some sort since 2009 at the latest. The system started life as a BBS in the 80s. The name comes from the anime “Super Dimension Fortress”, which I’ve never seen.

    • Five
      link
      fedilink
      English
      116 days ago

      You’re very welcome! [email protected] / @[email protected] is an incredible admin, and up until now has declined donations, funding the entire instance several years now from his own pocket.

      This current downtime is the exception that proves the rule. I hope he’s not stressing out too much. We’ll all be ready to jump right back into it when he gets back.

      • @[email protected]OP
        link
        fedilink
        English
        0
        edit-2
        15 days ago

        Already thinking about how to make the best of it 🤷 Maybe we can use this opportunity to try and migrate to Piefed? I had this in the back of my mind for some time already and Rimu seems optimistic that it is possible.

        • Draconic NEO
          link
          fedilink
          English
          0
          edit-2
          15 days ago

          Wouldn’t that mean the instance would completely restart? I mean I don’t know much about piefed but logically it seems like the DB structure would be incompatible. Thus “Migrating” to piefed means a complete reboot, all posts, all users, all content just bye bye.

          • @[email protected]OP
            link
            fedilink
            English
            015 days ago

            Not necessarily, no. We aim to preserve users, communities and posts/comments. Image uploads might get lost though.

            Such an in-place migration will need extensive database operations and likely some support by the Piefed developer (to add support for bcrypt hashed passwords), but we are hopeful to make it happen and maybe this will result in a database migration script other Lemmy instances could also use.

            If this turns out to be infeasible, we will stay with Lemmy rather than reset everything.

            • Draconic NEO
              link
              fedilink
              English
              115 days ago

              Oh awesome, I mean I’ve heard of other platforms like sublinks offering database migration but I didn’t think it would be feasible with piefed since unlike sublinks it’s very different than lemmy.