I've had a set up with a dedicated server, running some stuff through ZeroTier as well as publicly. Last night, I thought I'd switch from ZeroTier to Tailscale. The following happened, in order:
- Purged ZeroTier (
sudo apt purge zerotier*, which removed zerotier-one and zerotier-systemd-manager) - Installed Tailscale through their script (
curl -fsSL https://tailscale.com/install.sh | sh) - Set up Tailscale (
sudo tailscale up), did not configure - At this point, I got a notification from my status page, saying one of my services relying on Ruby was returning an error 500.
As soon as I got the message saying my public site wasn't working, I immediately found that Ruby was hanging. Not long after when I went to try a different website I run, I found that my Redis server was also hanging. The following hour consisted of trying to get an error code, but instead getting nothing useful. Both redis-server and Ruby weren't returning anything.
I tried reinstalling the latest version of Ruby using rbenv, but even reinstalling has left it hanging. With Redis, I've tried restarting the server, but that's also kept hanging; the only thing close to an error that I got was:
Job for redis-server.service failed because a timeout was exceeded.See "systemctl status redis-server.service" and "journalctl -xeu redis-server.service" for details.Diving into that, these is the only logs I can get:
Jun 05 13:39:22 ns433454 systemd[1]: Starting Advanced key-value store...Jun 05 13:40:52 ns433454 systemd[1]: redis-server.service: start operation timed out. Terminating.Jun 05 13:40:52 ns433454 systemd[1]: redis-server.service: Failed with result 'timeout'.Jun 05 13:40:52 ns433454 systemd[1]: Failed to start Advanced key-value store.Jun 05 13:40:52 ns433454 systemd[1]: redis-server.service: Scheduled restart job, restart counter is at 136.Jun 05 13:40:52 ns433454 systemd[1]: Stopped Advanced key-value store.Jun 05 13:40:52 ns433454 systemd[1]: Starting Advanced key-value store...So currently, for solutions I have tried
- Restarting Redis
- Reinstalling Ruby via rbenv
- Purging Tailscale
I'm at a bit of a loss to what can work, and why this has happened - particularly as other services such as Apache, Node programs and MySQL are working completely fine. I don't even think Ruby is trying to contact any internet endpoints when being installed with rbenv (I'm not an expert on that), which makes it even more confusing on why it's failing, as it seems to maybe not be a network thing. I'm convinced both Ruby and redis-server share some sort of library that Tailscale may have potentially tweaked, but that's the best I can do on this unfortunately.
Any ideas on what may have gone wrong here?