What goes where? #13

Open
opened 2026-04-13 00:48:15 -04:00 by chipmunkmc · 3 comments
Owner

So in the current setup of chipmunk.land, we have two main nodes (both running Alpine Linux), server0.local and server1.local.
The former machine is a Dell Inspiron 3268 w/ a 7th-gen Intel i3 CPU, 32gb of ram (fast), 1tb HDD, and runs everything but the virtual machines.
The latter one is some HP laptop w/ some Intel Pentium Silver CPU, 16gb of ram 1tb NVME SSD (fast), and runs said virtual machines.

So I've recently been questioning whether putting the great majority of services (including all new ones) on server0.local will always be sustainable, or even if so, whether it's optimal.
32gb of RAM is plenty, but even it is limited, and it seems I/O on server0 can be limited too.
Forgejo (due to mirrors), Minecraft servers, and some database daemons all require quite a bit of wiggle room when it comes to memory, however many smaller things currently share hardware with them too.

On the topic of I/O, when aggressive scrapers (those said to be AI-related, often) have slowed down this Forgejo instance before, they also made server0 as a whole much slower (perhaps because reading tons of Git object requires tons of disk I/O
I plan to look into different ways of filtering out these (accidental) attacks, though the current cookie hack just works and I'm a lazy cunt so probably not too soon

So server0's CPU is fairly good, which helps a lot with daemons such as:

  • Forgejo - ran quite slow when hosted on server1 at one point; reading Git objects likely uses the CPU a lot too
  • Minecraft servers - ditto
    However, some daemons don't need as much CPU and seemingly ran fine on server1:
  • ergo - the current IRCd in use
  • Nginx - static sites (no promises, but could even be faster due to NVME SSD speeds), proxying
  • linx-server - see the NVME remark

Moving some of these could perhaps take some load off of server0.
Note that if I ever need to add extra storage to either server, I may not use the original storage medium (i.e. the NVME benefit could disappear).
Also, proxying services over LAN likely adds a bit of latency (needed in both cases), unfortunately.

We don't need to stop there, though. We could add extra nodes:

  • A former server0 laptop, which was an SGIN X15 laptop, also did pretty well with CPU-intensive tasks, and has 24gb of ram and a 1tb M.2 SATA SSD.
    The fans of this laptop, ever since it was ran 24/7 for a while, started making weird noises at times, and eventually going between spinning and not spinning, so I decommissioned the machine to avoid hardware damage.
    The battery got inflated (I left it plugged in for a long time, oops) but I disconnected it from the motherboard and the laptop still boots. Seems to make a weird noise while booted without a battery (unless it always does? or did I forget?) though and the charging light blinks...
  • Macbook - a friend offered me one a while ago, no idea which one/what specs... could probably shoehorn Alpine Linux onto it (Intel Macs are nearly PCs, and work on Arm ones was done by the Asahi Linux project, I believe)

If I feel crazier, I could add on mobile hardware:

  • Semi-old flagship phones: Google Pixel XL, Samsung Galaxy S8, Samsung Galaxy S8 Active, some newer Pixel (if I can fix it)
    These notably seem to be able to run Minecraft servers well.
  • Less powerful phones for less intense tasks, for instance Samsung Galaxy S5, Motorola Moto E 2nd Generation, etc

However it's a little harder to put Alpine on these, though we can likely reuse work from postmarketOS.
Qualcomm S8 development is minimal (bootloader unlocking requires EM token fuckery) but the S8 phones likely use a known SoC, and all we need is storage/USB (should be an SoC feature)
However, one advantage of standard computers is that we can use ludicrous (edit: not really lol) amounts of RAM and storage, something hard to impossible on this hardware.

Also I thought of naming the Macbook satania as a joke, and then thought in retrospect, maybe I should name the servers after anime girls (or just characters in general) instead of server%d...
Perhaps that's too much of a weeb thing to do though I dunno.

So in the current setup of chipmunk.land, we have two main nodes (both running Alpine Linux), `server0.local` and `server1.local`. The former machine is a Dell Inspiron 3268 w/ a 7th-gen Intel i3 CPU, 32gb of ram (fast), 1tb HDD, and runs everything but the virtual machines. The latter one is some HP laptop w/ some Intel Pentium Silver CPU, 16gb of ram 1tb NVME SSD (fast), and runs said virtual machines. So I've recently been questioning whether putting the great majority of services (including all new ones) on `server0.local` will always be sustainable, or even if so, whether it's optimal. 32gb of RAM is plenty, but even it is limited, and it seems I/O on server0 can be limited too. Forgejo (due to mirrors), Minecraft servers, and some database daemons all require quite a bit of wiggle room when it comes to memory, however many smaller things currently share hardware with them too. On the topic of I/O, when aggressive scrapers (those said to be AI-related, often) have slowed down this Forgejo instance before, they also made server0 as a whole much slower (perhaps because reading tons of Git object requires tons of disk I/O I plan to look into different ways of filtering out these (accidental) attacks, though the current cookie hack just works and I'm a lazy cunt so probably not too soon So server0's CPU is fairly good, which helps a lot with daemons such as: * Forgejo - ran quite slow when hosted on server1 at one point; reading Git objects likely uses the CPU a lot too * Minecraft servers - ditto However, some daemons don't need as much CPU and seemingly ran fine on server1: * ergo - the current IRCd in use * Nginx - static sites (no promises, but could even be faster due to NVME SSD speeds), proxying * linx-server - see the NVME remark Moving some of these could perhaps take some load off of server0. Note that if I ever need to add extra storage to *either* server, I may not use the original storage medium (i.e. the NVME benefit could disappear). Also, proxying services over LAN likely adds a bit of latency (needed in both cases), unfortunately. We don't need to stop there, though. We could add extra nodes: * A former server0 laptop, which was an SGIN X15 laptop, also did pretty well with CPU-intensive tasks, and has 24gb of ram and a 1tb M.2 SATA SSD. The fans of this laptop, ever since it was ran 24/7 for a while, started making weird noises at times, and eventually going between spinning and not spinning, so I decommissioned the machine to avoid hardware damage. The battery got inflated (I left it plugged in for a long time, oops) but I disconnected it from the motherboard and the laptop still boots. Seems to make a weird noise while booted without a battery (unless it always does? or did I forget?) though and the charging light blinks... * Macbook - a friend offered me one a while ago, no idea which one/what specs... could probably shoehorn Alpine Linux onto it (Intel Macs are nearly PCs, and work on Arm ones was done by the Asahi Linux project, I believe) If I feel crazier, I could add on mobile hardware: * Semi-old flagship phones: Google Pixel XL, Samsung Galaxy S8, Samsung Galaxy S8 Active, some newer Pixel (if I can fix it) These notably seem to be able to run Minecraft servers well. * Less powerful phones for less intense tasks, for instance Samsung Galaxy S5, Motorola Moto E 2nd Generation, etc However it's a little harder to put Alpine on these, though we can likely reuse work from postmarketOS. Qualcomm S8 development is minimal (bootloader unlocking requires EM token fuckery) but the S8 phones likely use a known SoC, and all we need is storage/USB (should be an SoC feature) However, one advantage of standard computers is that we can use ludicrous (edit: not really lol) amounts of RAM and storage, something hard to impossible on this hardware. Also I thought of naming the Macbook `satania` as a joke, and then thought in retrospect, maybe I should name the servers after anime girls (or just characters in general) instead of `server%d`... Perhaps that's too much of a weeb thing to do though I dunno.
Author
Owner

Oh and forgot to say, I could also name servers after hardware (like inspiron/i3/etc for server0.local), or perhaps use another scheme.

Oh and forgot to say, I could also name servers after hardware (like `inspiron`/`i3`/etc for `server0.local`), or perhaps use another scheme.
Author
Owner

I realized other nodes could be added for purposes other than taking load off of server0, such as redundancy and/or general load balancing.

Also the Forgejo incident seems less likely to happen with the current cookie hack, though in general separation could make other DoS (including intentional attacks) situations not break the rest of the services.
It's worth noting a Minecraft exploit outright froze the machine once, causing me to need to go to the server IRL and hard reset it.
Also, it could future-proof things as more users on a service and/or more services can put more load on a mahcine.

As for the proxying stuff, I'd like to note that HTTP/HTTPS traffic is handled entirely by nginx running on server0.local, and so it runs on one node and has direct access to my LAN including other servers.

I realized other nodes could be added for purposes other than taking load off of server0, such as redundancy and/or general load balancing. Also the Forgejo incident seems less likely to happen with the current cookie hack, though in general separation could make other DoS (including intentional attacks) situations not break the rest of the services. It's worth noting a Minecraft exploit outright froze the machine once, causing me to need to go to the server IRL and hard reset it. Also, it could future-proof things as more users on a service and/or more services can put more load on a mahcine. As for the proxying stuff, I'd like to note that HTTP/HTTPS traffic is handled entirely by nginx running on `server0.local`, and so it runs on one node and has direct access to my LAN including other servers.
Author
Owner

Note that @opt suggested using a separate disk for Forgejo, which I may or may not do.

Also the power strip the servers are plugged into has no unused slots for extra nodes, and last time I tried adding an extra computer (by swapping out the strip temporarily) it seemingly didn't boot?
Either way I'd be using the same outlet for a ton of things, which isn't ideal.

Note that @opt suggested using a separate disk for Forgejo, which I may or may not do. Also the power strip the servers are plugged into has no unused slots for extra nodes, and last time I tried adding an extra computer (by swapping out the strip temporarily) it seemingly didn't boot? Either way I'd be using the same outlet for a ton of things, which isn't ideal.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
chipmunk.land/misc#13
No description provided.