Infrastructure
Servers are split by failure domain: what goes down together, and what is needed to bring the rest back.
Servers
GitLab
- own VPS - see GitLab
- 16 GB RAM and heavy upgrades would starve neighbours
- CI runners execute arbitrary code
Runners
- GitLab CI jobs, 3 at once - see Runners server
- 4 vCPU and 8 GB RAM
- no inbound ports but SSH, it only connects out to GitLab
Tools
- Vaultwarden, Uptime Kuma and similar small services - see Tools server
- 1 vCPU and 2 GB RAM are enough
- at another provider than GitLab, staging and production
it holds the keys to restore every other server, and monitors them from outside - it must not fail together with them
Production
- production applications of every project, several projects share one server
Staging
- staging applications of every project, several projects share one server
- same setup as production, can be smaller
staging runs unreviewed branches, load tests and infrastructure upgrades first - none of it may reach production resources, data or secrets
Where a new service goes
Production and staging
- a lightweight service that nothing needs in order to recover
Tools
- needed to restore other servers: secrets, backups, keys
- watches other servers: monitoring, uptime checks, alerts
Own server
- heavy or spiky on resources
- runs untrusted code
Exposure
- Vaultwarden - public, it is used from browsers and phones
- Uptime Kuma - behind the firewall, only its status page goes public if needed