Benin server-to-server handover — machines to provide

Read this first · S4 archive

The S4 archive is under an incident quarantine. Nothing may be copied off it, deleted on it, or moved around inside it until the investigators clear it. A Hetzner Cloud firewall named incident-2026-09-16-isolement-archive-bj was created on 2026-09-16 at 08:21 UTC on archive.monentreprise.bj. It permits traffic in both directions only with a single external address, under a rule labelled "Enqueteurs uniquement" (investigators only) — a label someone typed into the firewall's description, not an observed fact. The host's own firewall also drops all outgoing traffic, so a server-to-server copy fails silently as the machine stands.

We make no claim about what the incident is; that is for the investigators to determine. What the handover needs from them is written clearance, an agreed export list and a transfer window. The V4 platform at services.monentreprise.bj is not under quarantine and can proceed on its own timetable.

1 · The two systems at a glance

Two different products on two different machines, needing two different methods. V4 is the live platform citizens and officers use today. S4 is the legacy archive that keeps old records readable — and it shares its machine with other countries' sites.

V4 — services.monentreprise.bj

What it is: The live eRegistrations platform. Applications are filed and processed here.
What runs on it: A single-node Docker Swarm, 28 services and 37 running containers under the stack name eregistrations. PostgreSQL 16.15, MongoDB 6.0.29 and Redis 6.0.16 run directly on the operating system, not in containers. HAProxy 2.4.30 terminates HTTPS and routes by host name.
Its data: 1004 GB of PostgreSQL, of which camunda alone is 654 GB, plus 20 GB of MongoDB. Alongside them, /opt/volumes holds 777 GB of application data bind-mounted into the containers — MinIO objects, OpenSearch indices, Graylog, GDB, ActiveMQ.
Current machine: Hetzner ccx53: 16 physical cores / 32 threads, dedicated (AMD EPYC-Milan), 128 GB RAM, a root disk of 600 GB provisioned, 564 GB usable on /, 77 GB in use, and a 4.9 TB volume on /opt. Ubuntu 22.04.5 LTS, kernel 5.15.0-191.
Tenants on the machine: Benin only.

S4 — archive.monentreprise.bj

What it is: The legacy archive. Its value is that old files still open. It is under quarantine (see the notice above).
What runs on it: Plesk Obsidian 18.0.80.7, with nginx 1.30.4 in front of Apache 2.4.29, four PHP versions (7.2, 7.3, 7.4, 8.0) and MariaDB 10.5.9. Mail, FTP, bind DNS, Grafana, an Ollama model server and five Docker containers also run here.
Its data: ereg_benin_prod is 17.08 GB; all schemas together are about 20.5 GB logical and 33 GB on disk. The production document store is 718 GB on a second volume, with a further 7.4 GB for the test site.
Current machine: Hetzner cpx51: 16 shared vCPU (AMD EPYC), 32 GB RAM, a 226 GB root disk that is 93% full, and a 984 GB volume. Ubuntu 18.04.6 LTS, kernel 4.15.0-213, 325 days of uptime, no swap configured.
Tenants on the machine: Eleven Plesk domains, including Cameroon's production site. Not Benin's alone — see blocker 2 in section 4.

The difference that decides the method. V4 can be rebuilt and restored as one system, because one country owns the whole machine. S4 cannot: a disk-level clone of that box would hand another country's production data to Benin.

2 · Machines to provide for the V4 platform

The specifications in this table are the authoritative list for V4. Every other mention on this page refers back to it.

The biggest lever on what you buy: about two thirds of the stored data is logs, history and leftovers

Of the 2.1 TB steady-state footprint on /opt, roughly 1.4 TB is history, logs and leftovers that can largely be reduced, and roughly 580 GB is business data that cannot. The single most striking figure: the log indices (512 GB) take more than twice the space of the actual uploaded business documents (217 GB).

Largely reducible — about 1.40 TB

1.40 TB

Camunda process history (`camunda` database)	654 GB
OpenSearch log indices (`/opt/volumes/opensearch`)	512 GB
Stale manual SQL dumps (`/opt/databases/postgresql/tmp`)	195 GiB
ActiveMQ message store (`/opt/volumes/activemq`)	42 GB

Business data — about 580 GB

580 GB

MinIO uploaded documents (`/opt/volumes/minio`)	217 GB
`gdb` database (the registry)	202 GB
`display_system` database — the applicant and officer screens	120 GB
`bpa` database and MongoDB	44 GB

The 8 TB recommendation below assumes no cleanup happens. Clean up first — Camunda history cleanup, an OpenSearch and Graylog retention policy, deleting the stale dumps after confirming them, draining the ActiveMQ backlog — then re-measure, and the requirement could fall from 8 TB to about 4 TB. That is the difference between two purchases, so it is worth doing before the order is placed, not after.

The one assumption behind every post-cleanup figure on this page. Assume only the certain part of the 1.4 TB comes back: the 654 GB of Camunda history and the 195 GiB of stale dumps, about 850 GB, or roughly 60% of it. The footprint then falls from 2.1 TB to roughly 1.25 TB. Log retention and the ActiveMQ backlog would take it lower, but both are negotiable, so nothing here counts on them. The "about 4 TB" disk below and the post-cleanup transfer times in the bandwidth note are both derived from this sentence and from nothing else.

Camunda is the workflow engine that drives every application in progress. History cleanup removes the recorded history of applications that have already finished; applications still in flight are not touched. UNCTAD operations runs it on the source machine, verifies the saving on a copy first, and only then is the new disk sized. The 654 GB is measured; the saving is not.

The 42 GB ActiveMQ store is abnormally large for a message broker and is worth investigating as a backlog before anything is copied.

Machine and what it does	Processor	RAM	Storage
V4 platform host (primary) — required Runs everything on one machine: 28 Docker Swarm services and 37 containers, plus PostgreSQL 16, MongoDB 6 and Redis installed directly on the operating system, behind HAProxy which terminates HTTPS. Measured versus recommended. The processor count and the 600 GB root disk are like-for-like with what runs today. The 256 GB of RAM and the 8 TB volume are recommendations: RAM because 13 to 14 GiB of swap is in use on the current 128 GB machine, storage because the volume must hold the data plus the transient full copy the nightly backup stages, plus growth. Order dedicated cores, not shared or oversubscribed virtual ones. If the cleanup above happens first, re-measure: on the recovery assumption stated in the lever above — roughly 1.25 TB left in place — about 4 TB is enough.	16 dedicated physical cores (32 threads)x86-64 with AVX — dedicated, not shared vCPU	256 GB128 GB is the floor, and is the size that swaps today	/ : 600 GB SSD /opt : 8 TB NVMe 32 GB swap
Public portal host — required if the portal moves, blocked on blocker 3 Serves `monentreprise.bj` and `www`, a separate Next.js application. It is not on the platform server: its DNS points to 37.27.8.11 while every other name points to 49.13.197.27. Do not order this one yet. Nobody has settled whether the portal is Benin's to run, which is blocker 3 in section 4. The host also sits outside the Hetzner project reachable with the available API tokens, so nothing could be collected from it: the sizes in this row are an unpriced placeholder, not a measurement. Settle the scope question, confirm the real size with whoever operates it, then order. A clone that copies only the platform server leaves the citizen-facing portal behind.	4 shared vCPUunpriced placeholder — no measurement exists	8 GBunpriced placeholder	100 GB SSDunpriced placeholder
V4 backup target — required Holds PostgreSQL base backups, MongoDB dumps and the MinIO object copy on a machine that is not the platform host. Recommendation, and the gap it closes is measured. Today `pg_basebackup` and `mongodump` write into `/opt/backup` on the same volume as the live data, so at best they protect against a mistake and not against losing that volume. One full copy is about 2.0 TB, measured from the directory totals: `/opt/databases` 1.2 TB plus `/opt/volumes` 777 GB. 8 TB therefore holds several generations plus growth.	4 shared vCPU	16 GB	8 TB
Second platform node (optional but advised) Turns the one-node Docker Swarm into a cluster that survives losing a machine. Recommendation, not measured. The live system is a single-node Swarm with one manager, so any hardware failure is a full outage. If the budget does not stretch to a twin, the backup target above plus a written rebuild procedure is the minimum acceptable substitute.	16 dedicated physical cores (32 threads)	256 GB	/ : 600 GB SSD /opt : 8 TB NVMe

The site internet link is part of the order, and takes longer to procure than servers. Ask for 1 Gbit/s symmetric as the minimum. The V4 payload to move is about 2.0 TB — /opt/databases 1.2 TB plus /opt/volumes 777 GB — not the 2.9 TB peak, because that peak is the nightly staging copy nobody transfers. At 1 Gbit/s that is about 4.5 hours of continuous transfer, and the S4 extract of about 790 GB is under 2 hours. At 100 Mbit/s the same two jobs are about 44 hours and about 18 hours — nearly two days for V4 alone, before any delta pass. If the V4 cleanup happens first and the footprint falls to roughly 1.25 TB on the recovery assumption stated in the lever above, that becomes about 3 hours at 1 Gbit/s or about 28 hours at 100 Mbit/s.

What was measured on the V4 machine

Processor. 16 physical cores / 32 threads, dedicated — AMD EPYC-Milan, KVM, AVX present — on Hetzner type ccx53.
Load. 13.13 / 12.01 / 11.25 over 1, 5 and 15 minutes across 32 threads: about 41% of capacity, and rising across the three windows.
Memory and swap. 122 GiB total, 53 GiB used, 67 GiB in buffers and cache, 67 GiB available — yet 13 to 14 GiB of the 31 GiB swap area is in use after 12 days and 23 hours of uptime. The kernel's memory pressure figure is 0.25% of the last five minutes stalled on memory, which is effectively idle; as a rule of thumb rather than a measured threshold, sustained double-digit figures would indicate real pressure. The machine is not struggling now, but it has already pushed 13 GiB out to disk, which is why 256 GB is recommended.
Four mule containers sit at 3.98–4.00 GiB against a 4.00 GiB limit on the swarm service eregistrations_mule. Mule is the integration service that carries messages between the platform and external systems. They are repeatedly killed at that ceiling — "non-zero exit (137)" tasks 24 hours and 6 days before collection. There are zero kernel out-of-memory entries, because a container limit is enforced by Docker, not by the kernel. Raising it is a configuration change, not a purchase: 67 GiB of host RAM is free.
Storage, and why it is a range. /opt is a 4.9 TB volume that read 2.1 TB used at 12:36 UTC and 2.9 TB an hour earlier. The measured cause is the nightly pg_basebackup, which stages a full second copy of the roughly 1 TB database into /opt/backup/data/postgres and then deletes it. Plan for 2.1–2.9 TB, not a point value.
Inside /opt. /opt/databases 1.2 TB — PostgreSQL 16/main 1004 GB, stale manual dumps in tmp 195 GiB, MongoDB 20 GB. /opt/volumes 777 GB. /opt/backup 58 GB. /opt/eregistrations 38 MB.
/opt/volumes is application data bind-mounted into the containers, not Docker's own volume store. docker service inspect shows bind mounts such as /opt/volumes/minio to /data and /opt/volumes/opensearch/data to the OpenSearch data directory. Docker's own named volumes total 211.6 MB. A copy that follows docker volume would move 211.6 MB and leave 777 GB behind. Inside it: OpenSearch 512 GB, MinIO 217 GB, ActiveMQ 42 GB, GDB 6.1 GB.
Only one PostgreSQL cluster exists (16/main, port 5432). The 195 GiB in /opt/databases/postgresql/tmp is not a second database but three leftover one-off dumps from June and October 2025, sitting on the production data volume. Confirm, then delete; do not copy them.
Databases. PostgreSQL server 16.15 (the psql client on the box is 18.6 and is not the server version): camunda 654 GB, gdb 202 GB, display_system 120 GB, bpa 24 GB, the rest under 2 GB. MongoDB 6.0.29 holds 20 GB; Redis 6.0.16 holds 243 MB of cache.
Growth: about 72 GB per month. /opt was created 2024-03-28 and read 2.1 TB on 2026-09-18 — 29.7 months. That is average accumulation, not a measured trend, and data may have been migrated in at the start: treat it as an upper bound and plan with headroom rather than precision.
Offsite backups are not running. The nightly offsite mirror runs mc mirror inside the minio-manager container, and that swarm service is stopped (0/1), so the log contains only "No such container: mc". postgresbackup.sql is 0 bytes, and the only retained archives are two MongoDB tarballs a year old. The nightly mongodump does work (46 GB of BSON) but is overwritten on the same volume as the live data. Losing this machine today loses PostgreSQL entirely.
Platform and network. Docker Engine 29.8.0, swarm active, one manager. Three ports face the internet (22, 80, 443); ufw on the host is the only packet filter, with no cloud firewall attached. The ports that matter for the rebuild are in section 6; that table is a working subset, not the full listener inventory.

3 · Machines to provide for the S4 archive

Nothing may move until the investigators clear it. When they do, take Benin's subscription, not the machine: this is a shared Plesk box whose eleven domains include Cameroon's production site. The specifications in this table are the authoritative list for S4. Order the hardware now if you wish — ordering is not copying — but no extraction, deletion or staging happens on the source host until clearance is in writing.

Machine and what it does	Processor	RAM	Storage
archive-bj-app — required The Benin legacy archive itself: Plesk with nginx in front of Apache, PHP, and the MariaDB databases. Benin subscription only — no other tenant. Recommendation. The processor count is halved because the source runs at a steady load of 1.00 on 16 shared vCPU; dedicated cores are worth more than shared ones, so 8 dedicated keeps a wide margin. Each volume is sized from the source. System 160 GB: the source root holds 200 GB used, of which 33 GB of MariaDB moves to the database volume and 39 GB of retained dumps moves to the backup target, leaving about 128 GB — 160 GB gives that room to grow instead of arriving at 93% full as the source is. Database 200 GB: 33 GB on disk today, with room for the dev, test and migration schemas and for dump-and-reload headroom. Documents 2 TB: a measured payload of about 790 GB, and the source volume was created 2021-04-09 and holds 725 GB, about 130 GB per year of accumulation.	8 dedicated cores	32 GBplus a 16 GB swap file; the source has no swap at all	3 volumes: 160 GB SSD system 200 GB SSD database 2 TB documents
archive-bj-backup — required Backup target on a separate machine: receives the nightly database dumps and a mirror of the document tree. Recommendation, and the gap it closes is measured. Today the 30 retained nightly dumps (39 GB) are written onto the archive's own root filesystem, and no offsite copy of that database exists on the host.	2 shared vCPU	8 GB	2 TB
archive-bj-test (optional) Rehearses the restore before cutover, then keeps the dev and test copies of the archive. Recommendation. The dev and test databases are 1.63 GB and 1.42 GB, and the test document tree measures 7.4 GB. A 200 GB data volume therefore leaves generous room; size it down if this machine is only ever used to rehearse the restore.	4 shared vCPU	8 GB	100 GB system 200 GB datare-size after measuring

Transfer time for S4: about 790 GB is roughly 18 hours of continuous transfer at 100 Mbit/s, or under 2 hours at 1 Gbit/s. The link requirement is stated once, in section 2.

What was measured on the S4 machine

Quarantine. S4 is isolated by an incident firewall; the full account is in the notice at the top of this page. The consequence for the handover is that a server-to-server copy fails silently as the machine stands, and that every instruction below is conditional on the investigators clearing the host first. Behind that quarantine the host still allows 80 and 443 from 137.255.8.0/22, a Benin network.
Benin's payload is about 790 GB. Precisely: 725 GB of documents on the volume, 48 GB of web content once the 39 GB of dumps are excluded, and about 20 GB of Benin databases — 793 GB, call it 790. Production alone is about 783 GB: 718 GB production documents, 48 GB web content and 17.08 GB for ereg_benin_prod.
The root filesystem is nearly full. /dev/sda1 is 226 GB and 93% full: 200 GB used, 18 GB free, with no swap configured. Top consumers: /var/www 89 GB, /home 44 GB, /var/lib/mysql 33 GB, /var/lib/docker 4.4 GB, /var/qmail 4.2 GB.
The biggest avoidable consumer on that filesystem is 39 GB of retained nightly database dumps in /var/www/vhosts/benin.eregistrations.org/_backup_prod — 30 full dumps of about 1.23 GB each, 19.5% of everything used on the root disk. Once the investigators clear the host, moving those dumps to the backup target is the fastest way to relieve the 93%. Until then, leave them exactly where they are.
Those dumps sit beside the web root, not inside it. The document root is .../httpdocs, confirmed by plesk bin subscription --info and by the vhost configuration. They are not served over HTTP: this is a disk-space and credential problem, not a public exposure.
The second disk. /dev/sdb is 984 GB with 725 GB used (78%) and 209 GB free as measured by df — subtracting 984 − 725 gives 259 GB, which is wrong: the difference is ext4 reserved space, so never size a staging area by subtraction. It is mounted at /mnt/volume-bj, holding only two directories: monentreprise.bj at 718 GB — the archive's production document store — and test.benin.eregistrations.org at 7.4 GB. Those documents are the real payload; the 17 GB MariaDB database is small by comparison.
Operating system. Ubuntu 18.04.6 LTS on kernel 4.15.0-213, per /etc/os-release — the Hetzner API reports 16.04, which is the original 2019 install image, not the running system. Ubuntu 18.04 left standard support in April 2023.
Databases. MariaDB 10.5.9: ereg_benin_prod 17.08 GB, ereg_benin_dev 1.63 GB, ereg_benin_test 1.42 GB, ereg_benin_migration 0.12 GB and smaller schemas — about 20.5 GB logical, 33 GB on disk.
Tenancy. Plesk Obsidian 18.0.80.7 manages 11 domains on this single box: seven are Benin's, four are not. Seven plus four is the whole box. Both lists are written out once, side by side, in blocker 2 of section 4.
Backups. A root cron job at 19:00 dumps ereg_benin_prod into the subscription directory; an 18:00 job deletes dumps older than 30 days. The most recent is dated 2026-09-17 19:02, so backups ran through the incident. No offsite copy exists on the host, and the cron line embeds the database password in plain text; the value is not reproduced here.
Other software on the box. Five Docker containers — the asin and stats stacks with their Redis instances, 797 MB of images of which 764 MB is reclaimable — a full mail stack on 8 ports, FTP, bind DNS, Grafana, and an Ollama model server on 127.0.0.1:11434.
Account ownership. The server sits in a Hetzner Cloud project separate from Benin's, so account access and billing must be settled before transfer. Server id 3406120, created 2019-10-07 in Nuremberg. See item 14 in section 4.
The bind DNS zones on this host are vestigial. It carries zones for six domains, but the public delegation uses Hetzner DNS, Cloudflare and abovedomains.com instead — Plesk creates these zones automatically. Stated explicitly because the opposite would have been an outage risk: the quarantine is not affecting DNS for any of these domains. The zones can be dropped once the investigators clear the host. Until then, change nothing in DNS on this machine.

4 · Decisions to make before anything is copied

Fourteen items, each with an owner, ordered by how hard they block the work. Three are blockers that people must settle before engineers start. This is the authoritative list of decisions; sections 2, 3 and 5 do not repeat it. Owners are: Benin IT, the receiving team; UNCTAD operations, who run the source machines today; and the investigation owner, who controls the S4 quarantine.

Blocker 1

S4 is under an incident quarantine — nothing leaves it, and nothing changes on it, until the investigators agree

Owner: investigation owner

FoundS4 is isolated by an incident firewall on server id 3406120; the full account is in the notice at the top of this page. Nothing leaves the host and nothing changes on it until the investigators agree.

DoObtain written clearance, the agreed export list and a transfer window from the investigation owner. Name a contact now: this gates every other S4 item on this page. Any transfer then needs either a temporary single-purpose firewall rule — opened, used, removed and re-verified against the quarantine — or offline media they have inspected. When the transfer is authorised, extract data, not disk images, so they can review exactly what left. UNCTAD operations prepares the export list; Benin IT countersigns it.

Blocker 2

S4 also holds Cameroon's production data — a whole-machine clone would hand it to Benin

Owner: UNCTAD operations + Benin IT

FoundPlesk manages 11 domains on the one box: seven are Benin's, four are not — 7 + 4 = 11, the whole box. MariaDB likewise holds Benin's schemas alongside others, and /mnt/volume-bj holds only Benin directories (718 GB production, 7.4 GB test).

The split, in fullThe two lists are set out here so the count can be checked without reading prose.

Benin's — seven

`archive.monentreprise.bj`	in
`benin.eregistrations.org`	in
`chat.archive.monentreprise.bj`	in
`dev.benin.eregistrations.org`	in
`old-benin.eregistrations.org`	in
`stats.archive.monentreprise.bj`	in
`test.benin.eregistrations.org`	in

Not Benin's — four, the exclusion list

`cameroun.eregistrations.org`	out
`cameroun-test.eregistrations.org`	out
`eregistrations.org`	out
`mybusiness.cm`	out

DoOnce the investigators have cleared the host, and not before: extract only the Benin items — the Plesk subscriptions for archive.monentreprise.bj and benin.eregistrations.org — and, if the counterpart wants them, the other five Benin names listed above — a fresh logical dump of ereg_benin_prod plus whichever of the dev, test and migration schemas are wanted, and the two Benin directories under /mnt/volume-bj. Both sides sign the four-entry exclusion list before that cleared copy starts — and, as with everything on this machine, only once the investigators have cleared it.

Blocker 3

The public portal is not on the V4 machine, and nobody has confirmed who operates it

Owner: UNCTAD operations, then Benin IT

Foundmonentreprise.bj and www.monentreprise.bj resolve to 37.27.8.11, a separate Next.js host outside the Hetzner project reachable with the available API tokens. Nine service subdomains resolve to the V4 server; the citizen-facing apex does not.

DoIdentify the operator of 37.27.8.11, obtain its specification and its content, and settle whether it moves with this handover. A migration that copies only the platform server leaves the page most citizens see behind.

High

Neither system survives losing its machine today — and on V4 the offsite mirror has been failing nightly

Owner: UNCTAD operations

FoundOn V4 the MinIO mirror fails nightly because the minio-manager service is stopped, the PostgreSQL backup log is 0 bytes, and the only working copy — a mongodump — lands on the same volume as the live data. On S4 the nightly dump is written to the archive's own root disk with no offsite copy.

DoOn V4: restart minio-manager, prove one complete mirror run, and fix retention so archives survive the night. Take one verified offsite backup of V4 before the migration starts, not after. On S4, the equivalent work waits for clearance (blocker 1).

High

Two production passwords sit in plain text in automation on the two hosts

Owner: UNCTAD operations

FoundOne in the V4 maintenance.sh (MongoDB), one in the S4 root crontab (the archive database, also visible in ps while mysqldump runs). Neither value is reproduced anywhere in this document; only their locations are recorded.

DoRotate the V4 credential now, so the rebuilt system is built with the new value. Rotate the S4 credential once the investigators clear the host — changing anything on a quarantined machine is their call, not ours.

High

Both machines run ageing or end-of-life operating systems — rebuild on a supported OS instead of imaging the disk

Owner: Benin IT with UNCTAD operations

FoundS4 runs Ubuntu 18.04.6, out of standard support since April 2023, on a machine built in 2019. V4 runs Ubuntu 22.04.5, supported but carrying two years of undocumented local change.

DoAgree now that the target is a rebuild on a current Ubuntu LTS with data restored into it, not a disk image. One check gates the S4 target release, and it waits for clearance because it reads the source host: which PHP version the archive vhost uses — four are installed and the command-line default is 8.0.5. That check reads the source host, so it waits for clearance like everything else on S4. If the vhost needs 7.2 or 7.3, a current release may not offer it.

High

The V4 databases run directly on the host, not in containers — that changes how the copy is done

Owner: UNCTAD operations

FoundPostgreSQL 16.15, MongoDB 6.0.29 and Redis 6.0.16 are installed on the operating system, with /var/lib/postgresql and /var/lib/mongodb symlinked onto the big volume. Docker's own named volumes total 211.6 MB; the 777 GB of application data is bind-mounted from /opt/volumes.

DoInstall the databases natively on the target and restore with their own tools, or copy the directories cold with the services stopped. A copy driven by docker volume would move 211.6 MB and leave 777 GB behind.

High

S4's root disk is 93% full, which constrains how the extraction can be staged

Owner: investigation owner, then UNCTAD operations

Found226 GB total, 200 GB used, 18 GB free, no swap. The 8 GB that does not add up is ext4 reserved space, and the 93% is measured against the usable part rather than the raw 226 GB. The obvious space to reclaim is the 39 GB of retained nightly dumps — but it sits on the quarantined host.

DoOnce the investigators clear it, move those dumps to the backup target and re-measure. Until then the planning can be done on paper, and nothing at all is written on the host. When the extraction is authorised, stage nothing on /: /mnt/volume-bj has 209 GB free as measured by df, which a full ~790 GB extract does not fit into either, so streaming straight off the host over the approved channel is the primary option and any staging there can only be partial.

Medium

Clean up the V4 data before sizing the disk — it is the difference between an 8 TB and a 4 TB purchase

Owner: UNCTAD operations, decision by Benin IT

FoundAbout 1.4 TB of the 2.1 TB footprint is Camunda process history, OpenSearch log indices, stale SQL dumps and an oversized ActiveMQ store. Section 2 sets this out in full.

DoRun Camunda history cleanup on the source, agree an OpenSearch and Graylog retention policy, confirm and delete the three stale dumps, and investigate the ActiveMQ backlog. Then re-measure, and order the disk from the new figure. The 8 TB recommendation assumes none of this happens.

Medium

V4 is under memory pressure today — do not size the new hardware below the old, and raise the mule limit

Owner: UNCTAD operations

Found13 to 14 GiB of swap in use on a 128 GB machine, and four mule containers — the integration service, glossed in section 2 — pinned at their 4 GiB ceiling with repeated exit-137 kills.

DoSpecify 256 GB on the new host. Separately, raise the container limit — 6 GiB is a reasonable first step — then observe the new steady state and set the final value from it. This is a configuration change on the current machine, not a purchase.

Medium

Network controls, the site link and external dependencies have to be rebuilt deliberately on-premises

Owner: Benin IT

FoundOn V4 the host firewall is the only packet filter, and management ports are open only to named external addresses. The platform also calls four external services: translations.eregistrations.org, warmer.eregistrations.org, benin.eregistrations.org and archive.monentreprise.bj.

DoRebuild the allow-lists from section 6 on day one, confirm each external dependency is still required and reachable from the new site, and procure the site internet link early: at least 1 Gbit/s symmetric, per section 2. Bandwidth takes longer to obtain than servers.

Medium

Agree what on the S4 machine is in scope, and leave the rest behind

Owner: Benin IT decides, UNCTAD operations lists

FoundBesides the archive, the box runs mail, FTP, bind DNS, Grafana, an Ollama model server and five Docker containers, and the subscription home holds build tooling and working directories the archive application does not require: .cargo, .bun, bison-3.8.2, check_trade_name, ASIN-REQUEST.

DoDraw up an explicit in-scope list and have the counterpart confirm what is wanted. Anything not on it is excluded from the copy. No judgement is implied about how the machine has been run; this is only about what the new one needs to carry.

Medium

TLS and DNS cutover needs its own plan, with a certificate that renews without Hetzner

Owner: Benin IT with UNCTAD operations

FoundThe V4 wildcard certificate for *.monentreprise.bj expires 2026-12-07 and is renewed by certbot using DNS-01 with the Cloudflare plugin, read by HAProxy. The monentreprise.bj zone is delegated to Hetzner DNS.

DoMove the renewal automation to the new host, rotate the Cloudflare API credentials at handover, and prove one successful renewal on the new machine before any DNS record is switched.

Medium

Settle who owns the cloud account holding the S4 source server, and who decommissions it

Owner: UNCTAD operations with Benin IT

FoundThe archive server sits in a Hetzner Cloud project separate from Benin's, so account access and billing are not where the rest of the estate is.

DoAgree in writing who pays for and controls that account during the migration, how long the source stays running after Benin accepts the rebuilt archive, and who decommissions it — which cannot happen until the investigation releases the host.

5 · Method: how each system is copied

Method only. The findings behind it are in section 4 and the specifications in sections 2 and 3.

No disk images, on either machine. On S4 an image would copy other tenants' data and transplant a machine under investigation. On both it carries the old operating system and years of undocumented change forward. On V4 a block-level copy of a running database directory will not start cleanly. Owner: Benin IT with UNCTAD operations — a decision, not a task
V4 — rebuild the platform. Install a current Ubuntu LTS, then Docker Engine with a single-node swarm matching today's shape: 28 services and 37 containers under the stack name eregistrations. The stack definition is 38 MB, so the platform is quick to stand up. The data is what takes time. Owner: Benin IT — builds the target
V4 — install the databases natively and restore with their own tools. PostgreSQL 16, MongoDB 6.0 and Redis 6 on the operating system, not in containers. Restore with pg_restore and mongorestore, or stop both services and copy cold. Redis holds only cache and can be rebuilt empty. Re-create the two symlinks onto the large volume, or point the data directories at it. Owner: Benin IT receives and restores; UNCTAD operations takes the dumps on the source
V4 — copy /opt/volumes as a filesystem tree. That is the 777 GB of bind-mounted application data. Use a resumable file-level copy with a second delta pass, and verify counts and sizes per subdirectory. Do not rely on the offsite MinIO bucket: that mirror is failing. Owner: UNCTAD operations sends from the source; Benin IT receives and verifies
V4 — do the cleanup before the copy, not after. Camunda history cleanup, retention policy, stale dumps, ActiveMQ backlog. Verify the saving on a copy first, then re-measure and size the target from the new figure. Owner: UNCTAD operations runs it on the source; Benin IT decides on the result
S4 — only after written clearance. Once the investigators clear the host: take a Plesk subscription export for the Benin domains, a fresh logical mysqldump of ereg_benin_prod plus whichever dev, test and migration schemas are wanted, and the two Benin directories under /mnt/volume-bj. The same clearance covers the whole step: apply the four-entry exclusion list from blocker 2, signed by both sides before the cleared export begins, and stage nothing on /. Owner: UNCTAD operations exports, Benin IT countersigns the list — as in blocker 2
S4 — transfer channel is decided by the investigators, not by convenience. Either a temporary single-purpose firewall rule that is opened, used, closed and re-verified byte-identical against the quarantine, or physical media they have inspected. Decide this with them before any technical work starts. Owner: investigation owner decides; UNCTAD operations executes
S4 — rebuild target. A current Ubuntu LTS running the same application stack — nginx in front of Apache, PHP, MariaDB 10.5 — with or without Plesk, as the counterpart prefers, and carrying only what the in-scope list names. Owner: Benin IT — builds the target
Order of operations. One: clearance and the agreed S4 export list. Two: rotate the V4 credential and fix the V4 offsite backup, so the rebuild starts from a system that can be recovered. Three: the V4 cleanup and re-measure, so the hardware order is right. Four: build the targets and restore into them. Five: run both in parallel and have Benin accept them. Only then switch DNS, and only then decommission — keeping the S4 source untouched for as long as the investigation requires. Owner: Benin IT with UNCTAD operations — sequencing is joint

6 · Network ports to open

What the rebuilt machines should allow, taken from what the live ones answer on today.

System	Direction and who from	Port	Purpose
V4	Inbound from the internet	443 · 80	HTTPS, terminated by HAProxy; port 80 redirects and issues certificates
V4	Inbound, administrators only	22	SSH administration; restrict to named networks
V4	Inbound, UNCTAD monitoring only (94.130.229.64)	10050 · 9200 · 8444	Health and log monitoring, restricted by source address
V4	Inbound, two named peers (78.47.114.185, 2a01:4f8:c17:2968::2), plus one forwarded peer (88.99.186.246)	61613	ActiveMQ STOMP. Three source addresses in all: two allowed inbound, one allowed through the forward chain. An allow-list rebuilt from the first two only would drop the third.
V4	Inbound, two named peers only (138.201.157.227, 94.130.229.64)	6009	Forwarded application port
V4	Internal only — must stay closed	5432 · 6379 · 27017 · 1337–1338 · 3001–3004 · 4200–4205 · 6009–6040 · 8080–8180 · 12201 · 61616	PostgreSQL, Redis, MongoDB and the platform's own services. They listen on all interfaces and are safe only because the host firewall blocks them. Never expose a database port to the internet.
V4	Internal, between swarm nodes	4789/udp · 7946/tcp+udp	Docker Swarm overlay networking and gossip
V4	Outbound	443 · 80	Images, updates, certificates and the four external dependencies named in section 4
S4	Inbound, a Benin network only (137.255.8.0/22)	80 · 443	The archive itself. The source host already limits web access to this range — start the rebuilt machine from the same list.
S4	Inbound, administrators only	22	SSH administration; restrict to named networks
S4	Only if Benin keeps mail on this machine	25 · 587 · 465 · 110 · 143 · 993 · 995 · 4190	Postfix and Dovecot, eight ports. Needed only if the archive machine keeps its own mail service; otherwise relay through an existing service and open none of these.
S4	Only if Benin keeps FTP or DNS on this machine	21 · 53	FTP and bind. Neither is needed by the archive application itself.
S4	Internal only — must stay closed	11434 · 3306	The Ollama model server was observed on the source bound to `127.0.0.1:11434`; on the rebuilt machine, bind it the same way or leave the service out altogether. 3306 is MariaDB's default port and was not verified on the source; confirm the listener on the rebuilt machine — or, only once the investigators have cleared the source host, on the source — before writing the rule.

While the quarantine stands, none of this applies to S4. The table describes what the rebuilt on-premises machine should allow, not what the source machine allows today.

If this page is published openly rather than handed to the counterpart, replace the external address literals above with role labels — "UNCTAD monitoring", "two named ActiveMQ peers" — and pass the addresses in the accompanying transfer note. They are useful to whoever rebuilds the firewall and equally useful to anyone probing it.

7 · DNS names

Nine names under monentreprise.bj point at the V4 machine and share one wildcard certificate. The apex is the exception, and it is the one that matters most to citizens.

DNS name	Resolves to	Used by	What it serves
`services.monentreprise.bj`	49.13.197.27	Applicants and officers	The Single Window itself: applications are filed and processed here
`login.monentreprise.bj`	49.13.197.27	Everyone signing in	Sign-in and accounts
`admin-home` · `admin-home-new` · `home-new`	49.13.197.27	Communication staff	Editing the public portal's content. Three names on the same machine — confirm which are still in use before cutover.
`bpa.monentreprise.bj`	49.13.197.27	Service designers	Configuring the registration services, forms and workflows
`gdb.monentreprise.bj`	49.13.197.27	Registry staff	The registry database: records created by the services
`stats.monentreprise.bj`	49.13.197.27	Managers	Statistics and performance dashboards
`graylog.monentreprise.bj`	49.13.197.27	The IT team	System logs, for troubleshooting
`monentreprise.bj` and `www`	37.27.8.11	The public	A separate Next.js host, not the V4 machine. The portal citizens land on. Not reachable with the available credentials, not inspected, and not included in a copy of the V4 server — see blocker 3.
`archive.monentreprise.bj`	78.46.197.71	Staff consulting old records	The S4 legacy archive, on the shared Plesk machine. Under quarantine since 2026-09-16.

The apex is the trap in this handover. Nine names move with the V4 machine; monentreprise.bj and www do not. The zone is delegated to Hetzner DNS — ns1.first-ns.de, robotns2.second-ns.de, robotns3.second-ns.com — so records change there, not on the S4 machine, whose own bind zones are vestigial copies.

Appendix · Where these numbers come from

Measured by SSH on 2026-09-18, on the live machines. services.monentreprise.bj (Hetzner ccx53) and archive.monentreprise.bj (Hetzner cpx51).
Measured and recommended are kept apart deliberately. Everything in the two "what was measured" lists is an observation from the running machines. Every machine size in sections 2 and 3 is labelled either like-for-like with what runs today or as a recommendation, with the reasoning on the same line.
Load and memory anchor the processor and RAM sizing. V4: 13.13 / 12.01 / 11.25 across 32 threads, and 13 GiB of a 31 GiB swap area in use. S4: 1.00 / 1.01 / 1.00 on 16 shared vCPU after 325 days, 14 GiB of 30 GiB used, no swap.
Storage anchors everything else, and V4's figure is a range. /opt read 2.9 TB at 11:08 UTC and 2.1 TB at 12:36 UTC the same morning; the measured cause is the nightly pg_basebackup staging a full second copy of the database and then removing it. Plan against 2.1–2.9 TB and size the volume to hold the data plus that transient copy.
The growth rate is an average, not a trend. /opt was created 2024-03-28 and held 2.1 TB on 2026-09-18 — 29.7 months, about 72 GB per month. Data may have been migrated in at the start, so treat it as an upper bound on average accumulation and plan with headroom rather than precision. On S4, the document volume was created 2021-04-09 and holds 725 GB: about 130 GB a year.
Two credentials were found in clear text and are not reproduced anywhere. One in the S4 root crontab, one in the V4 maintenance.sh. Only their locations are recorded, so they can be rotated.
What this page deliberately omits. The address the S4 quarantine permits, and the attribution of the incident. The firewall's identity and the fact of the isolation carry the operational meaning; the address is handed to the transfer engineer separately if it is ever needed.

What happens next — yours first

Move 1 is Benin's and can start today. The two that gate everything else sit on UNCTAD's side of the fence.

Move 1 · Benin

Order the four machines whose sizes are settled

The V4 platform host, the V4 backup target, archive-bj-app and archive-bj-backup — sections 2 and 3 give the specifications. Hold only the V4 platform disk, until move 3 re-measures: that one line is the difference between 8 TB and about 4 TB. The fifth machine, the public portal host, waits on move 4. Ordering is not copying, so none of this touches the quarantined archive.

Owner: Benin IT, specifications from UNCTAD operations

By:

Move 2

Get written clearance for the S4 archive

Name the investigation contact, agree the export list and the transfer window in writing. Until this exists, nothing on the archive may be copied, deleted or staged — including the 39 GB of dumps filling its root disk.

Owner: investigation owner, requested by UNCTAD operations

By:

Move 3

Clean up V4, then re-measure before ordering disks

Camunda history cleanup, a log retention policy, the three stale dumps, the ActiveMQ backlog. This is what decides whether the held disk in move 1 is 8 TB or about 4 TB — and it also fixes the failing offsite mirror on the way past.

Owner: UNCTAD operations, decision by Benin IT

By:

Move 4

Confirm the public portal and the site link

Find out who operates 37.27.8.11 and whether it moves with this handover — that answer releases or cancels the fifth machine — and start procuring at least 1 Gbit/s symmetric at the on-premises site. Both take longer than buying servers.

Owner: Benin IT, portal operator identified by UNCTAD operations

By: