Benchmarks / Developer platforms
vs Vercel and Railway
The method for this board is published and dated. The board itself is empty, because we have not run it. Publishing the rules before anyone knows who wins is the only moment at which publishing them proves anything.
No results published
- Methodology published
- Boards published
- 0
- Runs completed
- 0
- Platforms compared
- 2
- Spec cohort
- C8
We do not claim a hardware advantage. This board compares purchasable plans, not silicon. Nothing here asserts that Olympia designs, owns or operates the machines behind any plan on it, and nothing here should be read as a claim about proprietary hardware. The unit of comparison is a plan, measured against a fixed spec cohort and normalized by the price you actually pay for it.
Who is on this board
Two platforms plus us. Vercel and Railway do not sell a machine with a spec on it — they sell an execution model, and you are billed for what your application does rather than for what you reserved. Olympia sells the machine. The cohort rule in the methodology below is what makes the three comparable at all, and the Limitations section is honest about how far that stretches.
Included
- Vercel
- Railway
- Olympia
Excluded, and why
- Akamai (Linode). Akamai’s terms prohibit performance testing of the service, so Linode cannot appear on any Olympia board.
- Any platform whose current terms forbid benchmarking or publication. We re-read each platform’s terms and record the clause and the date we checked before a board goes live. A platform we cannot test openly is listed as excluded, not tested quietly.
Why each of these is on the board
One paragraph on what each platform is genuinely good at, because a comparison that only lists what a competitor lacks is not a comparison. None of this is a measurement. Nothing on this page is, yet.
Vercel
Vercel’s deployment experience for frontend work is genuinely excellent and we have no equivalent to it. If your bill is comfortable and the workflow is saving your team real time, there is no argument here worth listening to.
Railway
Railway’s deploy experience is a genuine product, not a wrapper, and at low volume a metered platform can cost less than a flat monthly server. If that is where you are, staying put is the right call.
Olympia
We sell one of the plans that will appear on this board. That is a conflict of interest, it does not go away by being disclosed, and it is the reason the raw output archive is not optional. Read the archive rather than our summary of it.
No figure appears in any of the paragraphs above, on purpose. Prices live on the comparison pages linked from each entry, where a figure is either printed with the platform’s own price page as its source and the date it was checked, or replaced by the reason it could not be verified. On this board that reason is structural more often than not: usage billing has no single monthly number to sit next to a flat price, which is why the price column here is a model with its assumption published rather than a quote.
Methodology
Version 1, published . Changes are versioned and dated on this page, so you can check whether the method moved after the results came in.
01 — What is actually comparable
A platform sells an execution model, not a machine
You do not get a disk or a raw network port from a deploy platform, so a disk and network benchmark would be measuring an abstraction that was never sold to you. This board therefore drops most of the hardware tests and keeps the ones that describe the thing you are buying: the same application, deployed unchanged, serving the same requests, at a stated cost. That is a narrower comparison than the hyperscale board, and we would rather say so than pad the table.
02 — Spec cohort
Cohort C8: 8 vCPU, 16 GB RAM, 320 GB NVMe-class block storage
The same cohort as the hyperscale board, so the two pages describe the same size of thing. Linux baseline where we control it: Debian 13, stock kernel as shipped, no tuning beyond first boot. Scores are never compared across cohorts, and never across the two boards.
03 — Tier equivalence
The cheapest plan that meets or exceeds the cohort — or no row at all
Comparing our tier against a competitor’s premium tier, or their entry tier, is how these tables get rigged, so the plan is chosen by rule: at each platform, the lowest-priced plan or instance size that satisfies every dimension of the cohort. Where a platform does not publish a resource size at all, it does not get an approximated row — it is absent from the board for that cohort, and the board says which platforms are absent and why. We do not interpolate a plan that was never sold.
04 — What we run
Exact tools, exact command lines
One application image, byte-identical across every platform on the board, driven from a fixed external load generator outside all of them. All tools are open-source and license-clear for publishing results commercially.
# Environment capture, wherever the platform gives us a shell
lscpu; free -g; uname -a; cat /etc/os-release
# HTTP under load — the primary measurement on this board
oha -z 300s -c 100 --no-tui --latency-correction $TARGET_URL
wrk -t8 -c100 -d300s --latency $TARGET_URL # cross-check
# Cold path — first byte after a deliberate idle period
# run 30x, report the distribution, not the best case
for i in $(seq 1 30); do
sleep 900
curl -o /dev/null -s -w '%{time_starttransfer}\n' $TARGET_URL
done
# Database, only where the platform sells a Postgres meeting the cohort
pgbench -i -s 500 benchdb
pgbench -c 32 -j 8 -T 300 -M prepared --progress=30 benchdb
# CPU, only where the platform gives us a general-purpose runtime
sysbench cpu --cpu-max-prime=20000 --threads=$(nproc) --time=60 run
7z b -mmt$(nproc)
openssl speed -evp aes-256-gcm -seconds 10
# Key-value — Valkey, not redis-benchmark (see note below)
valkey-benchmark -t get,set -n 1000000 -c 50 -P 16 --csv
# NOT RUN on this board, and why:
# fio - no raw block device is sold, so the number would be fiction
# iperf3 - no raw port is sold; HTTP under load is the honest substitute
We use valkey-benchmark, not redis-benchmark. Redis moved to RSALv2/SSPL, which is why we do not offer Redis hosting at all. Valkey is BSD-licensed, is what we actually offer, and measures the same thing, so the license question simply does not arise.
If a Geekbench 6 figure ever appears here it will be run under a paid Geekbench 6 Pro license (US$99). The free tier is licensed for non-commercial use only and cannot be used to produce numbers for a company’s marketing page.
05 — Sampling
Three runs, three separately provisioned deployments
Each plan is measured on three deployments created independently and torn down between runs, in the same region, started at three different hours of the same UTC day. Three runs against one deployment measures that deployment and whoever it is sharing with; it does not measure the plan. The first run after deploy is discarded as warm-up. We report the median of the three deployment means, the standard error across deployments, and N — always in the same cell.
06 — The price column is a model, not a quote
Usage billing has no monthly price until you state the traffic
A usage-metered platform does not have a $/mo figure at all; it has a function of how much you use it. So the board publishes the traffic assumption — request count, response size, egress volume, build minutes, hours running — directly beside the number, and the column is labeled a modeled monthly cost at that stated traffic. It is not a quotation, it will not match your invoice, and changing the assumption changes the ranking. Change it and see: that is what the assumption is published for.
07 — Scoring
A cohort composite, then divided by modeled price
Score is the geometric mean of each test’s result normalized against the cohort median, multiplied by 100 — so the cohort median scores 100 by construction. It is a position within one cohort on one board, not an absolute capability, and it does not transfer to the hyperscale page. Points per $10/mo is Score divided by the modeled monthly cost in units of ten dollars. Price-normalized value inside a matched cohort is a claim that survives regardless of whose hardware is underneath, which is why it is the column that matters.
08 — Disclosure and recency
Raw output ships with the board; results expire
Every board publishes the full output of every run, the application image digest, the load-generator location, the region, the platform plan names as sold on the day, and UTC timestamps — including our own runs, including the ones that flatter us least. A result older than 180 days is marked stale; older than 270 days the row is removed rather than left to quietly age. Prices and pricing-model assumptions are re-checked from each platform’s public pricing page on the day of publication, dated and archived.
09 — Refusal to rank
When we will not print an ordering
If two plans’ medians fall within two standard errors of each other they share a rank, and the board says they are tied. We do not publish any row with N < 3. And because the price column here depends on an assumption, we additionally refuse to rank on price-normalized value if a plausible change to the traffic assumption reverses the order — in that case the board reports the crossover point instead of a winner.
Cohort C8 board
This is the table the results will land in, with its real columns, published now so the shape cannot be changed later to suit an outcome. The one row below shows the format of each cell. It contains no data.
| Rank | Provider & plan | Region | Score | ± SE (N) | $/mo | Points per $10/mo | Last run |
|---|---|---|---|---|---|---|---|
| Format example | Platform — plan name | region code | — | ± — (N = —) | $— | — | YYYY‑MM‑DD |
| No rows. This board has not been run. | |||||||
The $/mo column on this board is a modeled monthly cost at a published traffic assumption, not a quoted price. The assumption will be printed immediately beneath the table when there is a table to print it beneath.
Limitations
- We have not run these yet. Every table on this page is empty on purpose. There is no result, no partial result, and no internal number we are holding back.
- This board measures less than the hyperscale board. No disk test, no raw network test, because neither is sold to you. What is left is one application under load and its cost. Treat it as narrower evidence, not equivalent evidence.
- The price column depends on an assumption you may not share. Usage billing means the ranking is a function of traffic. If your traffic pattern differs from the published one, the ordering may differ too, and the crossover point matters more than the row order.
- Convenience is not on the board. These platforms sell a deployment experience, a preview-environment workflow and an operational burden of roughly zero. None of that is measurable here, none of it is worthless, and a table that ignores it is not the whole comparison.
- Platform behavior is not a constant. Regions, runtimes, and pricing models change on the platform’s schedule, not ours. Hence the 180-day stale mark and the 270-day removal.
- We are a party to this comparison. We ran it, we chose the cohort and the traffic assumption, and we sell one of the plans on it. That is exactly why the raw output and the assumptions are published — read those rather than our summary.
- No hardware claim is being made. Olympia has not decided whether it runs its own hardware or resells upstream capacity, and this board does not depend on the answer. It compares plans at their prices. Read nothing about silicon into it.
Both sides of it
The board above is empty, so it cannot help you decide anything today. This can. We sell the server, so the column on the left is the easy one to write. The column on the right is the one worth reading — if anything in it is a dealbreaker for you, it is a dealbreaker.
The case for
Why you’d pick us
- The bill stops tracking traffic. Usage billing means a good week and a bad week both show up on the invoice. A server is priced per server: flat, published on our configurator, and the same number whether nobody visits or everybody does.
- There is no platform runtime in the middle. Your application runs on a machine you have root on, against the same operating system and the same runtimes you would use anywhere else. Nothing has to be adapted to a proprietary execution model, and nothing has to be un-adapted to leave.
- You pick the region, and it stays picked. Twelve regions across six continents, all offered on the same footing. The data sits on a machine you control, in a country you chose.
- The method was published before the results existed. Everything above — the cohort, the commands, the traffic assumption, the archive rule — was written down and dated while the board was still empty. You can check later whether any of it moved once the numbers came in.
- We will do the move, and the quote costs nothing. Scoped in a conversation and quoted in writing before you commit to anything.
The case against
Why you wouldn’t
- You are giving up the thing these platforms actually sell. Push-to-deploy, a preview environment per pull request, instant rollback, and an operational burden of roughly zero. None of that is on the board above because none of it is measurable there, and none of it is worthless. On a server, that workflow is yours to build.
- You are taking on an operations job. We look after the server — patching, backups, uptime. What runs on it is yours: deployments, TLS, outbound mail, storage that grows, and periodically proving a backup actually restores. We do sell managed hosting where we operate the application too, but it is quoted per engagement and there is no published price for it you can read off a page today.
- We hold no compliance certification. No SOC 2, no ISO 27001, no completed third-party penetration test — our own security page says so in those words. If your procurement process needs a report, we cannot hand you one.
- There is no uptime SLA and no uptime history. We do not run automated uptime monitoring yet, which is why our status page carries no percentage and no incident log.
- At low volume, metered billing simply wins. A flat monthly price does not shrink when your traffic does. Below a certain volume the usage-billed platform is cheaper, and the crossover point matters far more than any row order on a board. Any comparison of ours that fails to say so is broken, so we are saying it here.
- Provisioning is manual, within 24 hours. Orders are fulfilled by a person, not an API. If you are used to a deploy completing while you read the pull request, that is not what we do.
- This board is empty, and an empty board proves nothing. A published method is a commitment, not evidence. Until cohort C8 has actually run, there is no measurement on this page to weigh against anything, and you should not treat the absence of one as a result in our favor.
- We are a party to this comparison. We chose the cohort, we chose the traffic assumption, and we sell one of the plans on it. Read the raw output archive and the assumptions rather than our summary of them.
What ships first
The hyperscale board at cohort C8 runs first. This board follows it, using the same cohort and the same application image so the two are readable side by side, published with its traffic assumption and its complete raw output archive.
There is no date attached to that, because the runs are not scheduled. A date invented to look organized would be the first unverified number on this page, and not doing that is the entire point of publishing the method before the results.