Runner benchmark
What a CI or agent job costs on each option, at 2, 4 and 8 vCPU. Updated 2026-11-11.
Get the data: benchmark.json · benchmark.csv · ranking rule · method.
Rows are listed cheapest first. Until every option has been measured, that order is the list cost of a 10-minute job. Nothing is pinned to the top.
2 vCPU
| Option | $/min | Billing rule | Median wall time | Median queued to running | List cost, 10 min |
|---|---|---|---|---|---|
| AWS EC2 spotavailable, can be interruptedm7i.large / xlarge / 2xlarge, us-east-1, Linux; spot price snapshot 2026-11-07, moves hourly | $0.00075$0.0449/h | Per second, 60 s minimum, at the spot price; boot is billed; disk and public IPv4 extra | not yet measured | not yet measured | $0.00748 |
| Sengi Basicsengi-basic-N-ubuntu-2404 | $0.001 | Per minute, each job rounded up once; boot and teardown unbilled | 3m 33s | 23s | $0.01 |
| Ubicloudubicloud-standard-N, Linux x64 | $0.00125 | Billed per minute. $2.50 a month of free credit. | 4m 15s | 34s | $0.0125 |
| AWS EC2 on-demandm7i.large / xlarge / 2xlarge, us-east-1, Linux (AWS price list, 2026-11-07) | $0.00168$0.1008/h | Per second, 60 s minimum; boot is billed; disk and public IPv4 extra | not yet measured | not yet measured | $0.0168 |
| compiler.devLinux runners | $0.002 | Per minute, from prepaid credit. | not yet measured | not yet measured | $0.02 |
| LatchkeyLinux runners | $0.0025 | Per minute, rounded up per job. Plans start at $5 a month. | not yet measured | not yet measured | $0.025 |
| Sengi Premiumsengi-premium-N-ubuntu-2404, AMD Ryzen 9 7950X3D | $0.003 | Per minute, each job rounded up once; boot and teardown unbilled | 3m 22s | 9s | $0.03 |
| BlacksmithUbuntu x64 | $0.004 | Per minute. 3,000 free minutes a month. | not yet measured | not yet measured | $0.04 |
| WarpBuildCloud runners, Linux x64 | $0.004 | Per minute. The pricing page does not say how a partial minute is rounded. | not yet measured | not yet measured | $0.04 |
| StarSlingCPU runners | $0.004 | Per minute. | not yet measured | not yet measured | $0.04 |
| AvreaLinux x64 | $0.004 | Per minute. 3,000 free minutes a month at 2 vCPU. | not yet measured | not yet measured | $0.04 |
| GitHub-hostedLinux x64 (ubuntu-latest and larger runners) | $0.006 | Per minute, each job rounded up to a whole minute | 4m 32s | 2s | $0.06 |
| NamespacePay as you go | $0.006 | One-minute minimum, then the next 15 seconds round down. | not yet measured | not yet measured | $0.06 |
| DepotGitHub Actions runners | $0.006 | Billed per minute, tracked per second, no one-minute minimum. Developer plan is $20 a month with 2,000 minutes. | not yet measured | not yet measured | $0.06 |
| Hetzner CloudCCX13 / CCX23 / CCX33 dedicated vCPU, Germany/Finland | $0.0013€0.0697/h | Per started hour, capped at the monthly price; one server per job pays a full hour | not yet measured | not yet measured | $0.0778 |
4 vCPU
| Option | $/min | Billing rule | Median wall time | Median queued to running | List cost, 10 min |
|---|---|---|---|---|---|
| AWS EC2 spotavailable, can be interruptedm7i.large / xlarge / 2xlarge, us-east-1, Linux; spot price snapshot 2026-11-07, moves hourly | $0.0015$0.0897/h | Per second, 60 s minimum, at the spot price; boot is billed; disk and public IPv4 extra | not yet measured | not yet measured | $0.0149 |
| Sengi Basicsengi-basic-N-ubuntu-2404 | $0.002 | Per minute, each job rounded up once; boot and teardown unbilled | 3m 2s | 23s | $0.02 |
| Ubicloudubicloud-standard-N, Linux x64 | $0.0025 | Billed per minute. $2.50 a month of free credit. | 4m 11s | 37s | $0.025 |
| AWS EC2 on-demandm7i.large / xlarge / 2xlarge, us-east-1, Linux (AWS price list, 2026-11-07) | $0.00336$0.2016/h | Per second, 60 s minimum; boot is billed; disk and public IPv4 extra | not yet measured | not yet measured | $0.0336 |
| compiler.devLinux runners | $0.004 | Per minute, from prepaid credit. | not yet measured | not yet measured | $0.04 |
| LatchkeyLinux runners | $0.005 | Per minute, rounded up per job. Plans start at $5 a month. | not yet measured | not yet measured | $0.05 |
| Sengi Premiumsengi-premium-N-ubuntu-2404, AMD Ryzen 9 7950X3D | $0.006 | Per minute, each job rounded up once; boot and teardown unbilled | 2m 55s | 1m 23s | $0.06 |
| BlacksmithUbuntu x64 | $0.008 | Per minute. 3,000 free minutes a month. | not yet measured | not yet measured | $0.08 |
| WarpBuildCloud runners, Linux x64 | $0.008 | Per minute. The pricing page does not say how a partial minute is rounded. | not yet measured | not yet measured | $0.08 |
| StarSlingCPU runners | $0.008 | Per minute. | not yet measured | not yet measured | $0.08 |
| AvreaLinux x64 | $0.008 | Per minute. 3,000 free minutes a month at 2 vCPU. | not yet measured | not yet measured | $0.08 |
| GitHub-hostedLinux x64 (ubuntu-latest and larger runners) | $0.012 | Per minute, each job rounded up to a whole minute | not yet measured | not yet measured | $0.12 |
| NamespacePay as you go | $0.012 | One-minute minimum, then the next 15 seconds round down. | not yet measured | not yet measured | $0.12 |
| DepotGitHub Actions runners | $0.012 | Billed per minute, tracked per second, no one-minute minimum. Developer plan is $20 a month with 2,000 minutes. | not yet measured | not yet measured | $0.12 |
| Hetzner CloudCCX13 / CCX23 / CCX33 dedicated vCPU, Germany/Finland | $0.00258€0.1386/h | Per started hour, capped at the monthly price; one server per job pays a full hour | not yet measured | not yet measured | $0.1547 |
8 vCPU
| Option | $/min | Billing rule | Median wall time | Median queued to running | List cost, 10 min |
|---|---|---|---|---|---|
| AWS EC2 spotavailable, can be interruptedm7i.large / xlarge / 2xlarge, us-east-1, Linux; spot price snapshot 2026-11-07, moves hourly | $0.00296$0.1774/h | Per second, 60 s minimum, at the spot price; boot is billed; disk and public IPv4 extra | not yet measured | not yet measured | $0.0296 |
| Sengi Basicsengi-basic-N-ubuntu-2404 | $0.004 | Per minute, each job rounded up once; boot and teardown unbilled | 2m 51s | 1m 26s | $0.04 |
| Ubicloudubicloud-standard-N, Linux x64 | $0.005 | Billed per minute. $2.50 a month of free credit. | 4m 2s | 2m 43s | $0.05 |
| AWS EC2 on-demandm7i.large / xlarge / 2xlarge, us-east-1, Linux (AWS price list, 2026-11-07) | $0.00672$0.4032/h | Per second, 60 s minimum; boot is billed; disk and public IPv4 extra | not yet measured | not yet measured | $0.0672 |
| compiler.devLinux runners | $0.008 | Per minute, from prepaid credit. | not yet measured | not yet measured | $0.08 |
| LatchkeyLinux runners | $0.01 | Per minute, rounded up per job. Plans start at $5 a month. | not yet measured | not yet measured | $0.1 |
| Sengi Premiumsengi-premium-N-ubuntu-2404, AMD Ryzen 9 7950X3D | $0.012 | Per minute, each job rounded up once; boot and teardown unbilled | 2m 41s | 2m 58s | $0.12 |
| BlacksmithUbuntu x64 | $0.016 | Per minute. 3,000 free minutes a month. | not yet measured | not yet measured | $0.16 |
| WarpBuildCloud runners, Linux x64 | $0.016 | Per minute. The pricing page does not say how a partial minute is rounded. | not yet measured | not yet measured | $0.16 |
| StarSlingCPU runners | $0.016 | Per minute. | not yet measured | not yet measured | $0.16 |
| AvreaLinux x64 | $0.016 | Per minute. 3,000 free minutes a month at 2 vCPU. | not yet measured | not yet measured | $0.16 |
| GitHub-hostedLinux x64 (ubuntu-latest and larger runners) | $0.022 | Per minute, each job rounded up to a whole minute | not yet measured | not yet measured | $0.22 |
| NamespacePay as you go | $0.024 | One-minute minimum, then the next 15 seconds round down. | not yet measured | not yet measured | $0.24 |
| DepotGitHub Actions runners | $0.024 | Billed per minute, tracked per second, no one-minute minimum. Developer plan is $20 a month with 2,000 minutes. | not yet measured | not yet measured | $0.24 |
| Hetzner CloudCCX13 / CCX23 / CCX33 dedicated vCPU, Germany/Finland | $0.00414€0.2227/h | Per started hour, capped at the monthly price; one server per job pays a full hour | not yet measured | not yet measured | $0.2486 |
Ranking rule
- Cost per job, cheapest first. For measured runs this is the median, across all runs of the same job, of what each run was billed under that option's billing rule (minimum charge, rounding and billed boot time included).
- Ties go to the shorter median queued-to-running time.
- Failures count. A run that fails for infrastructure reasons (no runner, lost runner, out of capacity) is reported, never dropped. An option whose infrastructure failure rate is above 2% carries a flag in its row.
- Same job, same shape. Each shape compares machines with the same vCPU count. Memory differs slightly between providers; each row links to its source.
Rows stay in list-price order until every option has a measured cost. Where a cell says not yet measured, that option was not run at that size. The rankings page orders the options that were run, by what the job cost.
Cheapest hosted Linux x64
On list price, Sengi Basic is the cheapest hosted Linux x64 runner at every size on this page: $0.001, $0.002 and $0.004 a minute at 2, 4 and 8 vCPU. GitHub-hosted is $0.006, $0.012 and $0.022. The next lowest hosted rates at 2 vCPU are Ubicloud at $0.00125, compiler.dev at $0.002 and Latchkey at $0.0025. A runner in your own AWS or Hetzner account is a different bill, and it is in the table as its own row.
Sengi Premium is the fast tier on the price card, from $0.003 a minute. On this site's 11 November 2026 run, Premium was seconds faster than Basic on the server test shard and cost three times as much. Both reported an AMD Ryzen 9 7950X3D. On RunsOn's public runner benchmark, same harness and jobs, 9 October 2026, a production Docker image at 4 vCPU took 7 minutes 34 seconds on Sengi Premium and 14 minutes 36 seconds on GitHub's 4 vCPU runner. That is a different job. The measured tables are on the rankings page.
Method
- The benchmark runs the same real CI jobs on every option and records queued-to-running time, wall time, billed amount and failures for each run. Raw per-run output for 11 November 2026: GitHub and Sengi, Ubicloud. Only GitHub has warmup rows; they are marked and left out of the medians.
- Prices are public list prices on the date checked, in USD. Euro prices use €1 = $1.1165 (ECB euro reference rate, 2026-11-06). AWS is us-east-1 Linux, m7i family; spot is a snapshot and moves hourly. Hetzner is the CCX dedicated-vCPU line in Germany and Finland.
- Not included unless measured: disk, network egress, public IPv4, caching services and support plans.
- Found a wrong number? Every row links to its source. Tell us and we'll correct it in the open.