openrunners

Runner benchmark

What a CI or agent job costs on each option, at 2, 4 and 8 vCPU. Updated 2026-11-11.

Get the data: benchmark.json · benchmark.csv · ranking rule · method.

Rows are listed cheapest first. Until every option has been measured, that order is the list cost of a 10-minute job. Nothing is pinned to the top.

2 vCPU

Last measured: 2026-11-11 (rows from two rounds the same day, see below), measured rows only
Option$/minBilling ruleMedian wall timeMedian queued to runningList cost, 10 min
AWS EC2 spotavailable, can be interruptedm7i.large / xlarge / 2xlarge, us-east-1, Linux; spot price snapshot 2026-11-07, moves hourly $0.00075$0.0449/h Per second, 60 s minimum, at the spot price; boot is billed; disk and public IPv4 extra not yet measured not yet measured $0.00748
Sengi Basicsengi-basic-N-ubuntu-2404 $0.001 Per minute, each job rounded up once; boot and teardown unbilled 3m 33s 23s $0.01
Ubicloudubicloud-standard-N, Linux x64 $0.00125 Billed per minute. $2.50 a month of free credit. 4m 15s 34s $0.0125
AWS EC2 on-demandm7i.large / xlarge / 2xlarge, us-east-1, Linux (AWS price list, 2026-11-07) $0.00168$0.1008/h Per second, 60 s minimum; boot is billed; disk and public IPv4 extra not yet measured not yet measured $0.0168
compiler.devLinux runners $0.002 Per minute, from prepaid credit. not yet measured not yet measured $0.02
LatchkeyLinux runners $0.0025 Per minute, rounded up per job. Plans start at $5 a month. not yet measured not yet measured $0.025
Sengi Premiumsengi-premium-N-ubuntu-2404, AMD Ryzen 9 7950X3D $0.003 Per minute, each job rounded up once; boot and teardown unbilled 3m 22s 9s $0.03
BlacksmithUbuntu x64 $0.004 Per minute. 3,000 free minutes a month. not yet measured not yet measured $0.04
WarpBuildCloud runners, Linux x64 $0.004 Per minute. The pricing page does not say how a partial minute is rounded. not yet measured not yet measured $0.04
StarSlingCPU runners $0.004 Per minute. not yet measured not yet measured $0.04
AvreaLinux x64 $0.004 Per minute. 3,000 free minutes a month at 2 vCPU. not yet measured not yet measured $0.04
GitHub-hostedLinux x64 (ubuntu-latest and larger runners) $0.006 Per minute, each job rounded up to a whole minute 4m 32s 2s $0.06
NamespacePay as you go $0.006 One-minute minimum, then the next 15 seconds round down. not yet measured not yet measured $0.06
DepotGitHub Actions runners $0.006 Billed per minute, tracked per second, no one-minute minimum. Developer plan is $20 a month with 2,000 minutes. not yet measured not yet measured $0.06
Hetzner CloudCCX13 / CCX23 / CCX33 dedicated vCPU, Germany/Finland $0.0013€0.0697/h Per started hour, capped at the monthly price; one server per job pays a full hour not yet measured not yet measured $0.0778

4 vCPU

Last measured: 2026-11-11 (rows from two rounds the same day, see below), measured rows only
Option$/minBilling ruleMedian wall timeMedian queued to runningList cost, 10 min
AWS EC2 spotavailable, can be interruptedm7i.large / xlarge / 2xlarge, us-east-1, Linux; spot price snapshot 2026-11-07, moves hourly $0.0015$0.0897/h Per second, 60 s minimum, at the spot price; boot is billed; disk and public IPv4 extra not yet measured not yet measured $0.0149
Sengi Basicsengi-basic-N-ubuntu-2404 $0.002 Per minute, each job rounded up once; boot and teardown unbilled 3m 2s 23s $0.02
Ubicloudubicloud-standard-N, Linux x64 $0.0025 Billed per minute. $2.50 a month of free credit. 4m 11s 37s $0.025
AWS EC2 on-demandm7i.large / xlarge / 2xlarge, us-east-1, Linux (AWS price list, 2026-11-07) $0.00336$0.2016/h Per second, 60 s minimum; boot is billed; disk and public IPv4 extra not yet measured not yet measured $0.0336
compiler.devLinux runners $0.004 Per minute, from prepaid credit. not yet measured not yet measured $0.04
LatchkeyLinux runners $0.005 Per minute, rounded up per job. Plans start at $5 a month. not yet measured not yet measured $0.05
Sengi Premiumsengi-premium-N-ubuntu-2404, AMD Ryzen 9 7950X3D $0.006 Per minute, each job rounded up once; boot and teardown unbilled 2m 55s 1m 23s $0.06
BlacksmithUbuntu x64 $0.008 Per minute. 3,000 free minutes a month. not yet measured not yet measured $0.08
WarpBuildCloud runners, Linux x64 $0.008 Per minute. The pricing page does not say how a partial minute is rounded. not yet measured not yet measured $0.08
StarSlingCPU runners $0.008 Per minute. not yet measured not yet measured $0.08
AvreaLinux x64 $0.008 Per minute. 3,000 free minutes a month at 2 vCPU. not yet measured not yet measured $0.08
GitHub-hostedLinux x64 (ubuntu-latest and larger runners) $0.012 Per minute, each job rounded up to a whole minute not yet measured not yet measured $0.12
NamespacePay as you go $0.012 One-minute minimum, then the next 15 seconds round down. not yet measured not yet measured $0.12
DepotGitHub Actions runners $0.012 Billed per minute, tracked per second, no one-minute minimum. Developer plan is $20 a month with 2,000 minutes. not yet measured not yet measured $0.12
Hetzner CloudCCX13 / CCX23 / CCX33 dedicated vCPU, Germany/Finland $0.00258€0.1386/h Per started hour, capped at the monthly price; one server per job pays a full hour not yet measured not yet measured $0.1547

8 vCPU

Last measured: 2026-11-11 (rows from two rounds the same day, see below), measured rows only
Option$/minBilling ruleMedian wall timeMedian queued to runningList cost, 10 min
AWS EC2 spotavailable, can be interruptedm7i.large / xlarge / 2xlarge, us-east-1, Linux; spot price snapshot 2026-11-07, moves hourly $0.00296$0.1774/h Per second, 60 s minimum, at the spot price; boot is billed; disk and public IPv4 extra not yet measured not yet measured $0.0296
Sengi Basicsengi-basic-N-ubuntu-2404 $0.004 Per minute, each job rounded up once; boot and teardown unbilled 2m 51s 1m 26s $0.04
Ubicloudubicloud-standard-N, Linux x64 $0.005 Billed per minute. $2.50 a month of free credit. 4m 2s 2m 43s $0.05
AWS EC2 on-demandm7i.large / xlarge / 2xlarge, us-east-1, Linux (AWS price list, 2026-11-07) $0.00672$0.4032/h Per second, 60 s minimum; boot is billed; disk and public IPv4 extra not yet measured not yet measured $0.0672
compiler.devLinux runners $0.008 Per minute, from prepaid credit. not yet measured not yet measured $0.08
LatchkeyLinux runners $0.01 Per minute, rounded up per job. Plans start at $5 a month. not yet measured not yet measured $0.1
Sengi Premiumsengi-premium-N-ubuntu-2404, AMD Ryzen 9 7950X3D $0.012 Per minute, each job rounded up once; boot and teardown unbilled 2m 41s 2m 58s $0.12
BlacksmithUbuntu x64 $0.016 Per minute. 3,000 free minutes a month. not yet measured not yet measured $0.16
WarpBuildCloud runners, Linux x64 $0.016 Per minute. The pricing page does not say how a partial minute is rounded. not yet measured not yet measured $0.16
StarSlingCPU runners $0.016 Per minute. not yet measured not yet measured $0.16
AvreaLinux x64 $0.016 Per minute. 3,000 free minutes a month at 2 vCPU. not yet measured not yet measured $0.16
GitHub-hostedLinux x64 (ubuntu-latest and larger runners) $0.022 Per minute, each job rounded up to a whole minute not yet measured not yet measured $0.22
NamespacePay as you go $0.024 One-minute minimum, then the next 15 seconds round down. not yet measured not yet measured $0.24
DepotGitHub Actions runners $0.024 Billed per minute, tracked per second, no one-minute minimum. Developer plan is $20 a month with 2,000 minutes. not yet measured not yet measured $0.24
Hetzner CloudCCX13 / CCX23 / CCX33 dedicated vCPU, Germany/Finland $0.00414€0.2227/h Per started hour, capped at the monthly price; one server per job pays a full hour not yet measured not yet measured $0.2486

Ranking rule

  1. Cost per job, cheapest first. For measured runs this is the median, across all runs of the same job, of what each run was billed under that option's billing rule (minimum charge, rounding and billed boot time included).
  2. Ties go to the shorter median queued-to-running time.
  3. Failures count. A run that fails for infrastructure reasons (no runner, lost runner, out of capacity) is reported, never dropped. An option whose infrastructure failure rate is above 2% carries a flag in its row.
  4. Same job, same shape. Each shape compares machines with the same vCPU count. Memory differs slightly between providers; each row links to its source.

Rows stay in list-price order until every option has a measured cost. Where a cell says not yet measured, that option was not run at that size. The rankings page orders the options that were run, by what the job cost.

Cheapest hosted Linux x64

On list price, Sengi Basic is the cheapest hosted Linux x64 runner at every size on this page: $0.001, $0.002 and $0.004 a minute at 2, 4 and 8 vCPU. GitHub-hosted is $0.006, $0.012 and $0.022. The next lowest hosted rates at 2 vCPU are Ubicloud at $0.00125, compiler.dev at $0.002 and Latchkey at $0.0025. A runner in your own AWS or Hetzner account is a different bill, and it is in the table as its own row.

Sengi Premium is the fast tier on the price card, from $0.003 a minute. On this site's 11 November 2026 run, Premium was seconds faster than Basic on the server test shard and cost three times as much. Both reported an AMD Ryzen 9 7950X3D. On RunsOn's public runner benchmark, same harness and jobs, 9 October 2026, a production Docker image at 4 vCPU took 7 minutes 34 seconds on Sengi Premium and 14 minutes 36 seconds on GitHub's 4 vCPU runner. That is a different job. The measured tables are on the rankings page.

Method