BasekickLabs

Hardware Sizing

Starting hardware sizes for Arc OSS ingestion, queries, and combined workloads, with WAL enabled or disabled.

Choose a starting size based on your write rate, query demand, and retention period. These guidelines cover native, single-node Arc OSS with SSD-backed storage, measured on AWS EC2 in us-east-1 on October 13–14, 2026. The reference build uses main commit 116a318f.

Starting sizes

Use these targets to begin your evaluation with WAL enabled. They reserve approximately 50% headroom below measured rates. The write and query columns describe separate workloads; use the combined table when both run together.

Instance or equivalentvCPURAMMessagePack points/secLine protocol points/secQuery-only QPS
m7i.xlarge416 GiB2.00M280.0k1.4
m7i.2xlarge832 GiB4.20M500.0k2.7
m7i.4xlarge1664 GiB4.00M950.0k5.4

Validate your workload

These are ballpark starting targets based on short benchmarks. Your schema, queries, and retained data volume determine the capacity you can sustain.

AWS vCPUs are logical CPU threads; these instances have two threads per physical core. Use SSD storage provisioned for 6,000 IOPS and 500 MiB/s for these reference write targets. Leave CPU, memory, and disk headroom for compaction.

Ingestion and WAL

MessagePack supports higher batched ingestion rates than line protocol in these measurements. Each point is one row containing a timestamp, a host tag, and three numeric fields.

InstanceWALMeasured MessagePack points/secMeasured line protocol points/sec
m7i.xlargeEnabled, shared disk4.02M561.4k
m7i.xlargeEnabled, separate disk4.44M561.3k
m7i.xlargeDisabled4.92M895.7k
m7i.2xlargeEnabled, shared disk8.43M1.01M
m7i.2xlargeEnabled, separate disk8.01M1.01M
m7i.2xlargeDisabled9.78M1.57M
m7i.4xlargeEnabled, shared disk8.15M1.90M
m7i.4xlargeEnabled, separate disk10.06M1.90M
m7i.4xlargeDisabled18.22M3.00M

These are measured operating points, not maximum capacities. Storage and network limits also apply when choosing a CPU size. Higher-rate shared-disk runs on the 16-vCPU server reached the volume's throughput limit while CPU had spare capacity; adding CPU alone does not remove that storage limit.

A separate WAL volume can reduce contention with data writes. The separate-disk measurements use the same 6,000 IOPS and 500 MiB/s per volume; both volumes still share the instance's total EBS bandwidth. These placement measurements cover ingestion only.

Keep WAL enabled when you rely on recovery of buffered writes. Disable it only when losing buffered writes after an abrupt failure is acceptable. For a WAL-disabled starting target, reserve the same 50% headroom below its measured ingestion rate. See Write-Ahead Log for configuration and recovery behavior.

Query capacity

Query demand often determines how much CPU you need. The following measured rates cover an aggregation query mix over 20 million rows, with WAL enabled. QPS means queries per second; p95 is the latency within which 95% of requests complete, reported for the slowest query type.

InstanceQPS, 10k hostsQPS, 100k hostsp95, 10k hosts
m7i.xlarge2.902.321.28s
m7i.2xlarge5.493.891.52s
m7i.4xlarge10.947.790.71s

More distinct hosts, larger scans, and more concurrent queries increase CPU and memory demand. Validate the queries your application actually runs, especially as retention grows.

Ingestion and queries together

For an evaluation ingesting approximately 100,000 MessagePack points/sec, the following combinations achieved their requested write and query rates. The dataset grew to 81.5 million rows during each run.

InstanceWALWrite points/secQuery QPSQuery p95
m7i.xlargeEnabled100.0k0.292.56s
m7i.xlargeDisabled100.0k0.292.55s
m7i.2xlargeEnabled100.0k0.551.38s
m7i.2xlargeDisabled100.0k0.541.37s
m7i.4xlargeEnabled100.0k1.090.72s
m7i.4xlargeDisabled100.0k1.090.72s

Choose a size that meets your application's latency requirement as well as its write rate. Higher query demand slowed responses and could not be sustained while ingestion continued. Leave additional CPU headroom for queries and for growth in retained data.

Storage and retention

Estimate storage from your write rate, retention period, and measured bytes per point:

retained data bytes = points/sec × 86,400 × retention days × bytes/point

For the measured three-field schema, a rounded 28 bytes per point gives these examples:

Write rateRetentionStored data
10,000 points/sec30 days~676 GiB
100,000 points/sec30 days~6.6 TiB

Add space for WAL, compaction output, temporary spill, and free disk headroom. Measure bytes per point with your own data: field count, types, values, and cardinality affect compression.

The benchmark used gp3 provisioned for 6,000 IOPS and 500 MiB/s. Check the instance's sustained storage limits as well as the volume's. The tested 4-, 8-, and 16-vCPU instances have EBS baseline bandwidths of approximately 156, 313, and 625 MB/s; smaller instances can burst above their baselines. Size for sustained throughput, including simultaneous WAL and data writes. Network baseline bandwidth is approximately 1.56, 3.13, and 6.25 Gbps for these sizes; short runs can benefit from burst capacity.

When to increase resources

  • CPU: increase it when queries consume ingestion headroom or exceed your latency target. Narrow query time ranges where possible.
  • RAM: increase it when process memory approaches available RAM or queries and compaction need larger working sets. Monitor native Arc process memory, including DuckDB and Arrow allocations.
  • Storage throughput: increase it when flushes fall behind, buffers grow, or disk wait rises. Check both volume and instance limits.
  • Storage capacity: increase it as retention grows, allowing space for compaction and operational headroom.

See Monitoring and File compaction for operational guidance.

Enterprise

These guidelines cover one OSS node. Enterprise sizing will cover cluster topology, replication, and storage separately.

Benchmark details

Ingestion and combined measurements run for ten minutes using batched synthetic data. Full workload definitions, configuration, limitations, and raw results are available in the benchmark report.

On this page