Configuration Overview

Every arc.toml section and its environment-variable override: server, log, database, storage, ingest, compaction, auth, delete, retention, query, and telemetry keys.

Arc uses a TOML configuration file (arc.toml) with environment variable overrides for flexibility.

Configuration files

Primary: arc.toml

The main configuration file with production-ready defaults:

# Server Configuration
[server]
# host = ""      # bind address (default: all interfaces, dual-stack)
port = 8000

# Logging
[log]
level = "info"       # debug, info, warn, error
format = "console"   # json or console

# Database (query engine)
[database]
# Auto-detected if not set (recommended)
# max_connections = 28    # 2x CPU cores
# memory_limit = "8GB"    # unset: engine uses 80% of the cgroup limit
# thread_count = 14       # unset: engine uses the CPU quota
enable_wal = false

# Storage Backend
[storage]
backend = "local"           # local, s3, minio, azure, azblob
local_path = "./data/arc"

# Ingestion
[ingest]
max_buffer_size = 50000     # records before flush
max_buffer_age_ms = 5000    # ms before force flush
# flush_workers = 16        # async flush workers (auto-detected)
# flush_queue_size = 64     # pending flush queue (auto-detected)
# shard_count = 32          # buffer shards

# Compaction
[compaction]
enabled = true
hourly_enabled = true
hourly_min_age_hours = 0
hourly_min_files = 5

# Authentication
[auth]
enabled = true

# Delete Operations
[delete]
enabled = true
confirmation_threshold = 10000
max_rows_per_delete = 1000000

# Retention Policies
[retention]
enabled = true

# Continuous Queries
[continuous_query]
enabled = true

Environment variables

Override any setting via environment variables with the ARC_ prefix:

# Server
ARC_SERVER_HOST=                   # bind address (empty = all interfaces, v26.06.1+)
ARC_SERVER_PORT=8000
ARC_SERVER_TLS_ENABLED=false
ARC_SERVER_TLS_CERT_FILE=/path/to/cert.pem
ARC_SERVER_TLS_KEY_FILE=/path/to/key.pem
ARC_SERVER_MAX_PAYLOAD_SIZE=1GB    # v26.01.1+

# Logging
ARC_LOG_LEVEL=info
ARC_LOG_FORMAT=json

# Database
ARC_DATABASE_MAX_CONNECTIONS=28
ARC_DATABASE_MEMORY_LIMIT=8GB
ARC_DATABASE_THREAD_COUNT=14

# Features
ARC_AUTH_ENABLED=true
ARC_COMPACTION_ENABLED=true
ARC_DELETE_ENABLED=true
ARC_RETENTION_ENABLED=true
ARC_CONTINUOUS_QUERY_ENABLED=true
ARC_MQTT_ENABLED=true              # Enables the MQTT subsystem and API routes

# Ingestion Concurrency (v26.01.1+)
ARC_INGEST_FLUSH_WORKERS=32
ARC_INGEST_FLUSH_QUEUE_SIZE=200
ARC_INGEST_SHARD_COUNT=64

Configuration priority

Settings are applied in this order (highest to lowest):

Environment variables (e.g., ARC_SERVER_PORT=8000)
arc.toml file
Built-in defaults

Storage backends

Local Filesystem - Default, simplest option for single-node deployments.

[storage]
backend = "local"
local_path = "./data/arc"

Environment variables:

ARC_STORAGE_BACKEND=local
ARC_STORAGE_LOCAL_PATH=./data/arc

Enterprise licensing

Licensing (the [license] section — license keys, boot resilience, and the air-gapped offline license file) is an Arc Enterprise feature and is documented in the Arc Enterprise configuration docs.

Key configuration areas

Server

Basic HTTP server settings:

[server]
host = ""      # bind address (default: empty = all interfaces, dual-stack IPv4 + IPv6)
port = 8000    # HTTP/HTTPS port to listen on

Bind address (`server.host`)

Available in v26.06.1

The server.host setting is available starting from Arc v26.06.1.

Controls which network interface Arc binds to. The default (empty string) preserves the historical behavior of binding to all interfaces with Linux dual-stack semantics (IPv4 + IPv6-mapped addresses).

Common values:

Value	Effect
`""` (empty)	All interfaces, dual-stack (default — same as pre-26.06.1)
`"0.0.0.0"`	All interfaces, IPv4 only
`"::"`	All interfaces, IPv6 (with IPv4-mapped on Linux)
`"127.0.0.1"`	Loopback only (IPv4) — useful behind a reverse proxy
`"::1"`	Loopback only (IPv6)
`"192.0.2.10"`	Specific interface address

Environment variable:

ARC_SERVER_HOST=127.0.0.1

Behind a reverse proxy

If Arc sits behind nginx / Traefik / a Kubernetes Ingress on the same host, set host = "127.0.0.1" so Arc is not directly reachable from the network.

Choosing `0.0.0.0` disables IPv6

Setting host = "0.0.0.0" binds IPv4 only — IPv6 clients will be unable to connect. Leave the value empty (default) to keep dual-stack behavior.

TLS/SSL (HTTPS)

Arc supports native HTTPS/TLS without requiring a reverse proxy:

[server]
port = 443
tls_enabled = true
tls_cert_file = "/etc/letsencrypt/live/example.com/fullchain.pem"
tls_key_file = "/etc/letsencrypt/live/example.com/privkey.pem"

Environment variables:

ARC_SERVER_TLS_ENABLED=true
ARC_SERVER_TLS_CERT_FILE=/path/to/cert.pem
ARC_SERVER_TLS_KEY_FILE=/path/to/key.pem

When to Use Native TLS

Native packages (deb/rpm): Use native TLS for simple deployments
Docker/Kubernetes: Use a reverse proxy (Traefik, nginx, Ingress) for TLS termination
Development: Use self-signed certificates for local HTTPS testing

When TLS is enabled, Arc automatically:

Adds the Strict-Transport-Security (HSTS) header
Validates certificate and key files on startup

Max payload size

Available in v26.01.1

This configuration option is available starting from Arc v26.01.1.

Configure the maximum request payload size for write endpoints (msgpack, line protocol):

[server]
# Maximum payload size (applies to both compressed and decompressed)
# Supports units: B, KB, MB, GB
# Default: 1GB
max_payload_size = "1GB"

Environment variable:

ARC_SERVER_MAX_PAYLOAD_SIZE=2GB

Large Bulk Imports

If you're importing large datasets and encounter 413 errors, you can:

Increase max_payload_size (e.g., "2GB")
Batch your imports into smaller chunks (recommended for reliability)

Database (query engine)

Query engine connection pool and resource settings:

[database]
# AUTO-DETECTION: If not set:
#   - max_connections: 2x CPU cores (min 4, max 64), set by Arc
#   - memory_limit:    left to the query engine, which uses 80% of the
#                      container's cgroup limit (or of host memory when
#                      unlimited). Percent and unit-less forms are rejected.
#   - thread_count:    left to the query engine, which uses the container's
#                      CPU quota rather than the host's core count.

# Manual override examples:
max_connections = 28      # Connection pool size
memory_limit = "8GB"      # Query engine memory limit
thread_count = 14         # Query execution threads
enable_wal = false        # Query engine WAL (not Arc WAL)

# Allow the engine to reorder results of queries WITHOUT an ORDER BY (SQL-standard
# semantics); can reduce memory usage on large un-ordered scans and exports.
# Queries with an explicit ORDER BY are unaffected. Set true to make un-ordered
# SELECTs return rows in file/insertion order (pre-26.09.1 behavior).
preserve_insertion_order = false

Behaviour change in v26.09.3

memory_limit and thread_count are unset by default now, and that is deliberate: the query engine performs its own cgroup-aware detection and uses 80% of the memory limit it finds — your container's limit if you set one, host memory if you did not. It reads the CPU quota for threads the same way.

Before v26.09.3 Arc overrode both from the CPU count: min(cores, 32) GB for memory and the core count for threads. runtime.NumCPU() reflects CPU affinity but not a CFS quota, and Kubernetes limits.cpu and docker --cpus are quotas — so a pod limited to 2 CPUs on a 64-core node saw 64 cores and was given a 32 GB memory limit inside a 2 Gi container. The engine then allocated against that budget rather than spilling to temp_directory, and the kernel OOM-killed the container.

Measured after the change: --memory=512m yields 409.5 MiB, --memory=2g yields 1.5 GiB, and --cpus=2 yields 2 threads on an 8-core host.

Two things to check when you upgrade.

If you run with a container memory limit, the engine's limit now follows it. That is the fix, and in the example above it drops from 32 GB to about 1.6 GiB.

If you run with no memory limit — including the OSS Helm chart, which ships resources: {} — the engine sees the node's total memory and takes 80% of it. On a 16-core/64 GiB node that is a rise, from Arc's old 16 GB to roughly 51 GiB. Set container limits, or set memory_limit explicitly, if you were relying on the old accidental cap.

A smaller limit also makes spilling to temp_directory more likely. Spilling is slow but survivable where an OOM kill is not, so size that directory (or its volume) for the queries you run.

Ingestion

Buffer and concurrency settings for write performance:

[ingest]
# Maximum records to buffer before flushing to Parquet
max_buffer_size = 50000

# Maximum age (ms) before forcing a flush
max_buffer_age_ms = 5000

# Concurrency settings (auto-detected if not set)
# flush_workers = 16       # async flush workers (2x CPU cores, min 8, max 64)
# flush_queue_size = 64    # pending flush queue (4x workers, min 100)
# shard_count = 32         # buffer shards for lock distribution

# Decimal128 precision columns (v26.04.1+)
# decimal_columns = ["trades:price=18,8;amount=18,8", "balances:balance=38,18"]
# default_decimal_columns = ""

# Parquet dictionary encoding at ingest (v26.09.1+). Off by default: ingest
# files are transient — hourly/daily compaction rewrites them,
# which re-encodes every column adaptively — and skipping ingest-time
# dictionaries improves sustained write throughput ~26%. The tradeoff is a
# temporarily larger uncompacted hot partition until the next compaction.
# Set use_dictionary = true to dictionary-encode string columns at ingest
# (numeric columns stay plain); add numeric_dictionary = true to restore
# the full pre-26.09.1 encoding.
# use_dictionary = false
# numeric_dictionary = false

Environment variables: ARC_INGEST_USE_DICTIONARY, ARC_INGEST_NUMERIC_DICTIONARY (v26.09.1+).

Data flushes when either condition is met:

Buffer reaches max_buffer_size records
Buffer age exceeds max_buffer_age_ms

Behaviour change in v26.09.3

When every flush worker is busy and the flush queue is full, Arc keeps the batch in its in-memory buffer and flushes it as soon as a worker frees a slot. Before v26.09.3 that batch was discarded and the write still returned success — with wal.enabled = false, the default, the records were lost.

Two things follow for tuning. Deferred records are held in memory and Arc does not bound that, so a node with many active measurements and slow storage grows its resident set rather than shedding writes — watch arc_buffer_deferred_buffers and the sum of arc_buffer_records_buffered, arc_buffer_records_queued and arc_buffer_records_inflight (the last two from v27.01.1). Lowering either setting below makes flushes smaller and more frequent, which is the lever if that becomes a problem. And max_buffer_size is a per-measurement trigger, so the worst-case memory is roughly active_measurements x max_buffer_size x bytes_per_record; with max_buffer_age_ms at its default the age trigger normally fires long before that, which is what keeps it small in practice.

High Concurrency

For deployments with many concurrent clients (50+), increase flush_workers and flush_queue_size:

[ingest]
flush_workers = 32
flush_queue_size = 200
shard_count = 64

Decimal Precision (v26.04.1+)

For financial or scientific data requiring exact decimal precision, configure decimal columns to store values as native Parquet DECIMAL instead of float64. See the Decimal Precision guide for details.

Compaction

Automatic file optimization:

[compaction]
enabled = true
hourly_enabled = true
hourly_min_age_hours = 0    # Files must be this old
hourly_min_files = 10       # Minimum files to trigger
daily_enabled = true        # Daily tier
daily_min_age_hours = 24
daily_min_files = 12
cycle_timeout = "30m"       # Budget for one cycle (v26.09.2+); positive Go duration

Authentication

Token-based API authentication:

[auth]
enabled = true              # Enable/disable auth
db_path = "./data/arc_auth.db"  # Token database
cache_ttl = 30              # Token cache TTL (seconds)
max_cache_size = 1000       # Max cached tokens

Delete operations

Safe deletion with confirmation:

[delete]
enabled = true
confirmation_threshold = 10000   # Require confirmation above this
max_rows_per_delete = 1000000    # Hard limit per operation

Query

Query execution limits and the query path's schema behavior. The schema anchor settings are explained in the stable field schema guide.

[query]
timeout = 300                 # Query execution timeout in seconds (0 = no timeout)
slow_query_threshold_ms = 0   # Log queries slower than this (0 = off)

# Range-independent field binding through schema anchors (v26.09.2+)
stable_schema = true
stable_schema_bootstrap = true            # build anchors for pre-26.09.2 measurements on first query
stable_schema_bootstrap_max_files = 500   # max files a bootstrap reads; larger measurements are sampled to this many
empty_range_anchor_scan = false           # EXPERIMENTAL: answer a proven-empty range from the anchor alone

# File-level time pruning of the live hour (v26.09.2+, experimental, local backend)
file_time_pruning = false
file_time_pruning_margin_seconds = 300    # writer clock-skew allowance

Retention policies

Automatic data expiration:

[retention]
enabled = true
db_path = "./data/arc_retention.db"

Continuous queries

Scheduled automated queries:

[continuous_query]
enabled = true
db_path = "./data/arc_cq.db"

Write-Ahead Log (WAL)

Optional durability guarantee:

[wal]
enabled = false              # Enable for zero data loss
directory = "./data/wal"
sync_mode = "fdatasync"      # none, fdatasync, fsync
max_size_mb = 500
max_age_seconds = 3600

Metrics

Timeseries metrics collection:

[metrics]
timeseries_retention_minutes = 60
timeseries_interval_seconds = 10

Quick configuration examples

[server]
port = 8000

[log]
level = "debug"
format = "console"

[storage]
backend = "local"
local_path = "./dev_data"

[auth]
enabled = false

[compaction]
enabled = false

Best practices

1. Use arc.toml for permanent settings

Store configuration in arc.toml and version control it (without secrets):

[storage]
backend = "s3"
s3_bucket = "arc"
s3_region = "us-east-1"
# Credentials via environment variables

2. Use environment variables for secrets

export ARC_STORAGE_S3_ACCESS_KEY="your_access_key"
export ARC_STORAGE_S3_SECRET_KEY="your_secret_key"

3. Let Arc auto-detect resources

Arc automatically detects optimal query engine settings based on your system. Only override if you have specific requirements:

[database]
# Leave commented for auto-detection
# max_connections = 28
# memory_limit = "8GB"
# thread_count = 14

4. Enable features progressively

Start simple, add features as needed:

Basic configuration (storage + auth)
Compaction (for query optimization)
Retention policies (for data management)
WAL (for zero data loss guarantee)

5. Monitor configuration impact

Check metrics after configuration changes:

# Memory usage
curl http://localhost:8000/api/v1/metrics/memory

# Query performance
curl http://localhost:8000/api/v1/metrics/query-pool

# Compaction status
curl http://localhost:8000/api/v1/compaction/status

Troubleshooting

Configuration not loading

# Verify TOML syntax (use any TOML validator)
# Check file exists in expected location
ls -la arc.toml

# Arc looks for arc.toml in:
# 1. Current directory
# 2. /etc/arc/arc.toml (native install)

Environment variables not working

# Verify they're set
env | grep ARC_

# Use correct prefix and format
export ARC_SERVER_PORT=8000     # Correct
export SERVER_PORT=8000         # Wrong - missing ARC_ prefix

Resource issues

# Check current settings via metrics
curl http://localhost:8000/api/v1/metrics/memory

# Adjust in arc.toml:
[database]
memory_limit = "4GB"
max_connections = 16

Next steps

Authentication - Token management
Advanced Features - Compaction and WAL

On this page