BasekickLabs
Configuration

Configuration Overview

Every arc.toml section for Arc Enterprise, including the license key, cluster, tiered_storage, audit_log, and governance blocks, plus the environment variables that override them.

Arc uses a TOML configuration file (arc.toml) with environment variable overrides for flexibility.

Enterprise license

Configure your Arc Enterprise license key to enable enterprise features:

[license]
key = "ARC-XXXX-XXXX-XXXX-XXXX"

Environment variable:

ARC_LICENSE_KEY="ARC-XXXX-XXXX-XXXX-XXXX"

On startup, Arc validates your license and enables the features included in your plan. See Arc Enterprise Overview for the full feature list and Licensing for how to get a key, core and machine limits, and expiry behavior.

As of 26.09.1, a transient license-server failure no longer crash-loops Enterprise pods: startup retries briefly and then falls back to the last signature-verified license cached on disk, honored until that license's own expiry. A definitive server rejection (revoked, expired, unknown key) still disables enterprise features immediately.

Air-gapped: Offline license file (26.09.1+)

For environments with no route to enterprise.basekick.net, download an offline license file from the activation server admin (an explicit site license: unbound, valid on any machine until expiry, audit-logged at mint) and point Arc at it:

[license]
file_path = "/etc/arc/license.json"   # or ARC_LICENSE_FILE_PATH

The file is verified from disk against Arc's pinned public key — no network calls of any kind. file_path wins over key; a rejected file means OSS mode (never a silent fallback to online licensing). Keep the file readable only by the Arc user (0600).

Configuration files

Primary: arc.toml

The main configuration file with production-ready defaults:

# Server Configuration
[server]
port = 8000

# Logging
[log]
level = "info"       # debug, info, warn, error
format = "console"   # json or console

# Database (query engine)
[database]
# Auto-detected if not set (recommended)
# max_connections = 28    # 2x CPU cores
# memory_limit = "8GB"    # ~50% system RAM
# thread_count = 14       # CPU cores
enable_wal = false

# Storage Backend
[storage]
backend = "local"           # local, s3, minio, azure, azblob
local_path = "./data/arc"

# Ingestion
[ingest]
max_buffer_size = 50000     # records before flush
max_buffer_age_ms = 5000    # ms before force flush
# flush_workers = 16        # async flush workers (auto-detected)
# flush_queue_size = 64     # pending flush queue (auto-detected)
# shard_count = 32          # buffer shards

# Compaction
[compaction]
enabled = true
hourly_enabled = true
hourly_min_age_hours = 0
hourly_min_files = 5

# Authentication
[auth]
enabled = true

# Delete Operations
[delete]
enabled = true
confirmation_threshold = 10000
max_rows_per_delete = 1000000

# Retention Policies
[retention]
enabled = true

# Continuous Queries
[continuous_query]
enabled = true

Environment variables

Override any setting via environment variables with the ARC_ prefix:

# Server
ARC_SERVER_PORT=8000
ARC_SERVER_TLS_ENABLED=false
ARC_SERVER_TLS_CERT_FILE=/path/to/cert.pem
ARC_SERVER_TLS_KEY_FILE=/path/to/key.pem
ARC_SERVER_MAX_PAYLOAD_SIZE=1GB

# Logging
ARC_LOG_LEVEL=info
ARC_LOG_FORMAT=json

# Database
ARC_DATABASE_MAX_CONNECTIONS=28
ARC_DATABASE_MEMORY_LIMIT=8GB
ARC_DATABASE_THREAD_COUNT=14

# Features
ARC_AUTH_ENABLED=true
ARC_COMPACTION_ENABLED=true
ARC_DELETE_ENABLED=true
ARC_RETENTION_ENABLED=true
ARC_CONTINUOUS_QUERY_ENABLED=true

# Ingestion Concurrency
ARC_INGEST_FLUSH_WORKERS=32
ARC_INGEST_FLUSH_QUEUE_SIZE=200
ARC_INGEST_SHARD_COUNT=64

Configuration priority

Settings are applied in this order (highest to lowest):

  1. Environment variables (e.g., ARC_SERVER_PORT=8000)
  2. arc.toml file
  3. Built-in defaults

Storage backends

Local Filesystem - Default, simplest option for single-node deployments.

[storage]
backend = "local"
local_path = "./data/arc"

Environment variables:

ARC_STORAGE_BACKEND=local
ARC_STORAGE_LOCAL_PATH=./data/arc

Key configuration areas

Server

Basic HTTP server settings:

[server]
port = 8000    # HTTP/HTTPS port to listen on

TLS/SSL (HTTPS)

Arc supports native HTTPS/TLS without requiring a reverse proxy:

[server]
port = 443
tls_enabled = true
tls_cert_file = "/etc/letsencrypt/live/example.com/fullchain.pem"
tls_key_file = "/etc/letsencrypt/live/example.com/privkey.pem"

Environment variables:

ARC_SERVER_TLS_ENABLED=true
ARC_SERVER_TLS_CERT_FILE=/path/to/cert.pem
ARC_SERVER_TLS_KEY_FILE=/path/to/key.pem

When to Use Native TLS

  • Native packages (deb/rpm): Use native TLS for simple deployments
  • Docker/Kubernetes: Use a reverse proxy (Traefik, nginx, Ingress) for TLS termination
  • Development: Use self-signed certificates for local HTTPS testing

When TLS is enabled, Arc automatically:

  • Adds the Strict-Transport-Security (HSTS) header
  • Validates certificate and key files on startup

Max payload size

Configure the maximum request payload size for write endpoints (msgpack, line protocol):

[server]
# Maximum payload size (applies to both compressed and decompressed)
# Supports units: B, KB, MB, GB
# Default: 1GB
max_payload_size = "1GB"

Environment variable:

ARC_SERVER_MAX_PAYLOAD_SIZE=2GB

Large Bulk Imports

If you're importing large datasets and encounter 413 errors, you can:

  1. Increase max_payload_size (e.g., "2GB")
  2. Batch your imports into smaller chunks (recommended for reliability)

Database (query engine)

Query engine connection pool and resource settings:

[database]
# AUTO-DETECTION: If not set, Arc automatically configures:
#   - max_connections: 2x CPU cores (min 4, max 64)
#   - memory_limit: ~50% of system memory
#   - thread_count: Number of CPU cores

# Manual override examples:
max_connections = 28      # Connection pool size
memory_limit = "8GB"      # Query engine memory limit
thread_count = 14         # Query execution threads
enable_wal = false        # Query engine WAL (not Arc WAL)

Ingestion

Buffer and concurrency settings for write performance:

[ingest]
# Maximum records to buffer before flushing to Parquet
max_buffer_size = 50000

# Maximum age (ms) before forcing a flush
max_buffer_age_ms = 5000

# Concurrency settings (auto-detected if not set)
# flush_workers = 16       # async flush workers (2x CPU cores, min 8, max 64)
# flush_queue_size = 64    # pending flush queue (4x workers, min 100)
# shard_count = 32         # buffer shards for lock distribution

Data flushes when either condition is met:

  1. Buffer reaches max_buffer_size records
  2. Buffer age exceeds max_buffer_age_ms

High Concurrency

For deployments with many concurrent clients (50+), increase flush_workers and flush_queue_size:

[ingest]
flush_workers = 32
flush_queue_size = 200
shard_count = 64

Compaction

Automatic file optimization:

[compaction]
enabled = true
hourly_enabled = true
hourly_min_age_hours = 0    # Files must be this old
hourly_min_files = 10       # Minimum files to trigger
daily_enabled = true        # Daily tier
daily_min_age_hours = 24
daily_min_files = 12
cycle_timeout = "30m"       # Budget for one cycle (v26.09.2+); positive Go duration

Authentication

Token-based API authentication:

[auth]
enabled = true              # Enable/disable auth
db_path = "./data/arc_auth.db"  # Token database
cache_ttl = 30              # Token cache TTL (seconds)
max_cache_size = 1000       # Max cached tokens
bootstrap_token = ""        # Pre-set admin token value (v26.04.1+)
force_bootstrap = false     # Add a recovery token without removing existing ones (v26.04.1+)

Available since v26.04.1

bootstrap_token — Set a known admin token at deploy time via ARC_AUTH_BOOTSTRAP_TOKEN instead of catching a randomly generated one from startup logs. On first run, Arc uses this value as the initial admin token. On subsequent restarts, it is a no-op.

force_bootstrap — Recovery path when the admin token is lost. Set ARC_AUTH_FORCE_BOOTSTRAP=true alongside ARC_AUTH_BOOTSTRAP_TOKEN to add a new arc-recovery admin token without removing existing tokens. Remove this flag after recovery. See the Authentication configuration guide for full details.

Delete operations

Safe deletion with confirmation:

[delete]
enabled = true
confirmation_threshold = 10000   # Require confirmation above this
max_rows_per_delete = 1000000    # Hard limit per operation

Query

Query execution limits and the query path's schema behavior. The schema anchor settings are explained in the stable field schema guide.

[query]
timeout = 300                 # Query execution timeout in seconds (0 = no timeout)
slow_query_threshold_ms = 0   # Log queries slower than this (0 = off)

# Range-independent field binding through schema anchors (v26.09.2+)
stable_schema = true
stable_schema_bootstrap = true            # build anchors for pre-26.09.2 measurements on first query
stable_schema_bootstrap_max_files = 500   # max files a bootstrap reads; larger measurements are sampled to this many
empty_range_anchor_scan = false           # EXPERIMENTAL: answer a proven-empty range from the anchor alone
# On a cluster whose nodes keep separate storage and receive each other's
# files by replication, leave empty_range_anchor_scan off (see the guide).

# File-level time pruning of the live hour (v26.09.2+, experimental, local backend)
file_time_pruning = false
file_time_pruning_margin_seconds = 300    # writer clock-skew allowance

Retention policies

Automatic data expiration:

[retention]
enabled = true
db_path = "./data/arc_retention.db"

Continuous queries

Scheduled automated queries:

[continuous_query]
enabled = true
db_path = "./data/arc_cq.db"

Write-Ahead Log (WAL)

Optional durability guarantee:

[wal]
enabled = false              # Enable for zero data loss
directory = "./data/wal"
sync_mode = "fdatasync"      # none, fdatasync, fsync
max_size_mb = 500
max_age_seconds = 3600

Metrics

Timeseries metrics collection:

[metrics]
timeseries_retention_minutes = 60
timeseries_interval_seconds = 10

Quick configuration examples

[server]
port = 8000

[log]
level = "debug"
format = "console"

[storage]
backend = "local"
local_path = "./dev_data"

[auth]
enabled = false

[compaction]
enabled = false

Best practices

1. Use arc.toml for permanent settings

Store configuration in arc.toml and version control it (without secrets):

[storage]
backend = "s3"
s3_bucket = "arc"
s3_region = "us-east-1"
# Credentials via environment variables

2. Use environment variables for secrets

export ARC_STORAGE_S3_ACCESS_KEY="your_access_key"
export ARC_STORAGE_S3_SECRET_KEY="your_secret_key"

3. Let Arc auto-detect resources

Arc automatically detects optimal query engine settings based on your system. Only override if you have specific requirements:

[database]
# Leave commented for auto-detection
# max_connections = 28
# memory_limit = "8GB"
# thread_count = 14

4. Enable features progressively

Start simple, add features as needed:

  1. Basic configuration (storage + auth)
  2. Compaction (for query optimization)
  3. Retention policies (for data management)
  4. WAL (for zero data loss guarantee)

5. Monitor configuration impact

Check metrics after configuration changes:

# Memory usage
curl http://localhost:8000/api/v1/metrics/memory

# Query performance
curl http://localhost:8000/api/v1/metrics/query-pool

# Compaction status
curl http://localhost:8000/api/v1/compaction/status

Troubleshooting

Configuration not loading

# Verify TOML syntax (use any TOML validator)
# Check file exists in expected location
ls -la arc.toml

# Arc looks for arc.toml in:
# 1. Current directory
# 2. /etc/arc/arc.toml (native install)

Environment variables not working

# Verify they're set
env | grep ARC_

# Use correct prefix and format
export ARC_SERVER_PORT=8000     # Correct
export SERVER_PORT=8000         # Wrong - missing ARC_ prefix

Resource issues

# Check current settings via metrics
curl http://localhost:8000/api/v1/metrics/memory

# Adjust in arc.toml:
[database]
memory_limit = "4GB"
max_connections = 16

Enterprise configuration

The following configuration sections are available with an Arc Enterprise license. See each feature's dedicated page for detailed configuration.

Clustering

See Clustering & High Availability for full configuration reference.

[cluster]
enabled = true
node_id = "writer-01"
role = "writer"
cluster_name = "production"
seeds = ["10.0.1.10:9000"]

Tiered storage

See Tiered Storage for full configuration reference.

[tiered_storage]
enabled = true
default_hot_max_age_days = 30

[tiered_storage.cold]
enabled = true
backend = "s3"
s3_bucket = "arc-archive"

Audit logging

See Audit Logging for full configuration reference.

[audit_log]
enabled = true
retention_days = 90

Query governance

See Query Governance for full configuration reference.

[governance]
enabled = true
default_rate_limit_per_min = 60

Query management

See Query Management for full configuration reference.

[query_management]
enabled = true

Next steps

Deployment Patterns

Choose between Arc Enterprise's two cluster topologies — shared object storage or local disks with peer replication — and what each implies for durability, cost, and operations.

Clustering & High Availability

Configure an Arc Enterprise cluster: assign writer, reader, and compactor roles, set seeds and Raft addresses, tune peer file replication and catch-up, and enable writer failover.

On this page

Enterprise licenseAir-gapped: Offline license file (26.09.1+)Configuration filesPrimary: arc.tomlEnvironment variablesConfiguration priorityStorage backendsKey configuration areasServerTLS/SSL (HTTPS)Max payload sizeDatabase (query engine)IngestionCompactionAuthenticationDelete operationsQueryRetention policiesContinuous queriesWrite-Ahead Log (WAL)MetricsQuick configuration examplesBest practices1. Use arc.toml for permanent settings2. Use environment variables for secrets3. Let Arc auto-detect resources4. Enable features progressively5. Monitor configuration impactTroubleshootingConfiguration not loadingEnvironment variables not workingResource issuesEnterprise configurationClusteringTiered storageAudit loggingQuery governanceQuery managementNext steps