BasekickLabs

Writing data

Send MessagePack or line protocol data to Arc with curl, Python, Go, or Telegraf.

Arc accepts MessagePack for efficient typed batches and InfluxDB line protocol for simple, portable writes. Both APIs return 204 No Content when a write is accepted.

Use caseRecommended path
Application writes and large batchesMessagePack columnar
Shell scripts and existing InfluxDB clientsLine protocol
Host, service, and infrastructure metricsTelegraf
Historical filesData import

The examples use a database named metrics and read the token from ARC_TOKEN. Create the database once before writing:

arcli db create metrics
export ARC_TOKEN="your-token"

MessagePack

Send MessagePack to POST /api/v1/write/msgpack. The columnar format groups values by column, which keeps batches compact:

{
  "m": "cpu",
  "columns": {
    "time": [1788973200000000, 1788973201000000],
    "host": ["server01", "server02"],
    "usage": [45.2, 67.8]
  }
}
  • m is the measurement name.
  • Every array in columns must have the same length.
  • Values within a column must use a consistent type.
  • time is a numeric Unix timestamp. Arc detects seconds, milliseconds, microseconds, or nanoseconds by magnitude.

curl can send an already encoded MessagePack file:

curl --request POST http://localhost:8000/api/v1/write/msgpack \
  --header "Authorization: Bearer $ARC_TOKEN" \
  --header "Content-Type: application/msgpack" \
  --header "x-arc-database: metrics" \
  --data-binary @cpu.msgpack

Use the Python or Go example to encode a payload, then write the resulting bytes to cpu.msgpack when a file-based workflow is useful.

For sustained ingestion, batch multiple rows per request. Arc also accepts gzip and zstd request compression through the Content-Encoding header.

Line protocol

Line protocol is convenient for scripts and tools that already speak the InfluxDB protocol:

measurement,tag=value field=value timestamp

This example writes a floating-point field, an integer field, and a nanosecond timestamp:

cpu,host=server01,region=us-east usage=45.2,requests=12i 1788973200000000000

Send it to Arc's native POST /api/v1/write/line-protocol endpoint.

curl --request POST http://localhost:8000/api/v1/write/line-protocol \
  --header "Authorization: Bearer $ARC_TOKEN" \
  --header "Content-Type: text/plain" \
  --header "x-arc-database: metrics" \
  --data-binary "cpu,host=server01,region=us-east usage=45.2,requests=12i $(date +%s)000000000"

Arc also supports the InfluxDB 1.x /write and InfluxDB 2.x /api/v2/write endpoints for compatible clients. See the write API reference for their parameters.

Partial writes

A line-protocol request, or a MessagePack request in row format, is written point by point: a point that cannot be stored is dropped, the other points in the request are written, and the response is 400 with partial write: N points dropped: ..., naming each measurement, column and reason (v27.01.1+, as InfluxDB does). A point is dropped when:

  • It has a tag or field named time. time is the timestamp, not a tag or field key (arc#1297). Before, a time field silently replaced the point's timestamp.
  • A value does not fit its column's type. A column takes the type of its first value in the request. A later string in a float column, a number in a tag column, or an unsigned value above the int64 range, NaN or Inf in an integer column drops that point; an integer in a float column is widened, and a float in an integer column is truncated. Before, such a value failed the whole measurement's write with a 500 (arc#1299).
  • Two of its keys land in one column. A field named like one of the point's tags is stored as <name>_value, so a point with a tag host, a field host and a field host_value gives one column two values.

Arc logs each request's drops at Error level (Line protocol points dropped (partial write), or MessagePack points dropped (partial write)), one line per measurement, column and reason (up to 10 per request). Before v27.01.1, a row-format MessagePack point of the wrong type failed its whole measurement with a 500, and some points with two keys in one column were accepted and then made every flush of their buffer fail.

Telegraf

For system and service metrics, use Telegraf's native Arc output plugin. It batches metrics as MessagePack, supports compression, and can collect from hundreds of existing input plugins.

Send Telegraf metrics to Arc

Install Telegraf, configure the Arc output, and verify your metrics.

Verify the write

Query the measurement with arcli:

arcli query --database metrics "SELECT * FROM cpu ORDER BY time DESC LIMIT 10"

The --database metrics option sends x-arc-database: metrics, so the SQL only needs the measurement name: FROM cpu.

Arc buffers writes briefly, so newly accepted data may take a few seconds to appear in a query.

On this page