BasekickLabs

Data Management

Manage Arc Enterprise retention policies, continuous queries, delete operations, and API tokens from Python, including tokens scoped by the cluster's RBAC roles.

How to manage data lifecycle, authentication, and administrative tasks using the Python SDK.

These examples need arc-tsdb-client 0.2.0 or later:

pip install "arc-tsdb-client>=0.2.0"

0.1.2 and earlier have no replace() for retention policies and continuous queries, which the update examples below use on Arc up to 26.09.4.

Overview

The SDK provides clients for managing Arc's data lifecycle features:

ClientPurposeUse Case
client.retentionRetention policiesDelete old data on a schedule
client.continuous_queriesContinuous queriesDownsample and aggregate data
client.deleteDelete operationsRemove data matching conditions
client.authAuthenticationManage API tokens

Scheduling in Arc Enterprise

Arc Enterprise runs retention policies and continuous queries automatically on configurable cron schedules — see Automated Scheduling.

The SDK methods below trigger these operations on demand, which is still useful for backfills, ad-hoc runs, and testing a policy before putting it on a schedule. See Scheduling with External Tools if you prefer to drive execution from an external orchestrator (cron, Airflow, etc.).

Retention policies

Retention policies define rules for deleting data older than a specified age. Use them to:

  • Control storage costs
  • Comply with data retention regulations
  • Remove stale data on a schedule

Create a policy

from arc_client import ArcClient

with ArcClient(host="localhost", token="your-token") as client:
    policy = client.retention.create(
        name="logs-30d",
        database="default",
        retention_days=30,
        measurement="logs",    # Optional: applies to specific measurement
        buffer_days=7,         # Optional: keep extra days as safety buffer
    )

    print(f"Created policy: {policy.name} (id={policy.id})")

The shorter snippets on this page run inside a with ArcClient(...) as client: block like this one, and reuse policy and cq from the create examples.

Parameters

ParameterTypeRequiredDescription
namestrYesUnique name for the policy
databasestrYesTarget database
retention_daysintYesDelete data older than N days
measurementstrNoLimit to specific measurement (all if omitted)
buffer_daysintNoExtra buffer days before deletion

List policies

policies = client.retention.list()

for p in policies:
    status = "active" if p.is_active else "inactive"
    measurement = p.measurement or "all measurements"
    print(f"{p.name}: {p.retention_days} days on {measurement} ({status})")

Execute a policy

Always use dry_run=True first to preview what will be deleted:

# Preview deletion
result = client.retention.execute(policy.id, dry_run=True)
print(f"Would delete {result.deleted_count} rows")

# Execute for real (requires confirm=True for large deletes)
result = client.retention.execute(policy.id, dry_run=False, confirm=True)
print(f"Deleted {result.deleted_count} rows")

Update a policy

Up to Arc 26.09.4, PUT /api/v1/retention/:id replaces the whole policy, so use replace() and pass every field. Copy the ones you are not changing from the existing policy:

client.retention.replace(
    policy.id,
    name=policy.name,
    database=policy.database,
    retention_days=60,              # Change retention period
    measurement=policy.measurement,
    buffer_days=policy.buffer_days,
    is_active=False,                # Disable the policy
)

retention.update() sends only the fields you pass, which Arc up to 26.09.4 refuses (400 invalid database name ""). Arc 27.01.1 accepts partial updates, so update() works from that release. replace() works on both.

On Arc 27.01.1 and later, pass CLEAR (from arc_client import CLEAR) to update() to set a nullable field back to null, for example client.retention.update(policy.id, measurement=CLEAR) to apply the policy to the whole database.

Delete a policy

client.retention.delete(policy.id)

Full example

from arc_client import ArcClient

with ArcClient(host="localhost", token="your-token") as client:
    # Create policy for logs
    policy = client.retention.create(
        name="logs-retention",
        database="default",
        retention_days=30,
        measurement="logs",
    )

    # Create policy for metrics (keep longer)
    metrics_policy = client.retention.create(
        name="metrics-retention",
        database="default",
        retention_days=90,
        measurement="metrics",
    )

    # List all policies
    for p in client.retention.list():
        print(f"  - {p.name}: {p.retention_days} days")

    # Dry run to see what would be deleted
    result = client.retention.execute(policy.id, dry_run=True)
    print(f"\nDry run: would delete {result.deleted_count} rows")

See Retention Policies for more details on how retention works in Arc.

Continuous queries

Continuous queries (CQs) define aggregation rules that transform data from one measurement to another. Use them to:

  • Downsample high-resolution data to save storage
  • Pre-compute aggregations for faster dashboard queries
  • Create materialized views of your data

Execution Required

CQs define what to aggregate and where to store results. The interval parameter documents the intended frequency, but you must trigger execution manually or via an external scheduler. See Scheduling with External Tools.

Create a continuous query

from arc_client import ArcClient

with ArcClient(host="localhost", token="your-token") as client:
    cq = client.continuous_queries.create(
        name="cpu-hourly-avg",
        database="default",
        source_measurement="cpu",
        destination_measurement="cpu_1h",
        query="""
            SELECT
                time_bucket('1 hour', time) as time,
                host,
                avg(usage_idle) as usage_idle,
                avg(usage_system) as usage_system,
                max(usage_user) as max_usage_user
            FROM default.cpu
            WHERE time >= {start_time} AND time < {end_time}
            GROUP BY 1, 2
        """,
        interval="1h",
        description="Hourly CPU averages per host",
    )

    print(f"Created CQ: {cq.name} (id={cq.id})")

Parameters

ParameterTypeRequiredDescription
namestrYesUnique name for the CQ
databasestrYesTarget database
source_measurementstrYesMeasurement to read from
destination_measurementstrYesMeasurement to write results to
querystrYesSQL aggregation query
intervalstrYesExecution interval (e.g., 1h, 15m, 1d)
descriptionstrNoHuman-readable description

Query guidelines

Your CQ query should:

  • Use time_bucket() to aggregate time into intervals
  • Include time as the first column in SELECT and GROUP BY
  • Use aggregate functions (avg, sum, count, min, max, etc.)
  • Contain the {start_time} and {end_time} placeholders, normally as WHERE time >= {start_time} AND time < {end_time}. Arc refuses a query without them and substitutes each run's window.
  • Name the source as FROM <database>.<source_measurement> (here FROM default.cpu). Arc finds the source by matching exactly that text and rewrites it to read the measurement's files; a bare FROM cpu is not rewritten and fails with Table with name cpu does not exist.

List continuous queries

cqs = client.continuous_queries.list(database="default")

for cq in cqs:
    status = "active" if cq.is_active else "inactive"
    print(f"{cq.name}: {cq.source_measurement} → {cq.destination_measurement}")
    print(f"  Interval: {cq.interval} ({status})")

Manual execution

Execute a CQ manually for a specific time range:

# Dry run first
result = client.continuous_queries.execute(
    cq.id,
    start_time="2024-01-01T00:00:00Z",
    end_time="2024-01-02T00:00:00Z",
    dry_run=True,
)
print(f"Would process {result.records_read or 0} records")
print(f"Would write {result.records_written or 0} records")

# Execute for real
result = client.continuous_queries.execute(
    cq.id,
    start_time="2024-01-01T00:00:00Z",
    end_time="2024-01-02T00:00:00Z",
    dry_run=False,
)

Update a CQ

Up to Arc 26.09.4, PUT /api/v1/continuous_queries/:id replaces the whole definition, so use replace() and pass every field. Copy the ones you are not changing from the existing CQ:

client.continuous_queries.replace(
    cq.id,
    name=cq.name,
    database=cq.database,
    source_measurement=cq.source_measurement,
    destination_measurement=cq.destination_measurement,
    query=cq.query,
    interval="30m",       # Change interval
    description=cq.description,
    tag_columns=cq.tag_columns,
    retention_days=cq.retention_days,
    delete_source_after_days=cq.delete_source_after_days,
    is_active=False,      # Pause the CQ
)

continuous_queries.update() sends only the fields you pass, which Arc up to 26.09.4 refuses (400 name is required). Arc 27.01.1 accepts partial updates, so update() works from that release. replace() works on both.

On Arc 27.01.1 and later, pass CLEAR to update() to set a nullable field back to null, for example client.continuous_queries.update(cq.id, description=CLEAR). The nullable CQ fields are description, tag_columns, retention_days and delete_source_after_days.

Delete a CQ

client.continuous_queries.delete(cq.id)

Full example

from arc_client import ArcClient

with ArcClient(host="localhost", token="your-token") as client:
    # Create hourly rollup
    hourly_cq = client.continuous_queries.create(
        name="cpu-hourly",
        database="default",
        source_measurement="cpu",
        destination_measurement="cpu_1h",
        query="""
            SELECT
                time_bucket('1 hour', time) as time,
                host,
                avg(usage_idle) as avg_idle,
                min(usage_idle) as min_idle,
                max(usage_idle) as max_idle
            FROM default.cpu
            WHERE time >= {start_time} AND time < {end_time}
            GROUP BY 1, 2
        """,
        interval="1h",
    )

    # Create daily rollup from hourly data
    daily_cq = client.continuous_queries.create(
        name="cpu-daily",
        database="default",
        source_measurement="cpu_1h",
        destination_measurement="cpu_1d",
        query="""
            SELECT
                time_bucket('1 day', time) as time,
                host,
                avg(avg_idle) as avg_idle,
                min(min_idle) as min_idle,
                max(max_idle) as max_idle
            FROM default.cpu_1h
            WHERE time >= {start_time} AND time < {end_time}
            GROUP BY 1, 2
        """,
        interval="1d",
    )

    print("Created CQ hierarchy: cpu → cpu_1h → cpu_1d")

See Continuous Queries for more details.

Delete operations

Delete data matching specific conditions. Use this for:

  • Removing erroneous data
  • Deleting data for specific hosts or time ranges
  • GDPR/compliance data removal

Delete with conditions

from arc_client import ArcClient

with ArcClient(host="localhost", token="your-token") as client:
    # ALWAYS dry_run first!
    result = client.delete.delete(
        database="default",
        measurement="logs",
        where="time < '2024-01-01' AND level = 'debug'",
        dry_run=True,
    )
    print(f"Would delete {result.deleted_count} rows")
    print(f"Affected files: {result.affected_files}")

    # Execute deletion (requires confirm=True)
    result = client.delete.delete(
        database="default",
        measurement="logs",
        where="time < '2024-01-01' AND level = 'debug'",
        dry_run=False,
        confirm=True,
    )
    print(f"Deleted {result.deleted_count} rows")

Parameters

ParameterTypeRequiredDescription
databasestrYesTarget database
measurementstrYesTarget measurement
wherestrYesSQL WHERE clause (without "WHERE" keyword)
dry_runboolNoPreview only, don't delete (default: True)
confirmboolNoRequired for large deletes

Common delete patterns

# Delete old data
client.delete.delete(
    database="default",
    measurement="logs",
    where="time < '2024-01-01'",
    dry_run=False,
    confirm=True,
)

# Delete data for a specific host
client.delete.delete(
    database="default",
    measurement="metrics",
    where="host = 'decommissioned-server'",
    dry_run=False,
    confirm=True,
)

# Delete data in a time range
client.delete.delete(
    database="default",
    measurement="events",
    where="time BETWEEN '2024-01-15' AND '2024-01-16'",
    dry_run=False,
    confirm=True,
)

See Delete Operations for more details.

Authentication

Manage API tokens for accessing Arc.

Verify current token

from arc_client import ArcClient

with ArcClient(host="localhost", token="your-token") as client:
    result = client.auth.verify()

    if result.valid:
        print(f"Token name: {result.token_info.name}")
        print(f"Permissions: {result.permissions}")
        print(f"Created: {result.token_info.created_at}")
    else:
        print("Token is invalid or expired")

Create a new token

result = client.auth.create_token(
    name="my-app-token",
    description="Token for my application",
    permissions=["read", "write"],
)

# IMPORTANT: Save this token - it's only shown once!
print(f"New token: {result.token}")

The response carries the secret but not the token's ID. Find the ID with list_tokens() when you need it for rotate_token() or revoke_token().

Available permissions

PermissionDescription
readQuery data
writeWrite/ingest data
adminManage tokens, retention policies, CQs

List tokens

tokens = client.auth.list_tokens()

for t in tokens:
    print(f"{t.name} (id={t.id})")
    print(f"  Created: {t.created_at}")
    print(f"  Last used: {t.last_used_at or 'never'}")

Rotate a token

Generate a new token value while keeping the same token ID and permissions:

new_token = client.auth.rotate_token(token_id=123)

# IMPORTANT: Save the new token - the old one is now invalid!
print(f"New token: {new_token}")

Revoke a token

client.auth.revoke_token(token_id=123)
print("Token revoked")

Error handling

All data management operations can raise specific exceptions:

from arc_client import ArcClient
from arc_client.exceptions import (
    ArcError,
    ArcNotFoundError,
    ArcServerError,
    ArcAuthenticationError,
)

with ArcClient(host="localhost", token="your-token") as client:
    try:
        client.retention.get(999)  # Non-existent policy
    except ArcNotFoundError:
        print("Policy not found")

    try:
        client.retention.create(
            name="",  # Invalid name
            database="default",
            retention_days=-1,  # Invalid days
        )
    except ArcServerError as e:
        print(f"Rejected by Arc: {e}")  # Arc's 400 surfaces as ArcServerError, e.g. "name is required"
    except ArcError as e:
        print(f"Other Arc error: {e}")  # base class of every SDK exception

# A missing, invalid or expired token raises ArcAuthenticationError
with ArcClient(host="localhost", token="not-a-valid-token") as client:
    try:
        client.retention.list()
    except ArcAuthenticationError as e:
        print(f"Token rejected: {e}")

Permission errors

Arc answers HTTP 403 for several different reasons, and the SDK raises a different exception for each so they can be handled separately:

from arc_client.exceptions import (
    ArcAuthenticationError,          # 401 — token missing, invalid or expired
    ArcPermissionError,              # 403 — token lacks the permission needed
    ArcScopedAccessError,            # 403 — may read, but not that database
    ArcPermissionDataUnavailableError,  # 403 — Arc could not load the grants
    ArcNotFoundError,                # 404
)

try:
    databases = client.databases.list().databases
except ArcScopedAccessError as e:
    # Expected for a token scoped to specific databases: Arc refuses to list
    # them all rather than returning a partial list, matching SHOW DATABASES.
    # Not a failure, and not worth retrying — name a database instead.
    print(f"scoped (refused for {e.database!r}); naming one instead")
    measurements = client.databases.list_measurements("production")
except ArcPermissionError:
    # A different problem: this token has no read permission at all, so
    # naming a database will not help.
    raise

ArcScopedAccessError.database carries the database the call was refused for. It is empty or "*" when the call asked for every database, so test it against None rather than for truthiness if you need to tell "not a scoped refusal" from "scoped, no database named".

ArcScopedAccessError and ArcPermissionDataUnavailableError both subclass ArcPermissionError, which subclasses ArcAuthenticationError — so code that already caught 403 as an authentication failure keeps working, and except ArcPermissionError catches any refusal if the distinction does not matter to you.

ArcAuthenticationError now means 401 only

Before this release a 403 also raised ArcAuthenticationError. Code that needs to distinguish "re-authenticate" from "this token lacks a permission" should catch ArcAuthenticationError and ArcPermissionError separately.

See RBAC for what each refusal means on the server side.

Async support

All data management operations have async equivalents:

import asyncio
from arc_client import AsyncArcClient

async def main():
    async with AsyncArcClient(host="localhost", token="your-token") as client:
        # Retention
        policy = await client.retention.create(
            name="async-policy",
            database="default",
            retention_days=30,
        )

        # CQs
        cq = await client.continuous_queries.create(
            name="async-cq",
            database="default",
            source_measurement="cpu",
            destination_measurement="cpu_1h",
            query="SELECT time_bucket('1 hour', time) as time, avg(usage) as usage FROM default.cpu WHERE time >= {start_time} AND time < {end_time} GROUP BY 1",
            interval="1h",
        )

        # Delete
        result = await client.delete.delete(
            database="default",
            measurement="logs",
            where="time < '2024-01-01'",
            dry_run=True,
        )

        # Auth
        verify = await client.auth.verify()

asyncio.run(main())

Best practices

1. Always dry run first

For any destructive operation (delete, retention execution), always preview first:

# ✅ Good: Preview before executing
result = client.delete.delete(..., dry_run=True)
print(f"Would delete {result.deleted_count} rows")
# Review the count, then execute

# ❌ Bad: Delete without preview
result = client.delete.delete(..., dry_run=False, confirm=True)

2. Use descriptive names

# ✅ Good: Descriptive names
client.retention.create(name="logs-30d-cleanup", ...)
client.continuous_queries.create(name="cpu-hourly-avg-by-host", ...)

# ❌ Bad: Generic names
client.retention.create(name="policy1", ...)

3. Document your CQ queries

# ✅ Good: Include description
cq = client.continuous_queries.create(
    name="cpu-hourly",
    description="Hourly CPU averages per host. Used by main dashboard.",
    ...
)

4. Secure token management

# ✅ Good: Use environment variables
import os
from arc_client import ArcClient

token = os.getenv("ARC_TOKEN")
client = ArcClient(host="localhost", token=token)

# ❌ Bad: Hardcode tokens
client = ArcClient(host="localhost", token="<api-token>")

Scheduling with external tools

Since Arc OSS doesn't include a built-in scheduler, you need to trigger retention policies and continuous queries externally. Here are several approaches:

Simple Python script with cron

Create a script that executes your policies and CQs:

arc_scheduler.py
#!/usr/bin/env python3
"""Execute Arc retention policies and continuous queries."""

import logging
from datetime import datetime, timedelta
from arc_client import ArcClient

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

def run_retention_policies(client: ArcClient):
    """Execute all active retention policies."""
    policies = client.retention.list()

    for policy in policies:
        if not policy.is_active:
            continue

        logger.info(f"Executing retention policy: {policy.name}")
        try:
            result = client.retention.execute(policy.id, dry_run=False, confirm=True)
            logger.info(f"  Deleted {result.deleted_count} rows")
        except Exception as e:
            logger.error(f"  Failed: {e}")

def run_continuous_queries(client: ArcClient):
    """Execute all active continuous queries for the last interval."""
    cqs = client.continuous_queries.list()

    for cq in cqs:
        if not cq.is_active:
            continue

        # Calculate time range based on CQ interval
        end_time = datetime.utcnow()
        # Parse interval (e.g., "1h" -> 1 hour)
        interval_hours = parse_interval_hours(cq.interval)
        start_time = end_time - timedelta(hours=interval_hours)

        logger.info(f"Executing CQ: {cq.name} ({start_time} to {end_time})")
        try:
            result = client.continuous_queries.execute(
                cq.id,
                start_time=start_time.isoformat() + "Z",
                end_time=end_time.isoformat() + "Z",
                dry_run=False,
            )
            logger.info(f"  Processed {result.records_read or 0} records")
        except Exception as e:
            logger.error(f"  Failed: {e}")

def parse_interval_hours(interval: str) -> int:
    """Parse interval string to hours (e.g., '1h' -> 1, '1d' -> 24)."""
    if interval.endswith("h"):
        return int(interval[:-1])
    elif interval.endswith("d"):
        return int(interval[:-1]) * 24
    elif interval.endswith("m"):
        return max(1, int(interval[:-1]) // 60)
    return 1

def main():
    import os

    with ArcClient(
        host=os.getenv("ARC_HOST", "localhost"),
        port=int(os.getenv("ARC_PORT", "8000")),
        token=os.getenv("ARC_TOKEN"),
    ) as client:
        logger.info("Starting scheduled Arc maintenance...")

        run_retention_policies(client)
        run_continuous_queries(client)

        logger.info("Scheduled maintenance complete")

if __name__ == "__main__":
    main()

Schedule with cron (run hourly):

# Edit crontab
crontab -e

# Add entry to run every hour
0 * * * * cd /path/to/project && ARC_TOKEN=your-token python arc_scheduler.py >> /var/log/arc_scheduler.log 2>&1

Using APScheduler

For more control, use APScheduler to run different tasks at different intervals:

arc_scheduler_advanced.py
#!/usr/bin/env python3
"""Advanced Arc scheduler with APScheduler."""

import os
import logging
from apscheduler.schedulers.blocking import BlockingScheduler
from arc_client import ArcClient

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

def get_client():
    return ArcClient(
        host=os.getenv("ARC_HOST", "localhost"),
        token=os.getenv("ARC_TOKEN"),
    )

def execute_hourly_cqs():
    """Run CQs with 1h interval."""
    logger.info("Running hourly CQs...")
    with get_client() as client:
        for cq in client.continuous_queries.list():
            if cq.is_active and cq.interval == "1h":
                try:
                    client.continuous_queries.execute(cq.id, dry_run=False)
                    logger.info(f"  Executed: {cq.name}")
                except Exception as e:
                    logger.error(f"  Failed {cq.name}: {e}")

def execute_daily_retention():
    """Run retention policies once per day."""
    logger.info("Running daily retention...")
    with get_client() as client:
        for policy in client.retention.list():
            if policy.is_active:
                try:
                    result = client.retention.execute(
                        policy.id, dry_run=False, confirm=True
                    )
                    logger.info(f"  {policy.name}: deleted {result.deleted_count} rows")
                except Exception as e:
                    logger.error(f"  Failed {policy.name}: {e}")

if __name__ == "__main__":
    scheduler = BlockingScheduler()

    # Run hourly CQs every hour at :05
    scheduler.add_job(execute_hourly_cqs, "cron", minute=5)

    # Run retention policies daily at 3:00 AM
    scheduler.add_job(execute_daily_retention, "cron", hour=3, minute=0)

    logger.info("Arc scheduler started. Press Ctrl+C to exit.")
    scheduler.start()

Install and run:

pip install apscheduler
python arc_scheduler_advanced.py

Docker deployment

Run the scheduler as a Docker container alongside Arc:

docker-compose.yml
services:
  arc:
    image: ghcr.io/basekick-labs/arc:latest
    ports:
      - "8000:8000"
    volumes:
      - arc-data:/app/data

  arc-scheduler:
    build:
      context: .
      dockerfile: Dockerfile.scheduler
    environment:
      - ARC_HOST=arc
      - ARC_PORT=8000
      - ARC_TOKEN=${ARC_TOKEN}
    depends_on:
      - arc
    restart: unless-stopped

volumes:
  arc-data:
Dockerfile.scheduler
FROM python:3.11-slim
WORKDIR /app
RUN pip install arc-tsdb-client apscheduler
COPY arc_scheduler_advanced.py .
CMD ["python", "arc_scheduler_advanced.py"]

Kubernetes CronJob

For Kubernetes deployments, use a CronJob:

arc-scheduler-cronjob.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: arc-maintenance
spec:
  schedule: "0 * * * *"  # Every hour
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: arc-scheduler
            image: python:3.11-slim
            command:
            - /bin/sh
            - -c
            - |
              pip install -q arc-tsdb-client
              python -c "
              from arc_client import ArcClient
              import os

              with ArcClient(
                  host=os.environ['ARC_HOST'],
                  token=os.environ['ARC_TOKEN']
              ) as client:
                  # Execute retention policies
                  for p in client.retention.list():
                      if p.is_active:
                          client.retention.execute(p.id, dry_run=False, confirm=True)

                  # Execute continuous queries
                  for cq in client.continuous_queries.list():
                      if cq.is_active:
                          client.continuous_queries.execute(cq.id, dry_run=False)
              "
            env:
            - name: ARC_HOST
              value: "arc-service"
            - name: ARC_TOKEN
              valueFrom:
                secretKeyRef:
                  name: arc-secrets
                  key: token
          restartPolicy: OnFailure

Apache Airflow

For complex workflows, use Airflow:

dags/arc_maintenance.py
from datetime import datetime, timedelta
from airflow import DAG
from airflow.operators.python import PythonOperator

default_args = {
    "owner": "data-team",
    "retries": 3,
    "retry_delay": timedelta(minutes=5),
}

def run_retention():
    from arc_client import ArcClient
    import os

    with ArcClient(host=os.getenv("ARC_HOST"), token=os.getenv("ARC_TOKEN")) as client:
        for policy in client.retention.list():
            if policy.is_active:
                client.retention.execute(policy.id, dry_run=False, confirm=True)

def run_hourly_cqs():
    from arc_client import ArcClient
    import os

    with ArcClient(host=os.getenv("ARC_HOST"), token=os.getenv("ARC_TOKEN")) as client:
        for cq in client.continuous_queries.list():
            if cq.is_active and cq.interval == "1h":
                client.continuous_queries.execute(cq.id, dry_run=False)

with DAG(
    "arc_maintenance",
    default_args=default_args,
    schedule_interval="@hourly",
    start_date=datetime(2024, 1, 1),
    catchup=False,
) as dag:

    retention_task = PythonOperator(
        task_id="run_retention",
        python_callable=run_retention,
    )

    cq_task = PythonOperator(
        task_id="run_hourly_cqs",
        python_callable=run_hourly_cqs,
    )

    retention_task >> cq_task

Next steps

On this page