Skip to main content

OTEL-056: filelog start_at:beginning without storage

Severity: warn (advisory)

Rule Details​

start_at: beginning tells filelog to read each new file from byte zero. Combined with no storage extension, every Collector restart re-reads every file — duplicating every log line already ingested since the file was created. In production this manifests as mysterious duplicate log spam after every rolling restart. Either use start_at: end or wire a persistent storage extension so filelog can checkpoint its read position.

This rule fires when a filelog receiver has start_at: beginning and no storage configured.

Options​

This rule has no options.

Examples​

Avoid
receivers:
  filelog:
    include: [/var/log/app/*.log]
    start_at: beginning             # no storage checkpoint
Prefer
extensions:
  file_storage/filelog:
    directory: /var/lib/otelcol/filelog

receivers:
  filelog:
    include: [/var/log/app/*.log]
    start_at: beginning
    storage: file_storage/filelog

service:
  extensions: [file_storage/filelog]

When Not To Use It​

Ephemeral containers where the file is guaranteed to exist only once and restart means the file is gone too — rare. In that case acknowledge the warning.

  • OTEL-057 — filelog overly broad include pattern
  • OTEL-065 — sending_queue without persistent storage
  • OTEL-066 — sending_queue.storage references undefined extension

Version​

Available since augur v0.1.0.

Further Reading​

Resources​