logfwd
A high-performance log forwarder built in Rust. Tails log files, parses JSON and Kubernetes CRI format with portable SIMD, transforms every batch with DataFusion SQL, and ships to OTLP, Elasticsearch, Loki, HTTP, or stdout.
log files → SIMD parse → Arrow RecordBatch → SQL transform → output
logfwd is a single static binary with no runtime dependencies. Point it at log files, write a SQL query to filter and reshape the data, and forward the results to any OTLP-compatible collector — or directly to Elasticsearch, Loki, or stdout. SQL transforms are the core idea: instead of learning a vendor-specific DSL, you write standard SQL to control exactly what gets shipped.
Get started
- Installation — Binary download, Docker, or build from source
- Quick Start — Working pipeline in 60 seconds, no external dependencies
- Your First Pipeline — Production config with monitoring and validation
Configure
- Configuration Reference — All YAML fields, input/output types, enrichment
- Input Types — File, TCP, UDP, OTLP receiver, generator
- Output Types — OTLP, HTTP, Elasticsearch, Loki, stdout
- SQL Transforms — Filter, reshape, extract — full DataFusion SQL
Deploy
- Kubernetes DaemonSet — Manifest, resource sizing, CRI format
- Docker — Container images and compose files
- Monitoring — Diagnostics API, Prometheus metrics, health checks
Learn more
- Pipeline Design — How data flows from input to output
- Scanner — How the parser works
- Performance — Benchmarks and tuning
- Troubleshooting — Common errors and how to fix them