Module batch 

Batching sink for logger plugins.

The featherbit analogue of APISIX’s batch processor (apisix/utils/batch-processor.lua): log entries produced on the request path are handed to a BatchSink with a fire-and-forget BatchSink::push and delivered downstream in batches by a background tokio task. A batch is flushed when it reaches batch_max_size, when no new entry has arrived for inactive_timeout, or when the oldest buffered entry is buffer_duration old — whichever comes first. Failed flushes are retried up to max_retry_count times, honouring the flusher’s first_fail hint so already-delivered entries are not re-sent (mirroring APISIX’s slice_batch).

push() never blocks and never awaits: entries go through a bounded mpsc channel and are dropped with a tracing::warn! when the channel is full (max_pending_entries). A rising drop rate is the operator’s signal to raise the capacity or fix the downstream.

One sink is created per logger node in a compiled policy graph. When the graph is recompiled the old sink is dropped and a new one spawned; dropping the sink closes the channel, the background task drains whatever is still buffered or queued, flushes it, and exits.

Structs

BatchConfig

Tuning knobs for a BatchSink, matching the field names (and defaults) of APISIX’s batch-processor schema plus featherbit’s channel capacity.

BatchSink

Handle to a spawned batching task. Cheap to clone; the background task flushes any remaining entries and exits when the last handle is dropped.

FlushError

Error returned by BatchFlusher::flush when a batch (or part of one) could not be delivered.

Traits

BatchFlusher

Delivers a batch of log entries downstream (HTTP endpoint, file, syslog, …). Implemented by each logger plugin; the batching, timing, and retry logic all live in BatchSink.

Functions

flush_with_retry 🔒

Flushes buffer, retrying per cfg on failure. Honours first_fail by dropping the delivered head and retrying only the tail. Entries arriving during a retry sleep simply queue in the channel and form the next batch. When retries are exhausted, the remaining entries are dropped with a tracing::error!.

int_key 🔒

Reads key from the config map as a non-negative integer. None (and JSON null) mean “absent, use the default”; any other non-integer value is an error.

run_loop 🔒

The background flush loop: buffers entries from rx and flushes on size, inactivity, age, or channel close (drain-on-drop).