Batch, Streaming, and Latency

Bounded and unbounded data

The first question to ask about a pipeline is not which tool to use. It is whether the input ever ends.

Batch processing operates on a bounded collection: a finite dataset or a group accumulated for a defined period or volume. The job can inspect the complete batch, make several passes, and finish. This suits work such as end-of-day reporting, payroll, backfills, and large repetitive transformations that do not require an immediate response.

Stream processing operates on an unbounded collection that continues to receive events. Logs, sensor readings, clicks, and transactions are common event streams.

One clarification saves a lot of confusion later. A stream can be micro-batched internally and still remain logically unbounded.

An engine may collect arriving events for a fraction of a second and process them as a small group, because that is efficient. That is an implementation detail of how the work is executed. It does not change the shape of the source: no final record is coming, so the computation is never "done" in the way an end-of-day batch is done.

Judge the input by whether it terminates, not by the size of the chunk the engine happens to process.

Why an unbounded stream needs windows

Because the complete dataset never arrives, streaming computations incrementally update results and often use finite windows over the flow.

Consider the difference between two questions. "What is total revenue?" asked of a stream has no final answer, because the next event could always change it. "What was revenue in the last five minutes?" does have an answer, because the window supplies the ending that the source refuses to.

A window is therefore not a performance trick. It is the device that turns an endless flow into a group with a boundary, so an aggregation can be computed and emitted. Windows say nothing about arrival order, and they do not free the pipeline from holding state — the running totals for open windows have to live somewhere until those windows close.

Latency, throughput, and what low latency costs

Batch systems can process large volumes efficiently and schedule compute when resources are readily available. Their tradeoff is waiting for the collection window and job completion. Batch latency is commonly measured in minutes, hours, or days, while streaming targets results in seconds or milliseconds.

Low latency is not free. Streaming keeps producers, brokers, consumers, state, monitoring, and recovery mechanisms operating as data arrives. It must handle bursts, out-of-order events, consumer lag, and a source whose final size is unknown. Batch scheduling and recovery are usually easier to reason about for delay-tolerant work.

Throughput and latency are separate measures, and confusing them leads to the wrong fix.

Throughput is how much work completes per unit of time. Latency is how long one item waits for its result. A system can process many records per second yet still build a long queue, or return each event quickly at a lower total rate.

That is why "we handle 200,000 records a second" is not an answer to "why is this alert seven minutes old". The first is a rate; the second is a delay. A backlog can grow steadily while throughput stays impressively high, because arrivals are simply faster still.

Choosing between the two

Choose batch when efficient high-volume completion matters more than immediate action; choose streaming when the value of a result declines with delay.

That is the whole decision rule, and it is about the value of the answer rather than the fashion of the tool. A payroll run is worth exactly as much at 03:00 as at 22:00, so the cheaper, simpler, delay-tolerant path wins. A fraud decision is worth a great deal in two seconds and almost nothing in two hours.

Hybrid designs can use streaming for timely signals and batch for deep history or reprocessing. The same events can feed a live alerting path and a nightly job that rebuilds the full history with corrections applied.

Apply it to a fraud service. Transactions arrive continuously, a decision is needed within seconds, and occasional bursts must not cost the system any events.

Streaming is forced by the first two facts: the source never ends and the deadline is seconds. The third fact sets the cost of admission. Durable buffering absorbs the bursts, recovery state lets the pipeline resume without losing its place, consumers scale out when the rate climbs, and monitoring exposes consumer lag before it becomes a missed decision.

The alternatives fail on the stated facts. A nightly batch over the complete day answers long after the reaction window has closed. A streaming consumer without durable buffering is exactly the design that a burst breaks — low latency does not prevent traffic spikes, it makes them more damaging.

Event time and processing time

Once results depend on when things happened, a pipeline is carrying two clocks and has to keep them apart.

Event time is the timestamp associated with when an event occurred. Processing time is when a pipeline stage actually processes that event.

Network delay, retries, and buffering mean these times can differ, and events may arrive out of order. A sale made at 10:00 on a phone with no signal may reach the pipeline at 10:07, after a retry, and land behind a sale made at 10:05.

Windows, watermarks, and late data

Event-time windows group records by occurrence time and preserve the intended business period. A sale made at 10:00 counts in the 10:00 window even if it is handled at 10:07, so the report says what actually happened in that period.

A watermark expresses when the system expects most data for a window to have arrived; later records may be treated as late according to the configured policy. It is the pipeline's declared patience: hold the window open long enough for the stragglers that are worth waiting for, then finalize.

Processing-time triggers instead decide output based on the pipeline clock, which is the right choice when the question really is about system handling — how much the pipeline got through in the last minute.

The correct clock follows the question: business occurrence or system handling. If events can arrive ten minutes late, results must be grouped by when they happened, and early figures may be revised, then event-time windows with a watermark that tolerates the required lateness are the fit.

Delivery guarantees

Failures create a choice between loss and repetition. A consumer that dies between doing the work and recording that it did the work leaves the system genuinely unsure, and each guarantee resolves that doubt differently.

At-most-once delivery avoids redelivery but can lose a record when failure occurs before durable processing. At-least-once delivery retries uncertain work, reducing loss risk while allowing duplicates. Exactly-once processing coordinates state and progress so each input affects the managed result once.

Checkpoints store processing state and help recovery maintain that guarantee within supported sources and sinks — they are how a restarted job knows what it had already accounted for.

Idempotent sinks

End-to-end behavior includes the sink. Some operations provide at-least-once delivery, and retries can repeat a write. Whether that repetition matters is decided by what happens at the destination.

An idempotent sink produces the same business result when the same input is applied more than once. A stable event key, merge, or deduplication rule can prevent repeated delivery from creating duplicate business records: the second arrival of key INV-4471 updates the row it already wrote instead of inserting a second one.

Acknowledgements and durable brokers also reduce loss, but they do not make a non-idempotent side effect safe from repetition. A blind insert is still a second row, and a second row on a billing table is a second bill.

Put the two halves together on a billing pipeline. The connector may retry a micro-batch after an uncertain failure, and a lost bill is far less acceptable than a repeated delivery attempt.

The loss preference settles the delivery mode: at-least-once, because retrying is what keeps a bill from vanishing. That choice creates the duplicate exposure, so the sink has to absorb it — a stable billing key plus a merge that updates the existing row rather than inserting a new one.

The failing designs each break one half. Disabling redelivery protects against duplicates by accepting exactly the loss the requirement rules out. Keeping at-least-once while inserting a fresh row per retry bills the customer twice. Overwriting the event's own timestamp with the retry time destroys the business record while doing nothing about the duplicate.

Key takeaways

• Batch consumes bounded data; streaming continuously processes unbounded data. Micro-batching inside an engine does not make an endless source bounded.
• Use streaming for low-latency action and batch for delay-tolerant, high-volume efficiency. Low latency is paid for in continuously operated state, recovery, and monitoring.
• Throughput and latency are separate: a high rate of completed work can sit alongside a long queue.
• Event time records occurrence; processing time records pipeline handling. Event-time windows plus a watermark preserve the business period while tolerating late arrivals.
• At-least-once needs idempotent handling to control duplicates, while weaker redelivery can increase loss risk. Stable keys and merges make a retry harmless.