Data Streaming

The Problem

Some CDP operations produce serious payloads:

Loading these into memory would crash PEN or eat the host alive. So PEN streams everything to temp files.

Heap Snapshot Streaming

CDP delivers heap snapshots as a stream of HeapProfiler.addHeapSnapshotChunk events — each one carries a string chunk of JSON.

Memory stays flat no matter how big the heap gets — chunks go straight to disk. A 2 GB hard cap protects the host: if the snapshot exceeds this limit, PEN aborts the capture, deletes the temp file, and returns a clear error explaining the limit.

The CDP event listener for addHeapSnapshotChunk is registered on a cancelable child context. When the handler returns, the child context is cancelled and the listener is automatically removed — preventing listener leaks across repeated calls.

Trace Streaming

Chrome traces use Tracing.start with transferMode: "ReturnAsStream" and optional gzip.

Buffer Management

Chrome has a finite trace buffer. PEN watches Tracing.bufferUsage events during capture. If percentFull > 0.9, PEN fires a progress warning. If the buffer fills completely, the trace gets truncated and PEN notes it in the output.

A 500 MB hard cap is enforced during trace streaming. If the accumulated data exceeds this limit, PEN stops reading, deletes the temp file, and returns a size-exceeded error. The trace completion listener uses the same cancelable child context pattern as heap snapshots, ensuring it is cleaned up after each call.

Temp Files

All temp files live under os.TempDir()/pen/:

• Created with 0600 permissions (owner-only read/write)
• Directory created with 0700 permissions
• Path validated via security.ValidateTempPath before any read/write
• Cleaned up on normal exit via defer cleanupTempDir(logger) in main.go
• Cleaned up on context cancellation via defer in each handler

Each temp file has a prefix indicating its type (heap-, trace-, etc.) and a unique suffix generated by os.CreateTemp.

Progress Notifications

Long-running tools send MCP progress notifications so the client knows things are moving:

server.NotifyProgress(ctx, req, bytesWritten, totalBytes, "streaming heap snapshot...")

Fires only if the client sent a progress token in the request. When no progress token is available (or the session is nil), PEN falls back to logging the progress via slog.Debug — so streaming activity is always observable, even in stateless mode or when the client doesn't support progress notifications. The percentage calculation is guarded against division by zero when total is 0.

Data Flow Diagram