The CompressionStream class is the primary abstract base for all streaming compression and decompression operations within GrindCore.net. It extends System.IO.Stream and provides a unified framework for buffer management, position tracking, and native interop lifecycle across multiple algorithms (ZLib, Brotli, LZMA, LZ4, ZStd, etc.).
CompressionStream manages the relationship between the .NET Stream API and the underlying native compression engines. It handles the complexities of partial reads/writes, internal buffering, and asynchronous task coordination.
The following diagram illustrates how the CompressionStream base class maps to specific implementation entities and their roles in the data pipeline.
CompressionStream Structure
Sources: src/CompressionStream.cs23-92 src/DeflateZLib/DeflateStream.cs10-14 src/ZStd/ZStdStream.cs15-19 src/Brotli/BrotliStream.cs10-14
Derived classes must implement the OnRead, OnWrite, and OnFlush methods to provide algorithm-specific logic.
| Method | Purpose | Key Responsibilities |
|---|---|---|
OnRead | Decompression | Consumes data from BaseStream via BaseRead, feeds it to the native decoder, and fills the caller's buffer. |
OnWrite | Compression | Accepts raw data, passes it through the native encoder, and writes results to BaseStream via BaseWrite. |
OnFlush | Finalization | Forces the native engine to emit pending bits/bytes (e.g., ZLib Z_FINISH or Brotli Flush). |
The OnRead implementation typically involves a loop that checks if the decoder needs more input or can produce more output.
Decompression Data Flow
Sources: src/DeflateZLib/DeflateStream.cs89-124 src/CompressionStream.cs281-300 src/Brotli/BrotliStream.cs109-130
GrindCore maintains multiple counters to ensure accurate reporting of stream progress, which is complicated by internal buffering and "overreading."
_position: Tracks the total number of bytes read from or written to the BaseStream src/CompressionStream.cs123_positionBase: Tracks the real amount read from or written to the base stream, used for limiting src/CompressionStream.cs124_positionFullSize: Tracks the total number of uncompressed bytes processed (the logical position of the stream) src/CompressionStream.cs125Position: Public property returning the uncompressed position (_positionFullSize) src/CompressionStream.cs156-157When decompressing, the library often reads more data from the BaseStream than the native engine consumes immediately to fill its internal CompressionBuffer. To allow the BaseStream to be used by other consumers after disposal, CompressionStream calculates "unused" bytes.
InternalBufferedBytes: Bytes held inside the native engine (e.g., ZLib's avail_in) src/CompressionStream.cs61 In DeflateStream, this accesses the AvailableInput of the inflater or deflater src/DeflateZLib/DeflateStream.cs33BufferedBytesUnused: Calculated as (Total - Used) + InternalBufferedBytes. This represents the exact amount the BaseStream must be rewound if it supports seeking src/CompressionStream.cs83Sources: src/CompressionStream.cs61-83 src/CompressionStream.cs123-157 src/DeflateZLib/DeflateStream.cs33
Each CompressionStream instance manages an internal CompressionBuffer.
BufferThreshold: The size at which the stream triggers a flush to the native engine during compression src/CompressionStream.cs53BufferSizeInput / BufferSizeOutput: Abstract properties defined by derived classes to optimize for specific algorithms. For example, BrotliStream uses 65520 bytes src/Brotli/BrotliStream.cs20-25 while Lz4Stream uses 2MiB for input src/Lz4/Lz4Stream.cs22RewindRead(int length): Allows moving the internal buffer's read position backward, effectively "un-reading" data. This also decrements _positionFullSize src/CompressionStream.cs102-106Sources: src/CompressionStream.cs49-106 src/ZStd/ZStdStream.cs25-30 src/Lz4/Lz4Stream.cs22-27 src/Brotli/BrotliStream.cs20-25
GrindCore provides deep integration with System.Threading.Tasks while maintaining compatibility with older .NET frameworks.
On modern runtimes, CompressionStream uses an AsyncLocal<bool> named _RunningOnAsyncWrapper src/CompressionStream.cs134-135 For older TFMs, a ThreadStatic shim is used src/CompressionStream.cs137-143 When an async operation is called (e.g., ReadAsync), it sets this flag. Lower-level BaseRead and BaseWrite calls check this flag to decide whether to invoke the BaseStream's synchronous or asynchronous methods, avoiding sync-over-async deadlocks src/CompressionStream.cs403-407
Instead of passing CancellationToken through every internal layer, GrindCore uses CancellableTask src/CompressionStream.cs89 This object wraps the token and provides ThrowIfCancellationRequested() to ensure cooperative cancellation during long-running native operations src/DeflateZLib/DeflateStream.cs95
Sources: src/CompressionStream.cs134-154 src/CompressionStream.cs281-285 src/CompressionStream.cs403-407 src/DeflateZLib/DeflateStream.cs95
The LeaveOpen property src/CompressionStream.cs31 determines if the BaseStream is closed when the CompressionStream is disposed.
LeaveOpen is false (default), BaseStream.Dispose() is called.LeaveOpen is true, the CompressionStream flushes its internal buffers and native state but leaves the BaseStream open src/CompressionStream.cs633-655BaseStream is seekable, the library attempts to rewind the BaseStream by the number of BufferedBytesUnused so the next consumer starts at the correct byte src/CompressionStream.cs647-650Sources: src/CompressionStream.cs31 src/CompressionStream.cs633-655
Refresh this wiki