GrindCore.net is built upon a unified abstraction layer that bridges high-level .NET Stream and block-based APIs with high-performance native compression libraries. The architecture is designed to handle multi-threaded operations, complex buffering requirements (such as LZMA2 sub-blocks), and transparent versioning across different native library releases.
The core of the library revolves around three primary entities that manage data flow and memory:
CompressionStream: The base class for all streaming operations. It handles the lifecycle of reading/writing, buffer management, and provides specialized logic for correcting stream positions when overreading occurs to fill native buffers src/CompressionStream.cs23-25CompressionBlock: The base class for "one-shot" or chunk-based compression. It defines the contract for algorithms that process a single block of data at a time, requiring a pre-determined output size src/CompressionBlock.cs17-18CompressionBuffer: A utility class used internally by both streams and blocks to manage byte arrays, tracking read/write positions (Pos, Size) and providing methods like Tidy() to reclaim space src/CompressionStream.cs92The following diagram illustrates how high-level concepts map to specific classes and interfaces within the codebase.
Core Architecture Mapping
Sources: src/CompressionStream.cs23-25 src/CompressionBlock.cs17-18 src/CompressionOptions.cs11-12 README.md61-66
The CompressionStream provides a robust implementation of the System.IO.Stream interface. It manages internal state such as _positionFullSize (uncompressed bytes) and _positionBase (bytes written to/read from the underlying stream) src/CompressionStream.cs124-126 It also includes an AsyncLocal wrapper to ensure that synchronous base streams can be safely used within asynchronous contexts src/CompressionStream.cs134-136 It provides advanced buffer management for precise stream rewinding when overreading occurs README.md48
For details, see CompressionStream Base Class.
Unlike streams, CompressionBlock implementations are stateless per operation. They are used when the entire payload is available in memory. Each implementation must define RequiredCompressOutputSize to allow the caller to allocate a sufficient destination buffer src/CompressionBlock.cs38 Algorithms like CopyBlock provide a pass-through implementation where this size equals the input block size src/Copy/CopyBlock.cs14-29
For details, see CompressionBlock Base Class.
The CompressionOptions class is the central point for configuring algorithm behavior, including ThreadCount, BlockSize, and CompressionType (e.g., Fastest vs. Optimal) src/CompressionOptions.cs16-38 It also supports CompressionVersion, allowing the library to dispatch calls to specific versions of native binaries (e.g., ZStd v1.5.2 vs v1.5.7) src/ZStd/ZStdStream.cs42-43 Advanced settings like WithLzmaDictionary allow fine-tuning of dictionary sizes and fast-bytes lookahead src/CompressionOptions.cs86-100
For details, see Configuration & Options System.
The library uses a factory pattern (CompressionStreamFactory and CompressionBlockFactory) to instantiate the correct concrete class based on a CompressionAlgorithm enum README.md64-67 This decouples the consumer from specific implementations like ZStdStream or Lz4Stream.
For details, see Factory Classes & Algorithm Registry.
Internal data handling relies on the CompressionBuffer, which manages a byte array and tracks AvailableRead and AvailableWrite windows src/Brotli/BrotliDecoder.cs100-101 Algorithms use these buffers to bridge managed memory with native pointers during P/Invoke calls src/Brotli/BrotliDecoder.cs109-117
For details, see Buffer Management & Utilities.
GrindCore.net relies on P/Invoke to execute high-performance C code. The Interop classes define the signatures for native functions across different platforms README.md103-115 It utilizes SafeHandle types, such as SafeBrotliEncoderHandle, to ensure native resources are properly released by the garbage collector src/Brotli/BrotliEncoder.cs18-42
For details, see Native Interop Layer.
The diagram below shows how data moves from a .NET consumer through the GrindCore abstractions into the native libraries.
Data Flow and Buffer Interaction
Sources: src/CompressionStream.cs46-53 src/ZStd/ZStdStream.cs140-155 src/Lz4/Lz4Stream.cs65-80
| Component | Responsibility | Key File(s) |
|---|---|---|
| Buffering | Aligning .NET small writes into large native blocks using BufferThreshold. | src/CompressionStream.cs |
| Position Tracking | Managing Position (compressed) vs PositionFullSize (uncompressed). | src/CompressionStream.cs |
| Versioning | Dispatched via CompressionVersion to specific native P/Invoke entry points. | src/Brotli/BrotliEncoder.cs |
| Resource Cleanup | Ensuring SafeHandle and OnDispose are called for native state. | src/CompressionBlock.cs, src/Brotli/BrotliDecoder.cs |
Sources: src/CompressionStream.cs123-126 src/Brotli/BrotliEncoder.cs34-45 src/CompressionBlock.cs94 src/Brotli/BrotliDecoder.cs49-53
Refresh this wiki