Fast-LZMA2 is a high-performance LZMA2 implementation provided by the GrindCore.net library, utilizing the FL2 native library. It supports multi-threaded compression and decompression, offering significantly improved performance over standard LZMA2 while maintaining high compression ratios.
The Fast-LZMA2 implementation bridges the managed .NET environment with native FL2 calls. It uses a streaming architecture that handles memory pinning and native context management.
The following diagram illustrates how managed classes in the Nanook.GrindCore.FastLzma2 namespace interact with the native Interop.FastLzma2 layer.
Managed to Native Interop Map
Sources: src/Fast-Lzma2/FastLzma2Encoder.cs57-60 src/Fast-Lzma2/FastLzma2Decoder.cs51-54 src/Fast-Lzma2/FastLzma2Block.cs45-46 src/Interop/Interop.FastLzma2.cs55-125
FastLzma2Stream acts as the primary entry point for users. It manages either a FastLzma2Encoder or FastLzma2Decoder depending on the CompressionMode.
The FastLzma2Encoder handles the complexity of native interaction:
CompressionParameters.Threads, it calls either FL2_createCStream (single-threaded) or FL2_createCStreamMt (multi-threaded) src/Fast-Lzma2/FastLzma2Encoder.cs57-60CompressionBuffer using GCHandle.Alloc(..., GCHandleType.Pinned) src/Fast-Lzma2/FastLzma2Encoder.cs76-82EncodeData method implements a loop that pushes source data into the native stream via FL2_compressStream and writes the resulting compressed output to the underlying stream using a provided delegate src/Fast-Lzma2/FastLzma2Encoder.cs126-156Encoder Data Flow
Sources: src/Fast-Lzma2/FastLzma2Encoder.cs126-160
FastLzma2Block provides a one-shot (non-streaming) compression and decompression implementation.
FL2_createCCtxMt and FL2_createDCtxMt src/Fast-Lzma2/FastLzma2Block.cs45-46_dictProp from the compression context, which is required for subsequent decoding src/Fast-Lzma2/FastLzma2Block.cs91RequiredCompressOutputSize is determined by calling the native FL2_compressBound function src/Fast-Lzma2/FastLzma2Block.cs93Sources: src/Fast-Lzma2/FastLzma2Block.cs11-133
Fast-LZMA2 exposes granular control through CompressionParameters and FL2Parameter.
| Class / Enum | Description |
|---|---|
CompressionParameters | A container for setting specific Fast-LZMA2 options like DictionarySize, Threads, and Strategy src/Fast-Lzma2/CompressionParameters.cs9-10 |
FL2Parameter | Enum defining native parameters such as OverlapFraction, HybridChainLog, and SearchDepth src/Fast-Lzma2/FL2Parameter.cs6-127 |
FL2Buffer structs | Native-compatible structures (FL2InBuffer, FL2OutBuffer) used for passing pointers and sizes to the P/Invoke layer src/Fast-Lzma2/FL2Buffer.cs1-76 |
When CompressionOptions are provided, the encoder automatically maps standard LZMA dictionary settings to Fast-LZMA2 parameters:
DictionarySize is capped at int.MaxValue src/Fast-Lzma2/FastLzma2Encoder.cs35-36Algorithm (0 for Fast, 1 for Normal) is mapped to Strategy (1 for Fast, 3 for Ultra) src/Fast-Lzma2/FastLzma2Encoder.cs50-54Sources: src/Fast-Lzma2/FastLzma2Encoder.cs26-55 src/Fast-Lzma2/CompressionParameters.cs41-197
The FastLzma2Decoder facilitates decompression, supporting multi-threaded operation if the stream was compressed with dictionary resets.
FL2_initDStream src/Fast-Lzma2/FastLzma2Decoder.cs56-58DecodeData method manages a do-while loop that calls FL2_decompressStream. It handles FL2ErrorCode.Buffer by refilling the input buffer from the underlying stream src/Fast-Lzma2/FastLzma2Decoder.cs104-153GCHandle to pin the input buffer for the duration of the decoder's life src/Fast-Lzma2/FastLzma2Decoder.cs63-69Sources: src/Fast-Lzma2/FastLzma2Decoder.cs46-70 src/Fast-Lzma2/FastLzma2Decoder.cs104-158
Native errors are handled by the FL2Exception class, which converts native return codes into managed exceptions.
FL2Exception.IsError(code) checks if a return value from a native call indicates a failure src/Fast-Lzma2/FL2Exception.cs110-117FL2ErrorCode enum covers common issues like CorruptionDetected, ChecksumWrong, and DstSizeTooSmall src/Fast-Lzma2/FL2Exception.cs9-48FL2ErrorCode.Buffer is often treated as a signal to flush or refill buffers rather than a fatal error src/Fast-Lzma2/FastLzma2Encoder.cs151-155 src/Fast-Lzma2/FastLzma2Decoder.cs131-136Sources: src/Fast-Lzma2/FL2Exception.cs53-146
Fast-LZMA2 is verified in Lzma2Tests.cs using known-good hashes and comparison against standard LZMA2 outputs.
XXHash64 to verify that data remains identical after a round-trip of compression and decompression tests/GrindCore.Tests/Lzma2Tests.cs93-96