README
¶
2025 revival
For IEEE checksums AVX512 can be used to speed up CRC32 checksums by approximately 2x.
Castagnoli checksums (CRC32C) can also be computer with AVX512, but the performance gain is not as significant enough for the downsides of using it at this point.
crc32
This package is a drop-in replacement for the standard library hash/crc32 package,
that features AVX 512 optimizations on x64 platforms, for a 2x speedup for IEEE CRC32 checksums.
usage
Install using go get github.com/klauspost/crc32. This library is based on Go 1.24
Replace import "hash/crc32" with import "github.com/klauspost/crc32" and you are good to go.
changes
- 2025: Revived and updated to Go 1.24, with AVX 512 optimizations.
performance
AVX512 are enabled above 1KB input size. This rather high limit is due to AVX512 may be slower to ramp up than the regular SSE4 implementation for smaller inputs. This is not reflected in the benchmarks below.
| Benchmark | Old MB/s | New MB/s | Speedup |
|---|---|---|---|
| BenchmarkCRC32/poly=IEEE/size=512/align=0-32 | 17996.39 | 17969.94 | 1.00x |
| BenchmarkCRC32/poly=IEEE/size=512/align=1-32 | 18021.48 | 17945.55 | 1.00x |
| BenchmarkCRC32/poly=IEEE/size=1kB/align=0-32 | 19921.70 | 45613.77 | 2.29x |
| BenchmarkCRC32/poly=IEEE/size=1kB/align=1-32 | 19946.60 | 46819.09 | 2.35x |
| BenchmarkCRC32/poly=IEEE/size=4kB/align=0-32 | 21538.65 | 48600.93 | 2.26x |
| BenchmarkCRC32/poly=IEEE/size=4kB/align=1-32 | 21449.20 | 48477.84 | 2.26x |
| BenchmarkCRC32/poly=IEEE/size=32kB/align=0-32 | 21785.49 | 46013.10 | 2.11x |
| BenchmarkCRC32/poly=IEEE/size=32kB/align=1-32 | 21946.47 | 45954.10 | 2.09x |
cpu: AMD Ryzen 9 9950X 16-Core Processor
license
Standard Go license. See LICENSE for details.
Documentation
¶
Overview ¶
Package crc32 implements the 32-bit cyclic redundancy check, or CRC-32, checksum. See https://en.wikipedia.org/wiki/Cyclic_redundancy_check for information.
Polynomials are represented in LSB-first form also known as reversed representation.
See https://en.wikipedia.org/wiki/Mathematics_of_cyclic_redundancy_checks#Reversed_representations_and_reciprocal_polynomials for information.
Index ¶
Examples ¶
Constants ¶
const ( // IEEE is by far and away the most common CRC-32 polynomial. // Used by ethernet (IEEE 802.3), v.42, fddi, gzip, zip, png, ... IEEE = 0xedb88320 // Castagnoli's polynomial, used in iSCSI. // Has better error detection characteristics than IEEE. // https://dx.doi.org/10.1109/26.231911 Castagnoli = 0x82f63b78 // Koopman's polynomial. // Also has better error detection characteristics than IEEE. // https://dx.doi.org/10.1109/DSN.2002.1028931 Koopman = 0xeb31d82e )
Predefined polynomials.
const Size = 4
The size of a CRC-32 checksum in bytes.
Variables ¶
var IEEETable = simpleMakeTable(IEEE)
IEEETable is the table for the IEEE polynomial.
Functions ¶
func Checksum ¶
Checksum returns the CRC-32 checksum of data using the polynomial represented by the Table.
func ChecksumIEEE ¶
ChecksumIEEE returns the CRC-32 checksum of data using the IEEE polynomial.
func New ¶
New creates a new hash.Hash32 computing the CRC-32 checksum using the polynomial represented by the Table. Its Sum method will lay the value out in big-endian byte order. The returned Hash32 also implements encoding.BinaryMarshaler and encoding.BinaryUnmarshaler to marshal and unmarshal the internal state of the hash.
func NewIEEE ¶
NewIEEE creates a new hash.Hash32 computing the CRC-32 checksum using the IEEE polynomial. Its Sum method will lay the value out in big-endian byte order. The returned Hash32 also implements encoding.BinaryMarshaler and encoding.BinaryUnmarshaler to marshal and unmarshal the internal state of the hash.
Types ¶
type Table ¶
type Table [256]uint32
Table is a 256-word table representing the polynomial for efficient processing.
func MakeTable ¶
MakeTable returns a Table constructed from the specified polynomial. The contents of this Table must not be modified.
Example ¶
package main
import (
"fmt"
"github.com/klauspost/crc32"
)
func main() {
// In this package, the CRC polynomial is represented in reversed notation,
// or LSB-first representation.
//
// LSB-first representation is a hexadecimal number with n bits, in which the
// most significant bit represents the coefficient of x⁰ and the least significant
// bit represents the coefficient of xⁿ⁻¹ (the coefficient for xⁿ is implicit).
//
// For example, CRC32-Q, as defined by the following polynomial,
// x³²+ x³¹+ x²⁴+ x²²+ x¹⁶+ x¹⁴+ x⁸+ x⁷+ x⁵+ x³+ x¹+ x⁰
// has the reversed notation 0b11010101100000101000001010000001, so the value
// that should be passed to MakeTable is 0xD5828281.
crc32q := crc32.MakeTable(0xD5828281)
fmt.Printf("%08x\n", crc32.Checksum([]byte("Hello world"), crc32q))
}
Output: 2964d064
Source Files