A drop-in fast path for standard base32 (StdEncoding, RFC 4648, padded),
byte-identical to encoding/base32 (and, on decode, with identical
CorruptInputError offsets). Both encoding and decoding run a SIMD kernel
generated by go-asmgen; the short tail,
padding and all error reporting reuse the standard library, so output matches
exactly.
s := base32.EncodeToString(data) // same bytes as encoding/base32.StdEncoding
b, err := base32.DecodeString(s) // same bytes AND same error offsets| op | amd64 | ppc64le | s390x | arm64 | loong64 / riscv64 |
|---|---|---|---|---|---|
| encode | AVX2 + SSE2 | VSX | vector facility | NEON on Go 1.27+, scalar on stable | scalar (stdlib) |
| decode | AVX2 + SSE2 | VSX | vector facility | scalar (stdlib) | scalar (stdlib) |
The encode fast path covers five ISAs across six architectures. ppc64le, s390x
and (on Go 1.27+) arm64 run the full kernel — the same algorithm amd64 uses —
because POWER (VSX), Z (vector facility) and NEON each provide the per-lane
variable shift / integer vector multiply the amd64 path relies on. On arm64 those
ops (VUMULL, VUSHL, VTBL) were only added to the Go assembler in Go
1.27, which is precisely what blocked the NEON port until now (see below); on
stable Go (≤ 1.26) arm64 encode falls back to encoding/base32. The ppc64le
and s390x kernels are qemu-validated (byte-and-error-identical to
encoding/base32 via the exhaustive + fuzz differential tests under QEMU
power9 / qemu); the arm64 NEON kernel is validated on native arm64 with the
gotip (1.27-devel) toolchain. ppc64le is now natively measured on real
POWER9 silicon (GCC Compile Farm, https://portal.cfarm.net/ , VSX, Go 1.26.4,
2026-06-26): SIMD decode runs ~5.5× the stdlib scalar decoder (621 vs 113
MB/s) — a real VSX kernel (VSRH) on hardware where arm64 stable can't run one.
s390x is now natively measured on real IBM z15 (VXE2) (2026-07-03,
-count=6): SIMD decode runs ~8.4× and encode ~3.4× the stdlib scalar
baseline.
Per 5-byte group → 8 chars (base32 packs 5-bit groups, so 5 input bytes = 40 bits = 8 output chars):
PSHUFBspreads the input into eight 16-bit lanes, each holding the big-endian 16-bit window that contains one output char's 5 bits.- A single per-lane unsigned high-multiply (
PMULHUW) by lane-specific powers of two shifts every char's 5-bit field down to bits[4:0];PAND 0x1fisolates it (value 0..31). - A second
PSHUFBpacks the eight values into the low 8 bytes, then a two-range ASCII map turns valuevinto its char —v<26 -> 'A'+v,v>=26 -> '2'+(v-26)— built asv + 65 - (PCMPGTB(v,25) & 41).
The AVX2 path processes two groups at once (one per 128-bit lane, placed via
VINSERTI128 at src+0 / src+5), 2×-unrolled, with a VPERMQ gather and a
full 16-byte store per group-pair. Constants come from go-asmgen's
emit.File.Data. Verified against encoding/base32 (table + exhaustive + fuzz,
on real AVX2 hardware).
Both run the same five-step kernel as amd64; only the spelling changes.
- ppc64le loads with
LXVB16X(big-endian element semantics: memory byte 0 = element 0's MSB), spreads withVPERM, and replacesPMULHUWwith a direct per-lane variable right shiftVSRH—field >> pis exactly what the multiply-by-2^(16-p)-then-take-high-word trick computes, and VSX exposes the variable shift natively. A secondVPERMpacks,VANDmasks, and the ASCII map usesVCMPGTUB/VADDUBM/VSUBUBM. VSX is baseline on POWER8+, so there is no runtime dispatch. (VSX↔VMX aliasing: load intoVS(32+k), do arithmetic onVk.) - s390x is the one shipped non-amd64 arch with a genuine vector integer
multiply-high (
VMLHH), so it reproduces the amd64PMULHUWstep almost instruction-for-instruction:VL→VPERMspread →VMLHHby2^(16-p)→VPERMpack →VNmask →VCHLB/VAB/VSBASCII map →VST. s390x is big-endian, butVL/VSTand theVPERMcontrol vectors are laid out in big-endian lane order (lane 0 = lowest address), which matches the amd64 byte layout because amd64's 16-bit windows are themselves big-endian — so no scan-direction reversal is needed. The cross-lane shuffle is pinned by a position-dependent qemu test. The vector facility is baseline on z13+.
Both kernels emit a full 16-byte store per 5-byte group (only the low 8 chars are
kept); the dispatcher caps the group count so the final store stays in bounds and
hands the tail to encoding/base32.
The NEON encode kernel is a faithful port of the amd64 five-step path, and is the
concrete demonstration of the three NEON ops the released Go arm64 assembler
does not expose but Go master / Go 1.27 does — the same gap ppc64le (VSRH)
and s390x (VMLHH) already worked around:
VUMULL/VUMULL2— the integer widening vector multiply (16×16 → 32). Released Go only assembled the polynomialVPMULL; the integer multiply mnemonics landed upstream in Go 1.27. The kernel multiplies each 16-bit window by2^(16-p)and keeps the high 16 bits (multiply-high == amd64PMULHUW).VUSHL— the per-lane register-variable shift (one count per lane, taken from a vector register), used to take the high half of each 32-bit product.VTBL— table-lookup permute, used twice: once to spread the 5-byte group into eight big-endian 16-bit windows, once as the 32-entry base32 alphabet LUT (a two-register table covering indices 0..31, replacing the two-range ASCII arithmetic the amd64 path uses).
The chain is VLD1 → VTBL spread → VUMULL/VUMULL2 → VUSHL (−16) → VXTN
→ VAND 0x1f → VTBL pack → VTBL alphabet → VST1. NEON is baseline on
arm64, so there is no runtime dispatch. The kernel is gated //go:build arm64 && go1.27; on stable Go (≤ 1.26) the build falls back to the scalar
encoding/base32 path (encode_generic.go). Validated byte-and-error-identical
to encoding/base32 (table + exhaustive + FuzzEncode) on native arm64 under
gotip, where it measures ~2.1× the stdlib scalar encoder.
Decode is the exact inverse of encode, per 8-char block → 5 bytes:
- Validate + map ASCII → 5-bit value, per lane, with two unsigned range
checks:
az = c in 'A'..'Z'andtw = c in '2'..'7'. On amd64 these use thePSUBUSB-then-compare-zero trick (PCMPGTBis signed); ppc64le usesVCMPGTUB+VNOR, s390x usesVCHLB+VNO.value = ((c-65) & az) | ((c-24) & tw); lanes matching neither range are invalid. - Spread the eight values into the low byte of each 16-bit lane, then a
per-lane left shift by
pplaces each value into its big-endian 16-bit output window — the inverse of the encoder's right shift. amd64 usesPMULLW(multiply low word by2^p), s390x usesVMLHW(vector multiply low, halfword), ppc64le uses the native variable shiftVSLH. - Three
PSHUFB/VPERMgathers + OR scatter each window's high/low byte into the five output bytes (each output byte takes up to three overlapping contributions).
A block is decoded only if all eight chars are valid; the kernel counts the
blocks it consumes and stops before the first block containing a non-alphabet
char, and always reserves the final block. The remainder — the final block,
any padding (=), the short tail, and everything from the first invalid char —
is handed to encoding/base32, with the CorruptInputError offset shifted back
over the SIMD-consumed prefix. This keeps decode byte-and-offset identical to
base32.StdEncoding.Decode (RFC 4648 padding + error semantics) while still
vectorising the common all-valid bulk. The AVX2 path decodes two blocks at once,
one per 128-bit lane. Validated against encoding/base32 (table + exhaustive +
fuzz) on real AVX2 hardware and under QEMU power9 / qemu for VSX / Z.
There is no pre-existing pure-Go SIMD base32 to compare against
(encoding/base32 ships zero assembly; pkg.go.dev / GitHub turn up only
alternative-alphabet, Crockford, or CLI packages — none vectorised). So the
comparison is purely this package vs the stdlib scalar encoder.
Encode throughput, 1 MiB buffer, native amd64 (GitHub Actions, AMD EPYC,
GOAMD64=v1, -count=6, median MB/s; CI-measured because the dev box is arm64):
| implementation | kind | MB/s | vs stdlib |
|---|---|---|---|
encoding/base32 (stdlib) |
scalar | ~1055 | 1.0× |
| this package (SSE2/SSSE3) | pure-Go SIMD | ~5430 | ~5.1× |
| this package (AVX2) | pure-Go SIMD | ~8285 | ~7.9× |
(A local emulated QEMU Haswell VM showed only ~1.4×, but QEMU's TCG does not model out-of-order execution — the native figures above are representative.)
Ratio re-confirmation (as of 2026-06-14): re-benched -count=6 medians on a
QEMU x86_64 lima VM (so absolutes are TCG-low, but the SSE-vs-stdlib ratio
holds): forced-SSE ~243 MB/s vs stdlib ~181 MB/s = ~1.34× — SIMD still beats
the scalar encoder even on the worst-case in-order TCG model. (Forced-AVX2 is
~parity under TCG, ~185 MB/s: 32-byte ops gain nothing without OoO execution —
exactly why the native EPYC numbers above are the representative ones.) On
native arm64 under gotip / Go 1.27 the NEON encode kernel engages and
measures ~7400 MB/s vs the stdlib scalar encoder ~3530 MB/s ≈ 2.1× (-count
medians on an Apple-silicon dev box; see the arm64 section above). On stable Go
(≤ 1.26) arm64 encode is an alias of encoding/base32 (stdlib parity by
construction). Verdict: SIMD encode wins on amd64 and on arm64/go1.27+; arm64 on
stable Go = stdlib fallback.
The speedup is somewhat below base64's because base32's 5-bit grouping forces an 8-byte store per 5-byte group (vs base64's full 16-byte store per 12 bytes) and a longer serial extract chain — inherent to the format.
Decode throughput (1 MiB, AVX2 path) was re-benched -count=4 on the same
QEMU x86_64 lima VM (TCG, in-order — absolutes are low, but the SIMD-vs-stdlib
ratio holds): ~108 MB/s vs encoding/base32 ~36 MB/s ≈ 3×, and the SIMD
decode is allocation-free (0 B/op) where the stdlib path allocates per call in
this harness. Native EPYC numbers (representative, with out-of-order execution)
are pending a CI bench run; expect a higher ratio there, as with encode. On
native arm64 / loong64 / riscv64, decode is an alias of encoding/base32
(stdlib parity by construction).
- ppc64le: full SIMD encode (above), now natively measured on real POWER9 (GCC Compile Farm, VSX, Go 1.26.4, 2026-06-26): SIMD decode ~5.5× the stdlib scalar decoder (621 vs 113 MB/s), a real VSX kernel where arm64 stable can't run one.
- s390x: full SIMD encode (above), now natively measured on real IBM z15
(VXE2) (2026-07-03,
-count=6): SIMD decode ~8.4× and encode ~3.4× the stdlib scalar baseline. POWER and Z supply the per-lane variable shift / multiply-high natively, so they run the whole kernel — and as of Go 1.27 arm64 NEON does too (the ops it was missing finally landed upstream). - arm64 (Go 1.27+): full NEON SIMD encode (above), validated on native
arm64 under
gotip, ~2.1× the stdlib scalar encoder. - riscv64: now natively measured on a real SpacemiT X60 (RVV 1.0, GCC
Compile Farm, Go 1.26.4, 2026-06-26). Honest result: scalar parity —
decode 27.4 vs stdlib 27.6 MB/s, encode 155 vs 155 (encode falls back
to
encoding/base32here, and decode is an alias of the stdlib decoder). No RVV win on this low-power, in-order core (currently the only widely-available RVV 1.0 silicon); the SIMD encode wins stay on amd64, ppc64le and arm64 (1.27).
Static analysis, NOT a hardware measurement — retained for historical
comparison. Both non-amd64 SIMD arches here are now natively measured: ppc64le
on real POWER9 silicon (GCC Compile Farm, VSX, Go 1.26.4, 2026-06-26 — SIMD
decode ~5.5× the stdlib scalar decoder, 621 vs 113 MB/s) and s390x on real IBM
z15 (VXE2), 2026-07-03, -count=6 — SIMD decode ~8.4×, encode ~3.4× the stdlib
scalar baseline. The cycle-model rows below are superseded by those hardware
numbers and kept only for reference.
The committed inner loops (one 16-byte vector iteration emits 8 base32 chars from
5 input bytes) were extracted from encode_ppc64le.s / encode_s390x.s and fed
to llvm-mca (LLVM 22, which has production PowerPC and SystemZ backends):
llvm-mca -mtriple=powerpc64le-unknown-linux-gnu -mcpu=pwr9 <loop.s>
llvm-mca -mtriple=s390x-unknown-linux-gnu -mcpu=z14 <loop.s>
A representative scalar base32 inner loop processing the identical 5-in/8-out group (per-char shift → mask → table load → store) was modelled the same way for an apples-to-apples ×scalar ratio:
| arch (cpu) | SIMD loop Block RThroughput |
scalar loop Block RThroughput |
est. ×scalar | est. out-bytes/cycle |
|---|---|---|---|---|
| ppc64le (pwr9) | ~5.3 cyc/iter | ~7.5 cyc/iter | ~1.4× | ~1.5 |
| s390x (z14, est.) | ~4.0 cyc/iter | ~13.0 cyc/iter | ~3.25× (measured z15: decode ~8.4×, encode ~3.4×) | ~2.0 |
Caveats: Block RThroughput is a steady-state throughput ceiling (no front-end,
cache, branch-mispredict or dependency-stall modelling); the scalar baseline is a
hand-modelled idealised loop (table-driven, no bounds checks), so the real Go
scalar fallback would be slower — i.e. these ×scalar figures are conservative
lower bounds for the SIMD win. z14's wider vector pipeline + the single-shot
VMLHH multiply (vs POWER's VSRH variable-shift dependency chain) is why Z
models meaningfully faster than POWER here. Every instruction in both loops is
modelled by llvm-mca (no unmodelable op). Treat these as ordering/ballpark
estimates only. The ppc64le estimate has now been superseded by a native POWER9
measurement (above), and the s390x estimate by the native IBM z15 (VXE2)
measurement (2026-07-03): SIMD decode ~8.4×, encode ~3.4× the stdlib scalar
baseline.
- arm64: on stable Go (≤ 1.26) encode falls back to
encoding/base32— the per-char 5-bit fields need a per-lane variable shift and an integer vector multiply, and the released Go arm64 assembler exposed neither a register-formVUSHLnor the integerVUMULL/VMUL, so the multiply-shift trick could not be expressed. Those mnemonics were upstreamed in Go 1.27, so on Go 1.27+ a//go:build arm64 && go1.27NEON kernel runs the full encode (above). The exact ops ppc64le (VSRH) and s390x (VMLHH) used to work around the gap are now natively available on arm64 too. - loong64 / riscv64: encode falls back to
encoding/base32. - decode runs the same SIMD coverage on amd64 (AVX2+SSE), ppc64le (VSX) and
s390x (vector facility); arm64/loong64/riscv64 use the scalar stdlib decode
(there is no NEON decode kernel — the arm64 fast path is encode-only). Error and
padding semantics stay exactly identical to the stdlib because every block that
isn't a full 8 valid-alphabet chars (and the final block) is delegated to
encoding/base32, withCorruptInputErroroffsets shifted to match.
Six SIMD targets, validated on seven architectures. SIMD acceleration stays on
the six targets above (amd64, ppc64le, s390x, arm64, loong64, riscv64). On top of
those, a seventh architecture — ppc64 (big-endian) — is now build+test
validated on real POWER9 silicon (GCC Compile Farm): the portable/generic
fallback path is proven bit-exact on a big-endian target distinct from s390x's
vector kernel. ppc64 carries no SIMD kernel; it exercises the scalar
encoding/base32 fallback, confirming byte-identical output on a big-endian
non-vector arch.
The CI gate enforces 100% coverage of the Go code on each arch job: native
amd64 + native arm64 (stable), a native arm64 / gotip (Go 1.27-devel) job
that compiles and covers the //go:build arm64 && go1.27 NEON kernel, plus
emulated ppc64le + s390x (cross-compiled test binary run under QEMU power9 /
qemu in a debian:trixie container, coverage profile extracted from the
binary). On stable arm64 the
!amd64 && !ppc64le && !s390x && !(arm64 && go1.27) generic fallback compiles and
is measured; on the gotip job the NEON path is measured instead. Coverage is of
the Go statements only: the generated .s SIMD kernels are not measured by go test -cover — they are validated by differential tests against the scalar
encoding/base32 reference plus fuzzing (FuzzEncode/FuzzDecode, run on the
gotip job so the NEON kernel is fuzzed too).
BSD-3-Clause.
