flate: use Peek instead of ReadByte for the bufio.Reader decode path

[joechenrh]

Problem

huffmanBufioReader fills the bit buffer with fr.ReadByte() per input byte. bufio.Reader.ReadByte doesn't inline, so each byte is a call barrier that spills the fill loop's registers — ~1.85× slower than the *bytes.Reader variant, whose ReadByte inlines to a slice index.

Change

Read via Peek instead: index bufio's buffered slice directly (pbuf[pos]), refilling when drained. Peek only views the buffer, so consumed bytes are Discarded before every return, keeping the reader positioned for nextBlock/moreBits/the gzip trailer. peekBufio refills from at most one Read (not Peek(Size()), which would block a sync writer).

Only the *bufio.Reader variant changes (in _gen/gen_inflate.go); the other four stay byte-identical.

Validation

• flate/gzip/zip suites + -race pass on linux/amd64, linux/386, darwin/arm64.
• FuzzInflateBufio (forces the bufio path, byte-exact round-trip): ~110k execs, no mismatch.

Numbers

*bufio.Reader decode, 8 MiB literal-heavy (median of 3):

arch	before	after
linux/amd64	128 MB/s	151 MB/s (~1.18×)
darwin/arm64	219 MB/s	287 MB/s (~1.31×)

Match-heavy data ~1.05–1.07× (huffman-table-load bound, unchanged).

Summary by CodeRabbit

• Bug Fixes

  • Improved compressed-data decompression when input arrives in very small or irregular buffered reads.
  • Ensured decompressed output remains accurate across buffered-reader boundary conditions and error scenarios.

• Tests

  • Added fuzz testing covering compressed and decompressed data across multiple compression levels and reader behaviors.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 50af2aed-2536-404b-a019-758f26bf7332

📥 Commits

Reviewing files that changed from the base of the PR and between 496a8b9 and 21318cf.

⛔ Files ignored due to path filters (1)

• flate/_gen/gen_inflate.go is excluded by !**/_gen/**

📒 Files selected for processing (1)

• flate/inflate_gen.go

💤 Files with no reviewable changes (1)

• flate/inflate_gen.go

---

📝 Walkthrough

Walkthrough

The flate bufio Huffman decoder now consumes buffered input through Peek and Discard, preserving state across refills and errors. A fuzz test exercises this behavior with multiple reader boundaries, compression levels, seeded inputs, and a size limit.

Changes

Bufio Inflate Decoding

Layer / File(s)	Summary
Peek-based Huffman decoding flate/inflate_gen.go	Adds peekBufio and updates huffmanBufioReader to decode from peeked bytes, discard consumed input, and maintain bit and error state across refill paths.
Boundary-condition round-trip validation flate/inflate_bufio_test.go	Adds round-trip checks for one-byte, half-buffer, and bufio readers across compression levels, plus seeded fuzzing with inputs capped at 1<<18 bytes.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant HuffmanDecoder
  participant BufioReader
  participant CompressedInput
  HuffmanDecoder->>BufioReader: Peek buffered bytes
  BufioReader->>CompressedInput: Refill when needed
  HuffmanDecoder->>BufioReader: Discard consumed bytes
  HuffmanDecoder->>HuffmanDecoder: Decode Huffman symbols

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: switching the flate bufio.Reader decode path from ReadByte to Peek.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[klauspost]

Did you investigate other types?

*bytes.Reader doesn't have it directly, but could be tricked into giving the backing bytes with WriteTo - which in it's current implementation will send all bytes as a single write. Seek can be used to forward.

*bytes.Buffer is less obvious. You can of course easily get the underlying bytes, but forwarding it cheaply without doing a Reset (which could break callers) seems more tricky.

Of course if ReadByte is inlined it is probably not worth pursuing.

[joechenrh]

Yes — I benchmarked each reader type with a flate-style fill loop (concrete type per func, mirroring the codegen), MB/s:

reader	fill loop
*bufio.Reader (ReadByte call)	~465
interface Reader	~490
*bytes.Reader (ReadByte inlined)	~895
*bufio.Reader + Peek (this PR)	~955

The bytes types are already ~2× faster than bufio — -gcflags=-m confirms inlining call to bytes.(*Reader).ReadByte, while bufio.(*Reader).ReadByte isn't inlinable at all. I haven't implemented a backing-slice variant for them, so I won't put a number on what WriteTo/Seek would actually buy.

huffmanGenericReader is the other slow one (~490), but I think the doc contract rules it out: "If r does not also implement io.ByteReader, the decompressor may read more data than necessary" — for a ByteReader we promise not to over-read, so buffering or peeking it would break that.

That's also why this PR stays on *bufio.Reader: peeking bytes it has already buffered reads nothing extra — peekBufio's Peek(1) triggers exactly the fill ReadByte would have, and Peek(Buffered()) reads none. Read-neutral, which is what the sync tests lean on.