Testing Infrastructure

Relevant source files

• .claude/docs/libxml2-parity.md
• .claude/docs/testing.md

The helium XML toolkit employs a comprehensive and layered testing infrastructure to guarantee correctness, compatibility, compliance, and robustness. This infrastructure integrates legacy compatibility testing with the libxml2 reference implementation, official W3C standards compliance tests, and modern fuzz testing to detect edge cases and robustness issues.

---

Testing Strategy Overview

The testing approach hinges on multiple complementary test suites and formats:

• Golden File Tests ensure bitwise or semantic equivalence against libxml2 outputs using large sets of curated XML test files.
• W3C Test Suite Compliance verifies standards conformance by automatically generating and running the official W3C XPath 3.1 (QT3) and XSLT 3.0 test collections.
• Fuzzing injects randomized and boundary inputs across all main packages to ensure resilience against malformed or adversarial data.

The testing pipeline orchestrates inputs from official sources and fuzz generators through various runners and validation layers.

Test Data Pipeline Overview

This diagram captures the transitioning of raw source test files and fuzz inputs into generated test code and runners, which exercise the relevant codebase components.

Sources: .claude/docs/testing.md3-40 .claude/docs/libxml2-parity.md3-41 .claude/docs/testing.md141-144

---

8.1 Golden File and libxml2-compat Tests

At the heart of compatibility validation is a suite of golden-file driven tests derived from libxml2's xmllint outputs:

• Test Data Layout: The full corpus is stored under testdata/libxml2-compat/, subdivided into domain-specific directories like c14n/, schemas/, relaxng/, schematron/, html/, and others. Each contains paired test inputs and expected outputs .claude/docs/testing.md10-26
• Golden File Formats: File extensions denote test artifacts:
  • .expected for DOM serialization roundtrips.
  • .sax2.expected for SAX2 event stream traces.
  • .err for validation or compilation error outputs.
  • .ser for serialized HTML outputs.
  • .xpath and .ns sidecar files facilitate XPath query tests for canonicalization.
  • .sax for HTML SAX event traces .claude/docs/testing.md30-39
• Golden File Generation: The generate.sh script copies from libxml2 source and normalizes its outputs. It applies fixes to SAX2 buffer artifacts (truncating displayed attribute values), merges consecutive SAX character events, and patches error messages to align with helium's parser-specific errors .claude/docs/testing.md43-47 Normalization helpers like mergeCharactersEvents internal/heliumtest/sax_helpers.go17-26 and newLibxml2EventEmitter internal/heliumtest/sax_helpers.go28-31 are used for this purpose.
• Environment Variable Filtering: To enable targeted runs, environment variables such as HELIUM_LIBXML2_TEST_FILES or HELIUM_HTML_TEST_FILES select subsets of tests, facilitating fast iteration during development .claude/docs/testing.md124-134
• Test Code Placement: Tests calling on these datasets reside in _test.go files per package, often employing external test packages to avoid breaking encapsulation .claude/docs/testing.md51-53

For exhaustive details, see Golden File and libxml2-compat Tests.

Sources: .claude/docs/testing.md5-47 .claude/docs/testing.md95-103 .claude/docs/testing.md124-134 internal/heliumtest/sax_helpers.go17-26 internal/heliumtest/sax_helpers.go28-31

---

8.2 W3C Test Suites (QT3 and XSLT 3.0)

The XPath 3.1 and XSLT 3.0 engines are verified against the authoritative W3C test suites:

• Generators: The large W3C collections (over 22,700 XPath and 13,100 XSLT tests) are consumed by tools/qt3gen/ and tools/xslt3gen/, which translate test XML catalogs into Go code files automating test case evaluation .claude/docs/libxml2-parity.md21-41 The fetch.sh script is used to retrieve these test suites.
• Test Execution: The generated suites are executed with runners such as qt3RunTests and xslt3RunTests. These runners configure contexts that address nuances like timezones, base URIs, and environment flags to mirror W3C testing conditions.
• Test Coverage & Skips: Detailed skip counts document unsupported scenarios, e.g., unavailable XQuery features, static typing, or legacy XSLT 2.0 behaviors. This informs focused enhancement efforts and accurate reporting .claude/docs/libxml2-parity.md25-69

For additional information, see W3C Test Suites (QT3 and XSLT 3.0).

Sources: .claude/docs/libxml2-parity.md21-69

---

8.3 Fuzzing

Robustness and security are enforced via fuzz testing integrated into CI:

• Fuzz Target Matrix: Fuzz harnesses are implemented across all major packages including c14n, catalog, html, relaxng, schematron, xpath1, xpath3, xsd, and xslt3 to cover parsing, evaluation, validation, and transformation code paths .claude/docs/testing.md141-144 The shim package is intentionally excluded from the fuzz matrix, and enum and sax are excluded from direct fuzzing due to their interface-only nature .claude/docs/testing.md145-146
• CI Integration: Two schedules ensure prompt and deep coverage:
  • fuzz-pr.yml performs smoke fuzzing on every pull request, delivering rapid feedback .claude/docs/testing.md153
  • fuzz.yml triggers detailed deep fuzzing weekly, exploring edge cases and rare failures .claude/docs/testing.md154
• Architectural Constraints: Fuzz targets must never panic or crash; instead, they gracefully return early on invalid or corrupt inputs, preserving CI stability and meaningful bug detection .claude/docs/testing.md149 Fuzz input sizes are bound early, and targets prefer in-memory stubs over filesystem/network access .claude/docs/testing.md147-148

See Fuzzing for detailed coverage and operational notes.

Sources: .claude/docs/testing.md141-156

---

Test Infrastructure Map

This table correlates testing responsibilities to code and data components across the project:

Testing Area	Test Runner / Generator	Test Data Location
DOM/SAX Parity	libxml2_compat_test.go	testdata/libxml2-compat/
XPath 3.1 Compliance	tools/qt3gen/ → qt3RunTests	W3C QT3 Suite files
XSLT 3.0 Compliance	tools/xslt3gen/ → xslt3RunTests	W3C XSLT 3.0 Suite files
Validation Engines	xsd_test.go, relaxng_test.go, schematron_test.go	testdata/libxml2-compat/schemas/ and subfolders
Fuzz Testing	fuzz_test.go in package dirs	Package-local fuzz corpora

Code Entity Association Diagram: Testing Runners and Helpers

This diagram links key test runners with their source generators and supporting helpers in the codebase.

Sources: .claude/docs/testing.md55-70 .claude/docs/testing.md95-112 .claude/docs/testing.md141-144

---

This page serves as the entry point to the helium test ecosystem. For in-depth technical details on each testing subsystem and tooling, consult the dedicated child pages:

• Golden File and libxml2-compat Tests
• W3C Test Suites (QT3 and XSLT 3.0)
• Fuzzing

Together, these suites enable the helium project to maintain high assurance levels of correctness, protocol compliance, and security robustness.