HTML Parser

Relevant source files

• .claude/docs/error-formatting.md
• .claude/docs/libxml2-parity.md
• .claude/docs/parser-internals.md

The html package provides a robust HTML 4.01 parser that produces a Helium DOM tree utilizing the same core structures as the XML parser. It addresses the idiosyncrasies of HTML parsing (including tag omission, entity references, and implicit element closure) while maintaining compatibility with Helium’s SAX-based architecture and DOM interfaces. This allows developers to apply XPath and XSLT capabilities seamlessly to HTML content.

---

Overview and Integration

The HTML parser is part of the Processing Layer within Helium's modular architecture, designed for parsing and tree building. The parser:

• Converts raw HTML input (byte slices or io.Reader) into a helium.Document with HTML-specific typings.
• Uses a SAX event-driven state machine specialized for HTML grammar and quirks.
• Supports a push parser model to consume chunked input data asynchronously.
• Integrates fully with Helium DOM and SAX2 event interfaces.

This integration enables complexity such as auto-closing elements and entity substitution while still producing node trees that conform to Helium’s common Node interface hierarchy.

Data Flow: HTML Input to DOM Tree

This flow highlights the event emission during HTML parsing and subsequent DOM tree construction which shares infrastructure with the XML parser pipeline.

Sources: `.claude/docs/parser-internals.md10-11 `.claude/docs/parser-internals.md21

---

The HTML Document Type in Helium

Helium models HTML documents distinctly to align with HTML semantics while reusing the core DOM abstractions.

Type	Implementation	Description
HTMLDocumentNode	Specialized Document subtype	Root node type representing an HTML document tree. Flags and behavior reflect HTML 4.01 specifics.
Node	Generic node interface	HTML elements, attributes, text, and comments all implement helium.Node, enabling uniform traversal and manipulation.

This typing enables queries, transformations, and serializations to behave correctly given the characteristics of HTML input.

Sources: `.claude/docs/libxml2-parity.md103

---

Parser Implementation Details

The HTML parser employs a parserCtx structure acting as a finite-state machine (FSM) driving the stateful lexical and syntactic analysis of HTML input. Parsing key points:

• Uses internal/strcursor.StrCursor for efficient navigation and context management.
• Implements HTML-specific parsing rules: implicit element closure, void elements, script/style content handling.
• Generates SAX2 events (StartElement, EndElement, Characters, etc.) routed to a sax.SAX2Handler.
• Compatible with HLTP-specific features like entity substitution tailored for HTML usage.
• Supports push parsing via PushParser, enabling incremental parsing in streaming scenarios.

Core API Overview

Function or Type	Purpose
Parse(ctx, []byte)	Entry point for HTML document parsing returning a DOM.
ParseReader(ctx, io.Reader)	Parses HTML data streamed from an io.Reader source.
NewPushParser(ctx)	Creates a PushParser for incremental chunk parsing.
parserCtx	Internal parsing context with current state and buffers.
PushParser	Wraps parserCtx to parse asynchronously with chunk input.

Sources: `.claude/docs/parser-internals.md10-11 `.claude/docs/libxml2-parity.md107

Parsing Approach

• State Machine: The parser context tracks the current parsing state to handle tag start/end, special element content (script/style), entity references, and whitespace normalization.
• SAX Event Emission: The parsing functions invoke registered SAX2 callback hooks to signal parser progress and build the DOM via helium.TreeBuilder.
• Entity Substitution: Distinguishes HTML-specific entities (e.g.,  , ©) and resolves them during parse.
• ML-based Content Parsing: Uses special handling for raw content scripts/styles blocks where typical parsing rules don’t apply.
• Push Parsing: The push parser runs the above in a separate goroutine, accepting data chunks through ParseChunk() and finalizing with Finish().

Sources: `.claude/docs/libxml2-parity.md103 `.claude/docs/parser-internals.md69-70

---

Error Handling and libxml2-Compatible Formatting

Error reporting in the HTML parser closely follows libxml2 conventions to meet golden output compatibility:

Error Output Format

Parsing errors are presented with precise location and context highlighting:

{filename}:{line}: HTML parser error : {message}
{source line snippet}
{spaces up to error column}^{caret}

Example:

./test/HTML/example.html:10: HTML parser error : Unexpected end tag : div
<p>Some invalid html</div>
                      ^

Implementation Notes

• Errors during SAX callbacks (e.g., in StartElement) are handled through handleSAXErr `.claude/docs/error-formatting.md101
• In non-strict mode (default), errors are forwarded as warnings (Warning(err)) allowing continued parsing `.claude/docs/error-formatting.md103-104
• In strict mode (Strict(true)), errors cause termination via parser.fatalSAXErr after parser stabilizes `.claude/docs/error-formatting.md105
• Context extraction of source lines and caret positioning uses a logic compatible with libxml2’s xmlParserInputGetWindow `.claude/docs/error-formatting.md91-92
• The error domain is tagged explicitly with "HTML parser error" to distinguish from other parser domains.

Error Handling Flow Diagram

Sources: `.claude/docs/error-formatting.md82-106

---

Push Parser Support

The package supports parsing streamed or chunked HTML input via the PushParser. Internally, it creates its own parserCtx and runs parsing in a separate goroutine.

Push Parser Workflow

This model supports parsing large or network streaming HTML in an incremental, non-blocking manner.

Sources: `.claude/docs/parser-internals.md10-11 `.claude/docs/libxml2-parity.md107

---

Usage Examples

Parsing HTML from Bytes

Parsing HTML from Reader

Incremental Parsing with PushParser

These examples demonstrate different ways of ingesting HTML content for DOM construction.

Sources: `.claude/docs/parser-internals.md5-6 `.claude/docs/parser-internals.md10-11

---

Summary

The html package plays a critical role in the Helium ecosystem by parsing HTML 4.01 documents into a fully navigable and transformable Helium DOM. It maintains fidelity to HTML parsing rules while leveraging Helium’s SAX and DOM infrastructure for consistency with XML processing. Its push parser model and libxml2-compatible error reporting make it suitable for demanding and production-level HTML processing situations.

Sources:

• `.claude/docs/libxml2-parity.md103-107
• `.claude/docs/error-formatting.md82-106
• `.claude/docs/parser-internals.md5-70