This proposal aims to introduce DQL (DOM Query Language) into XGo, a universal DOM query syntax. DQL provides a unified query interface applicable to various structured data formats and tree structures.

DQL is designed as a universal query language, applicable to but not limited to:

• JSON
• YAML
• HTML
• XML
• Go AST
• XGo AST
• File System
• Other tree-structured data

The language involves the following core concepts:

• NodeSet: Node set (ns NodeSet). Both input and output of Query operations are NodeSet. The initial query node set typically contains only the root node, referred to as doc NodeSet.
• Child: Child nodes. Nodes form a tree structure through parent-child relationships. Child nodes can be accessed via ns.name syntax. Node names may contain -, accessed via ns."elem-name". Wildcard syntax (ns.*) selects any child node. Use ns.**.name to query nodes named name at any level within the ns subtree. Index access ns[index] for list/array nodes (e.g., users[0] for the first user).
• Select: Node filtering. Syntax: ns@cond. Example: user@($age < 18) filters all users under 18 years old. Conditions can be:
  • fn(args) // Function call returning bool
  • (condExpr) // Parenthesized expression
  • name or "elem-name" // Pass node name to select only nodes with that name
• Attr: Attributes (data carried by the node itself). Two versions available (similar to map[index]):
  • Dual-value: val, err := ns.$name - explicit error handling
  • Single-value: val := ns.$name - defaults to zero value on error (equivalent to $name?:zeroValue)
  • Also supports error operators: $attr! (panic if err), $attr?:defVal (custom default)
  • Returns the attribute value from the first node in the NodeSet
  • For multiple nodes, use list comprehension: [n.$attr for n in ns]
  • Attribute names may contain -, accessed via ns.$"attr-name"
• Method: Methods (general mechanism for accessing node data beyond attributes). Two versions typically available:
  • Dual-value: result, err := ns.method(args) - explicit error handling
  • Single-value: result := ns.method(args) - defaults to zero value on error
  • Exception: Methods returning numeric types or other values where defaulting to zero could cause subtle bugs (e.g., age < 18 might incorrectly pass when age errors to 0) provide only the dual-value version
  • General syntax: ns._method(args) or ns.method(args) (in domain-specific contexts)
  • Methods operate on the first node in the NodeSet
  • For multiple nodes, use list comprehension: [n.method(args) for n in ns]
  • Examples include text, value, count, int, etc.
  • If a child node name starts with _, use quotes: node."_text" (child node) vs node._text (method)
• Cache and Performance: Standard cache control methods for optimizing lazy query evaluation:
  • _all / all: Materialize and cache all nodes. Use when the NodeSet will be accessed multiple times.
  • _one / one: Assume single-result query, stop at first match. Returns NodeSet marked with ErrNotFound if no match. Use when you expect one result and want early termination.
  • _single / single: Validate exactly one match. Returns NodeSet marked with ErrNotFound (0 matches) or ErrMultipleEntities (>1 matches). Use for uniqueness validation.

Definition: ns NodeSet

• Both input and output of Query operations are NodeSet
• Initially, the node set typically contains only the root node (referred to as doc NodeSet)
• All query operations are performed on node sets

Syntax:

• Basic access: ns.name

  • Access child nodes through parent-child relationships

• Names with hyphens: ns."elem-name"

  • Use quotes when node names contain -

• Index access: ns[index]

  • Access child nodes by index for list/array nodes using bracket notation
  • Example: users[0] represents the first user (0-based indexing)
  • Similar semantics to name-based access, but for ordered collections

• Wildcard: ns.*

  • Select any child node

• Deep query: ns.**.name

  • Query nodes named name at any level within the ns subtree

Notes:

• Nodes form a tree structure through parent-child relationships
• Node names may contain - characters
• List/array nodes use index-based access with bracket notation and 0-based indexing

Syntax: ns@cond

Example:

user@($age < 18)

Filters all users under 18 years old

Condition Types:

1. Function call: fn(args)

   • Function call that returns bool

2. Expression: (condExpr)

   • Conditional expression wrapped in parentheses

3. Name matching: name or "elem-name"

   • Pass node name instead of a bool expression
   • Selects only nodes with the corresponding name

Syntax: ns.$name

Two-Version Design (similar to map[index]):

DQL provides two versions for attribute access:

1. Dual-value version: val, err := ns.$attr

   • Explicitly returns both value and error
   • Allows manual error handling

2. Single-value version: val := ns.$attr

   • Returns only the value (equivalent to ns.$attr?:zeroValue)
   • Defaults to the zero value of the attribute's type if error occurs
   • Convenient for cases where errors can be safely ignored

Semantics:

• Attribute access operates on the first node in the NodeSet ns
• If ns contains multiple nodes, only the first node is considered
• To get attributes from all nodes in the set, use list comprehension

Error Handling Operators: In addition to the two standard versions, XGo's error handling operators can be used:

• $attr! - Panic if error occurs
• $attr?:defVal - Return custom default value if error occurs

Examples:

// Single-value version (defaults to zero value on error)
name := doc.users.*.$name

// Dual-value version (explicit error handling)
name, err := doc.users.*.$name
if err != nil {
    // handle error
}

// With error operators
name := doc.users.*.$name!           // Panics on error
name := doc.users.*.$name?:"Unknown" // Custom default value

// Get name attributes of all users
names := [user.$name for user in doc.users.*]              // Uses zero value as default
names := [user.$name! for user in doc.users.*]             // Panics on any error
names := [user.$name?:"N/A" for user in doc.users.*]       // Custom default for errors

Notes:

• Access data carried by the node itself
• Attribute names may contain -, use quotes in this case: ns.$"attr-name"
• Attributes are node metadata
• Always operates on the first node in the NodeSet
• The single-value version provides convenience similar to Go's map[key] access pattern

Syntax: ns._method(args) (general form) or ns.method(args) (in specific DOMs)

Two-Version Design:

Most methods provide two versions for result access:

1. Dual-value version: result, err := ns.method(args)

   • Explicitly returns both result and error
   • Allows manual error handling

2. Single-value version: result := ns.method(args)

   • Returns only the result (equivalent to ns.method(args)?:zeroValue)
   • Defaults to the zero value of the result's type if error occurs
   • Convenient for cases where errors can be safely ignored

Important Exception - Methods Returning Numeric or Sensitive Types:

Some methods only provide the dual-value version to prevent subtle bugs:

// Methods returning integers, floats, or other types where zero-default is dangerous
val, err := ns.int      // ✓ Correct - explicit error handling required
val := ns.int           // ✗ Error - single-value version not available
val := ns.int?:0        // ✓ Correct - explicit default value required

// Why? To prevent silent failures like:
age := ns.int           // If this silently returned 0 on error...
if age < 18 {           // ...this condition would incorrectly pass!
    // Wrong logic executed
}

// Correct approaches:
age, err := ns.int
if err != nil {
    // handle missing/invalid age
}

age := ns.int?:100      // Or use explicit safe default
if age < 18 {
    // Now logic is correct
}

Semantics:

• Methods operate on the first node in the NodeSet ns
• If ns contains multiple nodes, only the first node's method is called
• To call methods on all nodes, use list comprehension: [n.method(args) for n in ns]

Notes:

• Methods are a general mechanism for accessing node data beyond attributes
• In the general form, all methods are accessed with the _method syntax pattern
• Names starting with _ are reserved for method access in the general case
• Domain-specific optimization: In specific DOMs where method names don't conflict with child node names, the _ prefix can be omitted
  • Example: In HTML DOM, use node.text instead of node._text to access text content
  • This makes queries more natural and concise when the context is clear
• Combined with XGo's feature of omitting () in function calls, text or _text becomes a special case (equivalent to Text() or XGo_text())
• Common methods: text, value, count, int, float, bool, etc. (or _text, _value, _count, _int, _float, _bool in general form)

Special Cases:

• If a child node name starts with _, quotes are required for access
• Example: node."_text" represents a child node named _text
• While node._text represents calling the _text method on the node
• Method calls can omit parentheses: node.text is equivalent to node.Text()
• The choice between method and _method depends on the specific DOM implementation

NodeSet Error State:

Before discussing cache methods, it's important to understand that NodeSet carries error state internally:

• A NodeSet can be marked with an error (e.g., ErrNotFound, ErrMultipleEntities)
• When a NodeSet has an internal error, all $attr and method calls on it return that error
• This design eliminates the need for dual-value versions of cache methods (_all, _one, _single)
• Errors propagate through the query pipeline naturally

// NodeSet with internal error
ns := doc.users.*@($id == 999)._one  // Returns NodeSet marked with ErrNotFound

// Error propagates to attribute access
name, err := ns.$name  // err == ErrNotFound
age := ns.$age!        // Panics with ErrNotFound

// Can still use the NodeSet in normal ways
for user in ns {  // Loop doesn't execute (empty due to error)
    // ...
}

Lazy Evaluation Background:

DQL queries use lazy evaluation - a NodeSet doesn't immediately execute the query operations applied to it. Instead, it stores a query plan that is executed only when the nodes are actually accessed (e.g., when iterating, accessing attributes, or calling methods).

Benefits of Lazy Evaluation:

• Memory efficient: NodeSets are lightweight query descriptors, not materialized node collections
• Composition friendly: Complex queries can be built incrementally without overhead
• Optimization potential: Query engines can analyze and optimize the entire query plan before execution

The Trade-off:

When a NodeSet is used multiple times, the entire query chain is re-executed each time:

users := doc.users.*@($age > 18)

// Each of these causes a full re-execution of the query
names := [u.$name for u in users]    // Execution #1: scan, filter, iterate
ages := [u.$age for u in users]      // Execution #2: scan, filter, iterate
emails := [u.$email for u in users]  // Execution #3: scan, filter, iterate

For complex queries (especially deep queries with **) or large datasets, this repeated execution can become a performance bottleneck.

Standard Cache Control Methods:

To optimize query performance, DQL provides three standard methods for cache control and query termination. These methods are available in two forms:

• General form: _all, _one, _single
• Domain-specific form: all, one, single (when no naming conflicts exist)

cached := ns._all  // or ns.all in domain-specific contexts

Purpose: Execute the query immediately, materialize all matching nodes, and cache the results in the returned NodeSet.

Semantics:

• Returns a new NodeSet containing all nodes that match the query
• The returned NodeSet is fully materialized - subsequent operations on it don't re-execute the original query
• The original NodeSet ns remains lazy; only the returned NodeSet is cached
• No dual-value version needed - NodeSet carries error state internally

When to use:

• The NodeSet will be traversed or accessed multiple times
• Multiple attributes/methods will be extracted from the same set of nodes
• The query is expensive (e.g., deep queries with **) and results are reused
• You're working with a reasonably sized result set that fits comfortably in memory

Examples:

// Without cache - inefficient for multiple accesses
users := doc.users.*@($active == true)
for name in [u.$name for u in users] {      // Query execution #1
    echo(name)
}
for email in [u.$email for u in users] {    // Query execution #2
    sendEmail(email)
}

// With cache - query executes once
users := doc.users.*@($active == true)._all
for name in [u.$name for u in users] {      // Uses cached nodes
    echo(name)
}
for email in [u.$email for u in users] {    // Uses cached nodes
    sendEmail(email)
}

// Cache expensive deep queries
targets := doc.**.*@hasClass("widget")._all
// Now can efficiently access multiple attributes
ids := [t.$id for t in targets]
values := [t.$value for t in targets]

result := ns._one  // or ns.one in domain-specific contexts

Purpose: Optimize queries when you have prior knowledge that the result should contain only one node, allowing early termination after finding the first match.

Semantics:

• Returns a NodeSet (not just the first node) that can be used like any other NodeSet
• Acts as a cache operation similar to _all, but stops query evaluation after finding the first match
• Important: This is based on the assumption/expectation that there is exactly one matching node
• If no nodes match: returns a NodeSet marked internally with ErrNotFound error
• If multiple nodes match: returns the first match (does not validate uniqueness - that's _single's job)
• The returned NodeSet can be used in all normal ways (attribute access, method calls, iteration, etc.)
• Error propagation: If the NodeSet has an internal error (like ErrNotFound), all $attr and method calls on it will return that error

When to use:

• You have prior knowledge/expectation that only one node matches
• Performance optimization for queries where you know the result should be singular
• Early exit from expensive deep queries once a match is found
• Existence checks where you expect at most one match

Examples:

// Assumption: there's only one admin (or maybe zero)
admin := doc.users.*@($role == "admin")._one
name := admin.$name      // If _one returned ErrNotFound, this returns that error
email := admin.$email    // Same - error propagates through

// Error handling through attribute access
name, err := admin.$name
if err == ErrNotFound {
    echo("No admin user found")
}

// Or using error operators
name := admin.$name!              // Panics if ErrNotFound
name := admin.$name?:"Default"    // Uses default if ErrNotFound

// Existence check
target := doc.**.*@hasClass("error")._one
location := target.$location      // Returns ErrNotFound if no error element found
if location, err := target.$location; err == nil {
    echo("First error found at:", location)
}

// Performance benefit: stops searching after first match
config := doc.**.settings@($key == "api_url")._one
value := config.$value  // Only scanned until first match, not entire tree

Key Point - Not a Validator: _one is a performance optimization based on your expectation, not a validator:

// If there are actually 3 admin users:
admin := doc.users.*@($role == "admin")._one
// Returns the first admin (stops after finding it)
// Does NOT error for the other 2 admins
// Use _single if you need to validate uniqueness

unique := ns._single  // or ns.single in domain-specific contexts

Purpose: Validate that the query matches exactly one node, returning a NodeSet marked with an error if the uniqueness constraint is violated.

Semantics:

• Returns a NodeSet (not just a node) that can be used like any other NodeSet
• Acts as both a validator and cache operation
• Validates the uniqueness constraint by scanning enough to determine if there are 0, 1, or >1 matches
• If query matches exactly one node: returns a NodeSet containing that node
• If query matches zero nodes: returns a NodeSet marked internally with ErrNotFound error
• If query matches multiple nodes: returns a NodeSet marked internally with ErrMultipleEntities error
• The returned NodeSet can still be used (attributes, methods, iteration), but carries the error state
• Error propagation: If the NodeSet has an internal error, all $attr and method calls on it will return that error

When to use:

• You require exactly one match and want to validate this constraint
• Working with unique identifiers (IDs, primary keys, etc.)
• Validating data integrity and consistency
• Preventing silent logic errors from unexpected multiple matches
• Configuration or settings where exactly one value must exist

Examples:

// Database-like unique ID lookup
user := doc.users.*@($id == 12345)._single
name := user.$name  // If _single found 0 or >1 users, this returns the error

// Error handling through attribute access
name, err := user.$name
if err == ErrNotFound {
    return errors.New("user not found")
} else if err == ErrMultipleEntities {
    return errors.New("data corruption: duplicate user IDs")
}

// Using panic operator for cases where violation is a programming error
config := doc.config.*@($key == "api_endpoint")._single
endpoint := config.$value!  // Panics if ErrNotFound or ErrMultipleEntities

// Ensure exactly one primary configuration exists
primary := doc.settings.*@($primary == true)._single
value, err := primary.$value
if err != nil {
    log.Fatal("Configuration error: ", err)
}

// Validation in data processing
func getUniqueOwner(doc NodeSet) (owner string, err error) {
    ownerNode := doc.metadata.*@($role == "owner")._single
    return ownerNode.$name  // Returns (name, err) - err could be ErrNotFound or ErrMultipleEntities
}

Comparison with _one:

// _one: assumes single result, stops at first match (no validation)
admin := doc.users.*@($role == "admin")._one
name := admin.$name  // If 3 admins exist: returns first admin's name

// _single: validates single result, checks for multiple
admin := doc.users.*@($role == "admin")._single
name, err := admin.$name  // If 3 admins exist: err == ErrMultipleEntities

Why it returns NodeSet, not just error: The returned NodeSet can be used directly in pipelines:

// Can chain operations - errors propagate through
value := doc.config.*@($key == key)._single.$value
// If _single found 0 or >1 configs, $value returns that error

// Or handle errors at the end
value, err := doc.config.*@($key == key)._single.$value
if err != nil {
    // Handle ErrNotFound or ErrMultipleEntities
}

// Using error operators
value := doc.config.*@($key == key)._single.$value!
value := doc.config.*@($key == key)._single.$value?:"default"

Performance and Usage Guidelines:

1. Choose based on your expectations and needs:

   // Expect/assume single result, want performance → _one
   admin := doc.users.*@($role == "admin")._one
   name := admin.$name  // Stops at first match
   
   // Require exactly one result, need validation → _single
   admin := doc.users.*@($role == "admin")._single!
   name := admin.$name  // Errors if 0 or multiple found
   
   // Need all results, will use multiple times → _all
   admins := doc.users.*@($role == "admin")._all
   processNames([a.$name for a in admins])
   processEmails([a.$email for a in admins])

2. Understanding the differences:

   // Scenario: 3 admin users exist in the data
   
   // No cache: repeated query execution
   admins := doc.users.*@($role == "admin")
   // Uses first admin each time, but re-executes query
   
   // _one: assumes single, stops early, returns first
   admin := doc.users.*@($role == "admin")._one
   // Returns first admin, no error (doesn't check for others)
   
   // _single: validates uniqueness
   admin := doc.users.*@($role == "admin")._single
   // Returns NodeSet with ErrMultipleEntities
   
   // _all: caches all results
   admins := doc.users.*@($role == "admin")._all
   // Caches all 3 admins

3. Error handling patterns:

   // Error propagates through NodeSet to attribute/method access
   result := doc.config.*@($key == key)._one
   value, err := result.$value
   if err == ErrNotFound {
       // Handle missing case
   }
   
   // Or handle at the attribute access point
   value, err := doc.config.*@($key == key)._one.$value
   if err == ErrNotFound {
       // Handle missing case
   }
   
   // _single with full validation
   result := doc.config.*@($key == key)._single
   value, err := result.$value
   if err == ErrNotFound {
       // Handle missing case
   } else if err == ErrMultipleEntities {
       // Handle duplicate case (data integrity issue)
   }
   
   // Using error operators when violations are bugs
   config := doc.config.*@($key == key)._single
   value := config.$value!  // Panics if ErrNotFound or ErrMultipleEntities

4. Chain cache methods at the end of query pipeline:

   // ✓ Good - apply filters first, then cache
   results := doc.users.*@($age > 18)@($country == "US")._all
   results := doc.users.*@($age > 18)@($country == "US")._one
   results := doc.users.*@($age > 18)@($country == "US")._single
   
   // ✗ Less efficient - caches too early
   results := doc.users.*._all@($age > 18)@($country == "US")

5. Memory vs CPU trade-offs:

   // Small result sets: _all is usually fine
   admins := doc.users.*@($role == "admin")._all  // Probably <10 users
   
   // Large result sets with uniqueness requirement: _single
   user := doc.users.*@($id == userId)._single!  // Validates, doesn't cache all
   
   // Large result sets, need first: _one
   firstMatch := doc.products.*@($price < 100)._one  // Stops early
   
   // For large sets used once: no cache needed
   for user in doc.users.* {  // Stream processing, no cache
       process(user)
   }

Performance Summary:

*Scans enough to verify uniqueness (finds at most 2 nodes to distinguish between 1 and >1)
**Stops after finding 2 nodes (to detect multiple), doesn't scan entire set

Best Practices:

• Use lazy evaluation (no cache) for single-pass processing
• Use _all when the same NodeSet is accessed multiple times
• Use _one for existence checks and first-match scenarios
• Use _single when uniqueness is a requirement, not just an optimization
• Consider memory constraints when caching large result sets
• Profile your queries to identify performance bottlenecks before optimizing

*Note: Single-value method syntax is not available for methods returning numeric or other sensitive types to prevent subtle bugs.

// Select all users
doc.users.*

// Select first user by index
doc.users[0]

// Get first user's name
name := doc.users.*.$name                    // Single-value (defaults to zero value)
name, err := doc.users.*.$name               // Dual-value (explicit error)
name := doc.users.*.$name!                   // Panic on error
name := doc.users.*.$name?:"Unknown"         // Custom default

// Get third user's name specifically
name := doc.users[2].$name

// Get all users' names (list comprehension)
names := [user.$name for user in doc.users.*]              // Uses zero value as default
names := [user.$name! for user in doc.users.*]             // Panics on any error
names := [user.$name?:"N/A" for user in doc.users.*]       // Custom default

// Get user age (numeric - must use dual-value or error operator)
age, err := doc.users.*.$age.int             // ✓ Correct
age := doc.users.*.$age.int                  // ✗ Error - not available
age := doc.users.*.$age.int?:0               // ✓ Correct - explicit default

// Select users under 18 (safe because we use explicit default)
young_users := doc.users.*@($age.int?:100 < 18)

// Select users named john
johns := doc.users.*@($name == "john")

// Get ages of all young users
ages := [user.$age.int?:0 for user in doc.users.*@($age.int?:100 < 18)]

// Select all div elements
doc.**.div

// Get text content via method
text := doc.body.div.text                    // Single-value (first div only)
text, err := doc.body.div.text               // Dual-value (first div only)
texts := [div.text for div in doc.body.div]  // All divs (with zero value default)

// Get attribute from first matching element
href := doc.**.a.$href                       // Single-value (defaults to zero value)
href, err := doc.**.a.$href                  // Dual-value (explicit error)
href := doc.**.a.$href!                      // Panic on error
href := doc.**.a.$href?:"#"                  // Custom default

// Get attributes from all matching elements
hrefs := [a.$href for a in doc.**.a]         // Uses zero value as default
hrefs := [a.$href! for a in doc.**.a]        // Panics on any error
hrefs := [a.$href?:"#" for a in doc.**.a]    // Uses "#" as default

// Call method (operates on first node only)
parent := doc.**.a.parentN(3)

// Count method (numeric - must use dual-value or error operator)
count, err := doc.**.div.count               // ✓ Correct
count := doc.**.div.count                    // ✗ Error - not available
count := doc.**.div.count?:0                 // ✓ Correct - explicit default

// In HTML DOM, these work without _ prefix since there's no conflict

// List all files
root.**.file

// List all directories
root.**.dir

// Find all .go files
root.**.file@match("*.go", $name)

// Get names of all .go files
names := [f.$name for f in root.**.file@match("*.go", $name)]
names := [f.$name! for f in root.**.file@match("*.go", $name)]
names := [f.$name?:"unknown" for f in root.**.file@match("*.go", $name)]

// Get file sizes (numeric - requires explicit handling)
size, err := root.**.file.$size.int
size := root.**.file.$size.int?:0
sizes := [f.$size.int?:0 for f in root.**.file]

1. Type Safety: Ensure type safety of query results
2. Performance Optimization:
   • Implement lazy evaluation for memory efficiency
   • Optimize deep queries and large node sets
   • Provide cache control methods (_all, _one, _single) for performance tuning
   • Support early termination in _one queries
   • Consider query plan optimization before execution
3. Error Handling:
   • Provide two versions for most attribute and method accesses
   • For methods returning numeric or sensitive types, provide only dual-value version to prevent subtle bugs
   • Integrate seamlessly with XGo's error handling operators (! and ?:)
   • Provide clear error messages and debugging support
   • Define standard error types (ErrNotFound, ErrMultipleEntities) for _single
4. Extensibility: Support custom functions and operators
5. Compatibility: Integration with existing XGo syntax and toolchain
6. Domain-Specific Adaptations: Allow DOM implementations to optimize method syntax (e.g., using text instead of _text in HTML DOM when there's no naming conflict)
7. First-node Semantics: All data access operations (attributes and methods) operate exclusively on the first node in the NodeSet. This provides consistent, predictable behavior and requires explicit iteration (e.g., list comprehension) when multi-node access is needed.
8. Error Propagation: Ensure proper error propagation in list comprehensions and nested queries
9. Version Safety: Clearly document which methods provide single-value versions and which don't, to prevent runtime errors and subtle bugs
10. Cache Strategy:
    • Implement efficient caching mechanisms for _all
    • Ensure _one truly terminates early in query execution
    • Validate uniqueness constraints efficiently in _single
    • Consider memory limits and provide guidelines for large datasets

DQL provides XGo with a unified, powerful, and flexible query interface that simplifies operations on various structured data, improving development efficiency and code readability.

Key design features:

• Two-version design for attributes and methods (similar to Go's map[index]) provides both convenience and safety, while selective availability of single-value versions for numeric methods prevents subtle bugs
• First-node semantics for all data access operations ensures predictable behavior and encourages explicit handling of multi-node scenarios through list comprehensions
• Lazy evaluation minimizes memory overhead while enabling efficient query composition
• Cache control methods (_all, _one, _single) give developers fine-grained control over performance trade-offs, supporting use cases from streaming processing to batch operations with validation

The combination of expressive syntax, performance optimization options, and safety-focused error handling makes DQL suitable for a wide range of structured data processing tasks across different domains.