-
Notifications
You must be signed in to change notification settings - Fork 566
Domain‐Specific Text Literals
XGo's domain-specific text literals provide a powerful way to embed specialized languages directly into your code with full syntax highlighting and type safety. This feature bridges the gap between general-purpose programming and domain-specific needs, making your code more expressive and maintainable.
Domain-specific text literals allow you to write inline code in specialized formats—such as JSON, XML, regular expressions, or custom DSLs—without sacrificing the benefits of compile-time checking and editor support.
Basic syntax:
result := domainTag`content`With parameters:
result := domainTag`> param1, param2
content
`The ! suffix forces error handling, causing a panic if parsing fails—useful for literals you expect to always be valid.
This syntax is inspired by Markdown's code blocks. Just as Markdown uses triple backticks with a language identifier ( ```json) to denote code blocks in a specific language, XGo's domain-specific literals use a similar pattern—a tag followed by backticks—to embed domain-specific content directly in your code. This familiar syntax makes the feature intuitive for developers already comfortable with Markdown while bringing the same clarity and language-specific semantics to your programming workflow.
- Type Safety: Catch errors at compile time rather than runtime
- Syntax Highlighting: Full editor support for embedded languages
- Readability: Keep domain-specific code inline where it's used
- Maintainability: Easier to update and refactor than string concatenation
- Tooling Support: Enables semantic understanding by XGo tools like formatters and IDEs
XGo natively supports the following domain text literals. Except for tpl, all built-in DTLs are located under the encoding directory, with the directory name matching the DTL name.
A grammar-based alternative to regular expressions that emphasizes clarity and composability. Ideal for defining parsers and text processors.
grammar := tpl`
expr = term % ("+" | "-")
term = INT % ("*" | "/")
`!
result := grammar.parseExpr("10+5*2", nil)
echo resultLearn more in the TPL documentation.
Parse and validate JSON structures inline. The result is a DOM; accessing a known field by name is direct child navigation:
config := json`{
"server": "localhost",
"port": 8080,
"features": ["auth", "logging"]
}`!
echo config.server
echo config.portParse YAML documents inline. The result is a DOM with the same navigation model as JSON:
cfg := yaml`
server:
host: localhost
port: 8080
`!
echo cfg.server.host
echo cfg.server.portParse XML documents inline. The result is a DOM. Element names are navigated as direct children; text content is read via ._text and attributes via .$name:
doc := xml`
<configuration>
<database>
<host>localhost</host>
<port>5432</port>
</database>
</configuration>
`!
// Navigate a known path and read text content
host := doc.configuration.database.host._text
port := doc.configuration.database.port._text
echo host, portDefine tabular data inline. CSV has a simple, flat structure and does not support DQL queries:
data := csv`
name,age,city
Alice,30,NYC
Bob,25,SF
`!
for row in data.rows {
echo row.name, row.age, row.city
}Parse HTML documents inline. No external import is required. The result is a DOM. HTML element names form a fixed, well-known set, so methods are called without the _ prefix. Attributes are still accessed with $:
page := html`
Welcome
Domain-specific literals in action
XGo
Hello, XGo!
`!
// Navigate a known path and read text content
title := page.html.body.h1.text
echo title
// Read an attribute of a known element
href := page.html.body.a.$href
klass := page.html.body.p.$class
echo href, klassDefine regex patterns with improved readability. XGo supports both standard and POSIX regex:
pattern := regexp`^[a-z]+\[[0-9]+\]$`!
if pattern.matchString("item[42]") {
echo "Match found"
}
// Extract submatches
matches := regexp`(\w+)@(\w+)\.(\w+)`!.findStringSubmatch("user@example.com")
echo matches
// POSIX variant
posixPattern := regexposix`[[:alpha:]]+`!
words := posixPattern.findAllString("hello world 123", -1)
echo wordsParse Go source code into an AST DOM. Go AST node types are a well-known fixed set, so methods are called without the _ prefix:
src := golang`
package main
import (
"fmt"
"os"
)
func greet(name string) string {
return fmt.Sprintf("Hello, %s!", name)
}
func main() {
fmt.Println(greet(os.Args[1]))
}
`!Parse XGo source code into an AST DOM. Like the Go AST, XGo AST node types are a fixed set, so methods are called without the _ prefix:
src := xgo`
x, y := "Hi", 123
echo x
print y
`!Represent a directory as a queryable NodeSet. Unlike other DTLs, fs produces a NodeSet directly rather than a single-root DOM — fs\.`` is already a NodeSet containing the specified path. File system node types are a well-known fixed set, so methods are called without the _ prefix.
Several DTLs produce a DOM that supports DQL (DOM Query Language) operations: json, yaml, xml, html, golang, and xgo. Although these DOMs are not DQL NodeSet types per se, they behave as single-root NodeSets, so DQL traversal can be applied directly without any explicit conversion.
The special case is fs, whose literal result is already a NodeSet.
For full DQL documentation, see DQL - DOM Query Language.
Whether a method is called with or without the _ prefix depends on whether the DOM's node names are free-form or fixed:
Free-form node names (json, yaml, xml): use _method. Because any string can be a valid child name in these formats, a bare node.text would be interpreted as "navigate to a child named text". The _ prefix (which maps to XGo_ internally) unambiguously signals a method call.
Fixed node names (html, golang, xgo, fs): use method directly. Because the set of valid element or node names is well-known and finite, there is no ambiguity between child navigation and a method call.
| Format | Node names | Method syntax | Example |
|---|---|---|---|
| json, yaml, xml | Free-form |
node._text, node._all
|
item._text |
| html | Fixed HTML tags |
node.text, node.all
|
p.text |
| golang, xgo | Fixed AST types |
node.text, node.all
|
fn.text |
| fs | Fixed (file, dir) |
node.path, node.size
|
f.path |
doc := json`{
"animals": [
{"class": "zebra", "at": "Line 3"},
{"class": "gopher", "at": "Line 5"},
{"class": "zebra", "at": "Line 8"}
]
}`!
// Iterate all array elements
for animal in doc.animals.* {
echo animal.$class, animal.$at
}
// Filter by field value
for z in doc.animals.*@($class == "zebra") {
echo z.$at
}
// Collect field values into a list
classes := [a.$class for a in doc.animals.*]
// Materialize for multiple passes
animals := doc.animals.*._all
echo animals._countYAML works identically:
cfg := yaml`
services:
- name: auth
port: 8081
- name: api
port: 8080
`!
for svc in cfg.services.* {
echo svc.$name, svc.$port
}
api := cfg.services.*@($name == "api")._one
echo api.$portXML element names are free-form, so methods use the _ prefix. Attributes are accessed with $:
doc := xml`
Dune
Frank Herbert
A Brief History of Time
Stephen Hawking
`!
// Iterate all children
for book in doc.library.book {
echo book.$genre, book.title._text
}
// Filter by attribute
fiction := doc.library.book@($genre == "fiction")._one
echo fiction.title._text, fiction.author._text
// Deep search for all elements
for t in doc.**.title {
echo t._text
}
// Collect all authors
authors := [b.author._text for b in doc.library.book]HTML element names are fixed, so methods are called without the _ prefix:
page := html.source("https://example.com")
// Print all hyperlink URLs
for a in page.**.a {
if url := a.$href; url != "" {
echo url
}
}
// Collect text content from all paragraphs
texts := [p.text for p in page.**.p]
// Find element by attribute value
intro := page.**.*@($class == "intro").one
echo intro.text
// Count headings
headings := page.**.h1._all
echo headings.count
// Extract table cell values
for td in page.**.table.**.td {
echo td.text
}Go AST node types are fixed, so methods are called without the _ prefix:
src := golang`
package main
import (
"fmt"
"os"
)
func greet(name string) string {
return fmt.Sprintf("Hello, %s!", name)
}
func main() {
fmt.Println(greet(os.Args[1]))
}
`!
// Find all import paths
for spec in src.**.importSpec {
echo spec.path.basicLit.text
}
// Find all function declarations and their names
for fn in src.**.funcDecl {
echo fn.name.text
}
// Find all call expressions
for call in src.**.callExpr {
echo call.fun.text
}
// Find all string literals
for lit in src.**.*@(self.class == "BasicLit" && self.$kind == "STRING") {
echo lit.text
}XGo AST node types are also fixed, so methods are called without the _ prefix:
src := xgo`
x, y := "Hi", 123
echo x
print y
`!
// Find all expression statements
stmts := src.shadowEntry.body.list.*@(self.class == "ExprStmt")
// Extract function names from call expressions
for fn in stmts.x@(self.class == "CallExpr").fun@(self.class == "Ident") {
echo fn.text
}
// Deep search for all identifiers
for id in src.**.ident {
echo id.text
}
// Find all string literal values
for lit in src.**.*@(self.class == "BasicLit" && self.$kind == "STRING") {
echo lit.text
}fs produces a NodeSet directly; file/dir node types are fixed, so methods are called without the _ prefix:
// Walk current directory and print all .xgo files
for e in fs`.`.**.file.match("*.xgo") {
echo e.path
}
// Walk a specific directory
for e in fs`/path/to/project`.**.file.match("*.go") {
echo e.path
}
// Find all subdirectories
for d in fs`.`.**.dir {
echo d.path
}
// Collect all XGo source file names
names := [f.name for f in fs`.`.**.file.match("*.xgo")]
// Filter by file size
for f in fs`.`.**.file {
if f.size > 1024 {
echo f.path, "is larger than 1KB"
}
}Domain text literals compile to function calls to the corresponding package's New() function. For example:
json`{"key": "value"}`
// Compiles to:
json.New(`{"key": "value"}`)This design keeps the feature simple while allowing seamless integration with existing Go packages. The domainTag represents a package that must have a global func New(string) function with any return type.
Extend XGo with your own domain-specific languages by implementing a package with a global New(string) function:
// Package sql provides SQL query literals
package sql
type Query struct {
text string
}
func New(query string) (*Query, error) {
// Validate and parse SQL
if err := validateSQL(query); err != nil {
return nil, err
}
return &Query{text: query}, nil
}Usage:
import "myproject/sql"
query := sql`
SELECT id, name, email
FROM users
WHERE active = true
`!Domain text literals offer more than just convenient syntax. They enable XGo tooling to understand the semantics of these embedded texts rather than treating them as ordinary strings. This semantic understanding enables:
-
Code formatters like
xgo fmtto format both XGo code and supported domain texts simultaneously - IDE plugins to provide syntax highlighting and advanced features for recognized domain texts
- Static analysis tools to validate domain-specific content at build time
- Documentation generators to extract and document embedded domain content
-
Use the
!suffix for static literals that should always be valid—this catches errors early - Handle errors explicitly for dynamic content that might fail validation
- Keep literals focused on their domain—avoid mixing concerns
- Leverage syntax highlighting by configuring your editor for the embedded languages
- Document custom formats clearly to help other developers understand their usage
-
Use
_all/allfor repeated DQL queries over the same NodeSet to avoid re-execution
Without the ! suffix, domain literals return an error that you can handle:
query, err := sql`SELECT * FROM ${table}`
if err != nil {
return fmt.Errorf("invalid query: %w", err)
}With the ! suffix, invalid literals cause a panic:
// This panics if the JSON is malformed
data := json`{"invalid": }`!The journey of domain text literals in XGo began with a community proposal in early 2024 suggesting adding JSX syntax support to XGo. While JSX has gained widespread adoption in frontend development, particularly in React-based applications, the immediate benefits of building JSX syntax directly into XGo weren't immediately clear, causing the proposal to be temporarily shelved.
The turning point came when XGo needed to support TPL (Text Processing Language) syntax for the XGo Mini Spec project. This necessity prompted a reconsideration of how XGo should handle domain-specific notations more broadly.
A common understanding in programming language design suggests that Domain-Specific Languages (DSLs) often struggle to compete with general-purpose languages. However, this perspective overlooks the fact that numerous domain languages exist and thrive in specialized contexts:
- Interface description: HTML, JSX
- Configuration and data representation: JSON, YAML, CSV
- Text syntax representation: EBNF-like grammar (including TPL syntax), regular expressions
- Document formats: Markdown, DOCX, HTML
What distinguishes these domain languages is that they aren't Turing-complete. They lack the full capabilities of general-purpose languages, such as I/O operations, function definitions, and comprehensive flow control structures.
Rather than competing with general-purpose languages, these domain languages typically complement them. Most mainstream programming languages either officially support or have community-built libraries to interact with these domain languages.
This complementary relationship led to the term "Domain Text Literals" rather than "Domain-Specific Languages", emphasizing their role as specialized text formats that can be embedded within general-purpose code.
After considerable deliberation on how XGo should support domain text literals, inspiration came from Markdown's code block syntax. Initially, there was consideration to make XGo's domain text syntax identical to Markdown's. However, this would have prevented XGo code from being embedded as a domain text within Markdown documents, potentially reducing interoperability between XGo and Markdown. After careful consideration, the current syntax was chosen to ensure optimal compatibility while maintaining the familiar, intuitive pattern that developers already know from Markdown.
Domain-specific text literals make XGo uniquely suited for projects that need to work with multiple specialized formats. By treating domain-specific languages as first-class citizens, XGo helps you write cleaner, safer, and more maintainable code.