runesafe

func CapBytes ¶

func CapBytes(s string, n int) string

CapBytes truncates s to at most n bytes without splitting a multi-byte rune at the cut: the cut backs off to the nearest rune start, so the result never ends in a partial rune. It exists because sanitizing can grow a string (each invalid UTF-8 byte becomes the three-byte U+FFFD), so a pre-sanitize byte cap does not survive sanitization, and a naive re-cap can split a rune — leaving a partial-rune tail whose raw 0x80-0x9F bytes a non-UTF-8 terminal reads as C1 escape introducers, re-minting the very class of byte the sanitizer removed. For valid UTF-8 input (any Sanitize or SanitizeSingleLine output) the backoff discards at most three bytes below n and the result is a valid-UTF-8 prefix of s. A non-positive n returns the empty string.

package main

import (
	"fmt"

	"github.com/cplieger/runesafe"
)

func main() {
	s := "葬送のフリーレン" // three bytes per rune
	fmt.Printf("%q\n", runesafe.CapBytes(s, 7))
}

Output:
"葬送"

func IsBidiControl ¶

func IsBidiControl(r rune) bool

IsBidiControl reports whether r is one of Unicode's Bidi_Control format characters: the singleton marks U+061C (ALM) and U+200E/U+200F (LRM/RLM), the override/embedding range U+202A-U+202E (LRE/RLE/PDF/LRO/RLO), and the isolate range U+2066-U+2069 (LRI/RLI/FSI/PDI). Any of them in untrusted text can visually reorder rendered output (Trojan-Source-style report and link spoofing), so every output sanitizer treats the full set as unsafe. The set matches unicode.Bidi_Control exactly, without the table lookup.

package main

import (
	"fmt"

	"github.com/cplieger/runesafe"
)

func main() {
	fmt.Println(runesafe.IsBidiControl('\u202e'), runesafe.IsBidiControl('a'))
}

Output:
true false

func IsUnsafe ¶

func IsUnsafe(r rune, keepCRLF bool) bool

IsUnsafe reports whether r is unsafe in untrusted text bound for a log, JSON, or rendered-output sink: a C0 control, DEL, a C1 control (U+0080-U+009F, single-rune terminal-escape introducers), a Unicode bidi control (IsBidiControl), or the U+2028/U+2029 line separators. keepCRLF selects the CR/LF policy: true treats CR and LF as safe, for sinks whose encoder escapes them (JSON); false treats them as unsafe like every other C0 control, for single-line sinks where a raw newline forges a new record. Sanitize and SanitizeSingleLine apply the two policies to whole strings.

package main

import (
	"fmt"

	"github.com/cplieger/runesafe"
)

func main() {
	fmt.Println(runesafe.IsUnsafe('\n', true), runesafe.IsUnsafe('\n', false))
	fmt.Println(runesafe.IsUnsafe('\u009b', true), runesafe.IsUnsafe('\u009b', false))
}

Output:
false true
true true

func IsUnsafeNonASCII ¶ added in v1.1.0

func IsUnsafeNonASCII(r rune) bool

IsUnsafeNonASCII reports whether r is an unsafe rune above the ASCII range: a C1 control (U+0080-U+009F), a Unicode bidi control (IsBidiControl), or the U+2028/U+2029 line separators. It is the IsUnsafe policy minus C0, DEL, and the CR/LF axis (all ASCII), for composed escapers whose sink already covers ASCII: a URL percent-encoder escapes C0, DEL, and whitespace itself, but url.Parse accepts these non-ASCII runes raw, and a terminal or Markdown viewer must never receive them. The CR/LF policy switch is moot above ASCII, so there is no keepCRLF parameter: IsUnsafeNonASCII(r) equals IsUnsafe(r, keepCRLF) && r > unicode.MaxASCII under either policy.

package main

import (
	"fmt"

	"github.com/cplieger/runesafe"
)

func main() {
	fmt.Println(runesafe.IsUnsafeNonASCII('\u009b'), runesafe.IsUnsafeNonASCII('\u202e'))
	fmt.Println(runesafe.IsUnsafeNonASCII('\x1b'), runesafe.IsUnsafeNonASCII('é'))
}

Output:
true true
false false

func Sanitize ¶

func Sanitize(s string) string

Sanitize makes an untrusted string safe for slog/JSON sinks by replacing each unsafe rune with a space: C0 controls (except CR/LF, which JSON encoders escape), DEL, C1 controls (U+0080-U+009F, single-rune terminal-escape introducers emitted raw by encoding/json and slog's JSONHandler), Unicode bidi controls, and the U+2028/U+2029 line separators. Invalid UTF-8 bytes become U+FFFD (a strings.Map property), so the result is always valid UTF-8. Apply it to every untrusted attribute at the emit boundary — one policy shared by all of an app's sinks, so they cannot drift. For a single-line sink where CR/LF must also go, use SanitizeSingleLine.

package main

import (
	"fmt"

	"github.com/cplieger/runesafe"
)

func main() {
	title := "Frieren\x1b[2J \u202egpj.exe"
	fmt.Printf("%q\n", runesafe.Sanitize(title))
}

Output:
"Frieren [2J  gpj.exe"

func SanitizeBudgeted ¶ added in v1.4.0

func SanitizeBudgeted(s string, n int, marker string) (text string, cut bool)

SanitizeBudgeted is the one-value form of NewBudget: a Budget of n bytes, one Write of s, and its Result. It is SanitizeCapped with the cap moved AHEAD of the sanitizer — s is capped on a rune boundary at n bytes first, the chunk is sanitized under the keepCRLF=true policy, sanitization growth is re-capped, and the caller's marker is charged inside n — so the budget bounds the WORK done on s rather than only the size of what comes back. Reach for it when s comes from an upstream whose size the caller does not control (a response field, an error message interpolating one, a file name), where walking all of it is itself the exposure; reach for SanitizeCapped when s is already known to be small and only its output needs bounding.

The two orders agree on every valid-UTF-8 value that carries no unsafe rune, byte for byte, cut for cut, so a call site can move to this one without changing what an honest value emits. They diverge for exactly one input class: a value whose RAW form exceeds n but whose SANITIZED form would have fitted, which needs enough multi-byte unsafe runes to collapse it under the cap (each becomes a single-byte space) — the hostile shape the work bound exists for. This function cuts and marks such a value where SanitizeCapped returns it whole and unmarked, and the mark is honest either way: bytes really were dropped, before sanitizing.

SanitizeSingleLineBudgeted is the strict twin. Neither serves a caller aggregating SEVERAL values under one shared budget — that is Budget itself, which is what keeps ONE truncation fact across the writes instead of one per value.

package main

import (
	"fmt"
	"strings"

	"github.com/cplieger/runesafe"
)

func main() {
	huge := "A" + strings.Repeat("\u202e", 1<<20)

	text, cut := runesafe.SanitizeBudgeted(huge, 8, "...")
	fmt.Printf("%q %v\n", text, cut)

	text, cut = runesafe.SanitizeCapped(huge, 8, "...")
	fmt.Printf("%q %v\n", text, cut)
}

Output:
"A  ..." true
"A    ..." true

func SanitizeCapped ¶ added in v1.3.0

func SanitizeCapped(s string, n int, marker string) (text string, cut bool)

SanitizeCapped is Sanitize followed by a byte cap that INCLUDES the marker. It sanitizes s under the keepCRLF=true policy (CR and LF survive, for a sink whose encoder escapes them — JSON, slog's handlers), and if the sanitized form exceeds n bytes it cuts that form on a rune boundary at n-len(marker) bytes and appends marker, so the returned text is at most max(n, 0) bytes whatever marker is. A within-cap value comes back untouched — byte-identical to Sanitize, no marker.

cut reports whether the sanitized form was SHORTENED, not whether sanitizing rewrote anything: it is true exactly when that form did not fit in n bytes, and false whenever the returned text is the whole sanitized form. It exists because a marker cannot prove a cut — a value may end in the marker on its own — so a caller that must report truncation as a fact (a log attribute beside the value, a decision to keep a fuller copy elsewhere) would otherwise re-implement this composition just to set its own flag.

n bounds the TOTAL, which is what a caller with a real budget needs: a record persisted under a write limit, a payload assembled under a vendor byte cap, a fixed-width column. SanitizeSingleLineBounded puts its marker OUTSIDE the cap (a truncated result runs to n+3 bytes), which leaves such a caller subtracting the marker's width by hand at every call site.

A marker longer than n cannot be shown intact, and a partial marker is indistinguishable from content, so for n < len(marker) the result is the rune-boundary prefix of the sanitized form alone — the empty string for a non-positive n — and the elision is reported through cut only. An empty marker is legal and yields a silent cap (CapBytes over the sanitized form) with the fact still returned. An empty s returns "" and false under any n, negative included. marker is emitted verbatim, never sanitized: it is the caller's own program text, and rewriting it would silently alter a chosen mark — a marker assembled from untrusted input must be sanitized by the caller first.

The CR/LF policy is a second function rather than a parameter on this one, deliberately: the package already expresses that choice as the named Sanitize / SanitizeSingleLine pair, so a call site names its sink instead of passing an opaque boolean next to two other tuning arguments. Only IsUnsafe takes the flag, because a composed policy needs it as data. SanitizeSingleLineCapped is the strict twin.

Two consumer shapes this pair deliberately does NOT serve, and must not be widened to serve:

• Capping BEFORE sanitizing, to bound the sanitizer's WORK rather than its output. A caller that must never walk a multi-megabyte upstream value in a memory-limited process caps the raw bytes first (CapBytes, then Sanitize on the chunk); this function sanitizes all of s by construction, and no marker placement changes that. Such a caller also tends to aggregate one truncation fact across several appends, which a single-call primitive cannot express. Both shapes belong to SanitizeBudgeted and Budget — a separate primitive carrying the cap ahead of the sanitizer and a running remainder, sharing this pair's rune-boundary cut, marker-inside-the-cap and cut-as-a-fact rules while leaving the contract above exactly as it is.
• Keeping the TAIL behind a PREFIXED marker. When the identifying part of a value sits at its end (a path's file name), the caller wants marker + suffix; this function keeps the head. Compose the rune-boundary walk locally for that.

package main

import (
	"fmt"

	"github.com/cplieger/runesafe"
)

func main() {
	body, cut := runesafe.SanitizeCapped("line one\nline two\x1b[2J", 16, "...(truncated)")
	fmt.Printf("%q %v %d\n", body, cut, len(body))

	body, cut = runesafe.SanitizeCapped("short\nbody", 16, "...(truncated)")
	fmt.Printf("%q %v\n", body, cut)
}

Output:
"li...(truncated)" true 16
"short\nbody" false

func SanitizeSingleLine ¶

func SanitizeSingleLine(s string) string

SanitizeSingleLine makes an untrusted string safe for a single-line sink — a plain-text log line, a one-line error message, a rendered table cell — by replacing each unsafe rune with a space under the strict keepCRLF=false policy: everything Sanitize replaces, plus CR and LF, whose raw appearance in a single-line sink forges a record boundary. Invalid UTF-8 bytes become U+FFFD, so the result is always valid UTF-8 and carries no line break.

package main

import (
	"fmt"

	"github.com/cplieger/runesafe"
)

func main() {
	msg := "bad request\nlevel=ERROR forged\x1b[2J"
	fmt.Printf("%q\n", runesafe.SanitizeSingleLine(msg))
}

Output:
"bad request level=ERROR forged [2J"

func SanitizeSingleLineBounded ¶ added in v1.2.0

func SanitizeSingleLineBounded(s string, n int) string

SanitizeSingleLineBounded is SanitizeSingleLine followed by a byte cap: an over-cap result is truncated to at most n bytes on a rune boundary (CapBytes) with "..." appended to mark the cut, while a result within the cap is returned untouched — no marker, byte-identical to the unbounded form. It is the log-bound preset for VOLUME as well as rune safety: an upstream-controlled value headed for a log attribute (a query parameter, a JSON key name, an upstream error body) must not balloon a record past downstream log-pipeline limits, and every consumer hand-rolled exactly this sanitize-cap-mark composition. The cap applies to the SANITIZED form (sanitizing can grow invalid bytes into the three-byte U+FFFD, so a pre-sanitize cap does not survive), and the marker sits outside the cap: a truncated result is at most n+3 bytes and always ends in the marker. The converse does not hold — a within-cap input may itself end in "..." — so the marker marks the cut without proving one; a caller that must know whether truncation occurred, supply its own marker, or hold the total (marker included) under a hard budget uses SanitizeSingleLineCapped, which returns the fact and counts the marker inside the cap. A non-positive n yields "..." alone for a non-empty input ("" stays "", whatever n is). The marker's placement outside the cap is this preset's settled contract, not an accident of its implementation: it stays as it is so every existing call site keeps its byte-for-byte output.

package main

import (
	"fmt"

	"github.com/cplieger/runesafe"
)

func main() {
	fmt.Printf("%q\n", runesafe.SanitizeSingleLineBounded("bad\nrequest", 20))
	fmt.Printf("%q\n", runesafe.SanitizeSingleLineBounded("葬送のフリーレン", 7))
}

Output:
"bad request"
"葬送..."

func SanitizeSingleLineBudgeted ¶ added in v1.4.0

func SanitizeSingleLineBudgeted(s string, n int, marker string) (text string, cut bool)

SanitizeSingleLineBudgeted is SanitizeBudgeted under the strict keepCRLF=false policy: CR and LF become spaces along with every other unsafe rune, for a single-line sink where a raw newline forges a record boundary. The pre-cap, re-cap, marker and cut contract are identical; see SanitizeBudgeted for the full contract and for the one input class on which the pre-cap order diverges from SanitizeSingleLineCapped's.

func SanitizeSingleLineCapped ¶ added in v1.3.0

func SanitizeSingleLineCapped(s string, n int, marker string) (text string, cut bool)

SanitizeSingleLineCapped is SanitizeCapped under the strict keepCRLF=false policy: CR and LF become spaces along with every other unsafe rune, for a single-line sink where a raw newline forges a record boundary — a plain-text log line, a one-line error message, a rendered table cell. The cap, marker and cut contract are identical, marker counted INSIDE n and the n < len(marker) degenerate case included; see SanitizeCapped for the full contract, for why the CR/LF policy is a separate function, and for the two consumer shapes the pair does not serve.

package main

import (
	"fmt"

	"github.com/cplieger/runesafe"
)

func main() {
	reason, cut := runesafe.SanitizeSingleLineCapped("bad request\nlevel=ERROR forged", 14, "...")
	fmt.Printf("%q %v %d\n", reason, cut, len(reason))
}

Output:
"bad request..." true 14