incognigo

Incognigo

Incognigo is a dependency-free Go library for detecting and transforming personally identifiable information (PII) before it reaches logs, external services, or other data destinations.

Version v0.1.0 is the first production-candidate release. Its implementation is designed to fail closed, bound input-driven work, avoid source data in library diagnostics, and support concurrent reuse. The public API remains a v0 API: compatibility-breaking improvements are still possible before v1.

Incognigo provides:

• configurable policy and priority per PII type;
• byte, string, and bounded-retention stream processing;
• a protecting log/slog handler;
• built-in ASCII email, IPv4/IPv6, payment-card, and NANP phone detectors;
• removal, fixed replacement, byte masking, HMAC tokenization, and reversible AES-256-GCM tokenization; and
• extension interfaces for application-owned detectors and transformers.

The built-in detectors are deliberately syntactic and limited. Incognigo does not establish that a match belongs to a person, and it does not include names, postal addresses, international phone formats outside NANP, government identifiers, or contextual free-text entity recognition.

Requirements and installation

Incognigo supports Go 1.25 and Go 1.26. Use the latest patch release available for either line.

go get github.com/Chadi00/incognigo@v0.1.0

The module has no third-party runtime or test dependencies.

Quick start

This complete example protects every built-in PII type with a fixed marker. Always treat construction or processing errors as protection failures; never fall back to the original input.

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/Chadi00/incognigo"
	"github.com/Chadi00/incognigo/detector"
	"github.com/Chadi00/incognigo/transform"
)

func main() {
	if err := run(); err != nil {
		log.Fatal(err)
	}
}

func run() error {
	engine, err := incognigo.NewEngine(incognigo.Config{
		Detectors: []incognigo.DetectorRegistration{
			detector.Email(),
			detector.IPAddress(),
			detector.PaymentCard(),
			detector.NANPPhone(),
		},
		Policies: []incognigo.Policy{
			{
				Type:        incognigo.PIITypeEmailAddress,
				Priority:    100,
				Transformer: transform.Replace([]byte("[email]")),
				Determinism: incognigo.Deterministic,
			},
			{
				Type:        incognigo.PIITypeIPAddress,
				Priority:    90,
				Transformer: transform.Replace([]byte("[ip]")),
				Determinism: incognigo.Deterministic,
			},
			{
				Type:        incognigo.PIITypePaymentCardNumber,
				Priority:    80,
				Transformer: transform.Replace([]byte("[payment-card]")),
				Determinism: incognigo.Deterministic,
			},
			{
				Type:        incognigo.PIITypePhoneNumber,
				Priority:    70,
				Transformer: transform.Replace([]byte("[nanp-phone]")),
				Determinism: incognigo.Deterministic,
			},
		},
	})
	if err != nil {
		return fmt.Errorf("configure PII protection: %w", err)
	}

	protected, err := engine.ProtectString(
		context.Background(),
		"email owner@example.test from 192.0.2.1 "+
			"with card 4111 1111 1111 1111 or call 212-555-0100",
	)
	if err != nil {
		return fmt.Errorf("protect data: %w", err)
	}
	fmt.Println(protected)
	return nil
}

Output:

email [email] from [ip] with card [payment-card] or call [nanp-phone]

The repository's Example... functions compile and execute under go test. They cover every public workflow, including custom extensions, all built-in detectors and transformations, streaming, structured logging, detokenization, and inspectable failures.

Processing APIs

Engine.ProtectBytes is the canonical batch API. It never mutates or retains its input, and every successful result is newly owned by the caller. Engine.ProtectString is a byte-preserving adapter over the same path. On failure they return no partial or unprotected output.

protectedBytes, err := engine.ProtectBytes(ctx, inputBytes)
protectedString, err := engine.ProtectString(ctx, inputString)

Streaming

Engine.ProtectStream processes an arbitrary-length io.Reader while bounding undecided source retention:

var destination bytes.Buffer
written, err := engine.ProtectStream(
	ctx,
	&destination,
	strings.NewReader("contact owner@example.test from 192.0.2.1"),
)

written counts only protected bytes accepted by the destination. A reader, detector, transformer, or writer failure after an earlier write can leave a protected prefix. Incognigo never appends undecided or unprotected source bytes after that prefix. Use a transactional destination or buffer before commit when delivery must be all-or-nothing.

Structured logging

Wrap a downstream slog.Handler to protect record messages and attribute values before formatting or transport:

downstream := slog.NewJSONHandler(os.Stdout, nil)
handler, err := sloghandler.New(engine, downstream)
if err != nil {
	return err
}

logger := slog.New(handler)
logger.Info(
	"contact owner@example.test",
	"peer", "192.0.2.1",
	"backup", []byte("backup@example.test"),
)

String and exact []byte values keep their logical kinds. slog.LogValuer values are resolved, groups are traversed without recursive stack growth, and other non-nil slog.Any values are eagerly converted with fmt.Sprint, protected, and passed downstream as strings. This conversion prevents a downstream-only serializer from bypassing protection, but intentionally loses the original dynamic type.

Attribute and group keys, nil Any values, native non-textual values, and record metadata are preserved. Downstream handler errors are returned unchanged. A canceled record context does not cancel protection, as required by the slog.Handler contract.

Transformations

Detection and transformation are independent. A policy can choose any transformer for its PII type.

Masking counts bytes, not Unicode code points. Callers choose the retained prefix and suffix, one replacement byte, and whether unexpectedly short values are fully masked or rejected without output.

masker, err := transform.NewMasker(transform.MaskOptions{
	PrefixBytes: 2,
	SuffixBytes: 4,
	MaskByte:    '*',
	ShortInput:  transform.ShortInputMaskAll,
})

All built-in transformations are immutable and concurrency-safe after construction. Redaction and masking are directly idempotent; end-to-end engine idempotence additionally depends on whether detectors recognize transformed output. Tokenization is not idempotent.

Custom extensions

Applications can implement the single-method incognigo.Detector and incognigo.Transformer interfaces. Registrations declare stable detector IDs, authorized PII types, and finite match/lookaround bounds. Custom extensions are trusted code: they must treat borrowed values as read-only, must not retain or disclose them, must honor cancellation, and must be safe for concurrent calls.

NewEngine copies configuration slices and metadata, but retains detector and transformer implementations as executable interfaces.

Built-in detector formats

Built-ins are deterministic, byte-oriented syntactic detectors. Non-ASCII and malformed UTF-8 bytes are barriers, original byte offsets are preserved, and input is never normalized. Detection resumes after a barrier, so a valid ASCII candidate that follows a non-ASCII byte can still match independently.

Additional format details:

• Email local parts contain 1–64 bytes made from ASCII letters, digits, dots, and !#$%&'*+-/=?^_`{|}~. Dots separate non-empty atoms. Domain labels contain 1–63 letters, digits, or interior hyphens; the domain is at most 253 bytes and the complete address at most 254 bytes. A terminal period is surrounding punctuation.
• IPv4 spans the full 0.0.0.0 through 255.255.255.255 range. IP syntax is structurally validated with net/netip.
• Payment-card rendered length is at most 37 bytes. Passing Luhn does not prove that an account was issued or is active.
• NANP examples include 2125550100, 212-555-0100, (212) 555-0100, 1-212-555-0100, +12125550100, and +1 (212) 555-0100.

Accuracy evidence

The committed JSON corpora label exact expected byte spans using entirely synthetic or officially reserved values. TestAccuracyCorpora rejects any span drift, false positive, or false negative.

These figures are regression evidence only. They are not an estimate of recall or precision on arbitrary production traffic. Applications must validate detector scope and false-negative risk against representative, privacy-safe data before relying on Incognigo at a production trust boundary.

Behavioral contracts

Configuration and overlap resolution

Engines must be created with incognigo.NewEngine. Detector IDs and policies are unique. Every type declared by a detector has exactly one policy, and every policy corresponds to a declared type. Invalid or typed-nil implementations are rejected. A nil or zero-valued engine fails closed.

Overlapping candidates are preferred deterministically by:

1. policy priority, descending;
2. span length, descending;
3. start offset, ascending;
4. end offset, ascending;
5. PII type, lexicographically ascending; and
6. detector ID, lexicographically ascending.

Candidates are greedily accepted only when they do not overlap an accepted span. Adjacent spans do not overlap. Identical accepted matches collapse.

Byte and ownership semantics

All offsets and limits count bytes. Incognigo performs no Unicode normalization. Malformed UTF-8 outside a match is preserved exactly.

Detector input and transformer values are borrowed, read-only views valid only for the call. Extensions transfer ownership of returned slices. Batch processing never mutates or retains caller input and always returns newly owned output, including empty and no-match results.

One engine may be reused concurrently when its custom extensions are also concurrency-safe. Built-in detectors and transformations are immutable and safe for concurrent use.

Limits

Zero-valued fields select these defaults:

Maximum values are inclusive. Negative limits are invalid. Total stream input and output are not bounded by the batch byte limits.

Failures and diagnostics

Incognigo-generated runtime failures are *incognigo.Error values. Use errors.Is with the fixed sentinels and errors.As for allowlisted metadata:

var protectionErr *incognigo.Error
if errors.As(err, &protectionErr) {
	actual, maximum, hasCounts := protectionErr.Counts()
	_ = actual
	_ = maximum
	_ = hasCounts
}

Library errors may contain only fixed operation/field/limit names, validated detector IDs and PII types, and numeric counts. Arbitrary detector, transformer, reader, and writer errors are discarded rather than wrapped because their messages may contain source data. Batch failures return no partial output. Transformer output returned with an error is discarded.

Tokenization design

Both tokenizers require a caller-supplied, unpredictable 32-byte key and a public key ID. They accept arbitrary bytes, bind the PII type, use unpadded base64url output, default to an 8 MiB direct-call limit, and perform no network, file, secret-manager, database, or token-vault access.

Deterministic irreversible tokens use:

icg.v1.hmac-sha256.<key-id>.<32-byte-hmac>

The HMAC input is:

"incognigo\0token\0v1\0hmac-sha256\0"
|| complete-token-header
|| uint64be(len(pii-type)) || pii-type
|| uint64be(len(value))    || value

Randomized reversible tokens use:

icg.v1.aes-256-gcm.<key-id>.<12-byte-nonce || ciphertext || 16-byte-tag>

AES-GCM authenticates:

"incognigo\0token\0v1\0aes-256-gcm\0"
|| complete-token-header
|| uint64be(len(pii-type)) || pii-type

The complete header ends after the key-ID separator. Parsers reject unknown versions, algorithms, malformed encodings, wrong PII types, tampering, and non-canonical payloads.

Threat model and key rotation

HMAC tokens expose equality and frequency. Possession of their key permits offline guessing, especially for low-entropy values. Reversible tokens expose encoded plaintext length and become readable and forgeable if their key is compromised. Neither mode prevents disclosure from other plaintext copies, process memory, crash dumps, malicious extensions, or application code authorized to detokenize.

Go does not guarantee zeroization of compiler-managed key or plaintext memory. Callers own secret storage, process controls, aggregate usage accounting, and rotation. One reversible key must not encrypt more than 2^32 values across all processes and tokenizer instances.

To rotate a reversible key:

1. create a tokenizer with the new key as ActiveKey;
2. retain required prior keys in DecryptionKeys;
3. route new transformations through the new instance;
4. optionally detokenize and re-tokenize persisted old values; and
5. remove an old key only when recovery and retention policies permit.

Missing keys are never fetched automatically. A recognized reversible token with an unavailable key returns ErrTokenKeyUnavailable; malformed or unauthenticated tokens return ErrInvalidToken; oversized values return ErrTokenValueTooLarge. Every failure returns nil plaintext.

Test vectors

The following vectors use the synthetic value person@example.test, PII type email_address, key ID 2026-01, and a test-only key containing bytes 00 01 ... 1f.

icg.v1.hmac-sha256.2026-01.3rDSpGYbv2Mt9LkfD0SiBefyvnDQN47gLdInd8STLyk

The reversible decryption vector uses the fixed nonce 00 01 02 03 04 05 06 07 08 09 0a 0b only for verification:

icg.v1.aes-256-gcm.2026-01.AAECAwQFBgcICQoLN2ekaKqLgn71IPr73YxWGeal84tlruM4lmxUjjKxV2SbAHM

Production tokenization always obtains fresh randomness from Go's cryptographic implementation.

Security and limitations

Incognigo processes sensitive data at application trust boundaries. Report a suspected privacy bypass, source-data disclosure, unbounded resource path, concurrency leak, or cryptographic defect through GitHub's private security advisory flow:

https://github.com/Chadi00/incognigo/security/advisories/new

Do not include customer data, production logs, credentials, real PII, or public breach data. Use a minimal synthetic reproduction. If private advisories are unavailable, contact the repository owner privately without sending exploit details in the initial message.

Operational limitations:

• syntactic detection cannot guarantee that every production PII value is found;
• unsupported formats pass through unchanged unless a custom detector handles them;
• attribute keys and native non-textual slog.Value kinds are not transformed;
• custom extensions are trusted code;
• streaming can leave a protected prefix after failure;
• deterministic tokens disclose equality and reversible tokens depend on caller-owned key protection; and
• during v0, exported APIs may change to address material safety, correctness, or usability issues.

Never treat protection failure as permission to send the original value.

Development and release checks

Test fixtures, fuzz seeds, examples, benchmarks, issues, and pull requests must use synthetic data. Never commit customer samples, real-person data, public breach data, raw vulnerability-report values, or secrets. The canonical detector corpora live under detector/testdata.

Common checks:

go vet ./...
go test ./...
go test -race ./...
go test -run '^$' -bench . -benchmem -benchtime=1x ./...
go test -v ./detector -run '^TestAccuracyCorpora$'
./scripts/fuzz.sh 30s

The complete release gate checks formatting, module tidiness, the zero-dependency policy, vetting, ordinary and race tests, benchmark compilation, accuracy corpora, every fuzz target, and pinned vulnerability analysis:

./scripts/release-check.sh 30s

Pull requests affecting detection, transformation, streaming, logging, public or persisted behavior must explain failure behavior, resource bounds, test and fuzz coverage, and compatibility impact.

License

Incognigo is licensed under the Apache License 2.0.