phonmatchnet

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 13, 2026 License: Apache-2.0 Imports: 21 Imported by: 0

README

go-phonmatchnet

github.com/androiddrew/go-phonmatchnet (package phonmatchnet) detects text-defined wake words with ONNX Runtime. Type a keyword, give it a short clip of 16 kHz mono audio, and it returns the probability that the keyword was spoken. New wake words need no recordings or training.

It runs naptrap's independently trained English PhonMatchNet (Lee & Cho, Interspeech 2023) and matches naptrap's PyTorch scores to about 1e-5. All models are embedded (7.8 MB), so the only external requirement is an installed ONNX Runtime. Audio decoding, resampling, microphone windowing and any CLI or serving layer belong to the application.

import (
    "context"
    "errors"
    phonmatchnet "github.com/androiddrew/go-phonmatchnet"
)

func detect(ctx context.Context, samples []float32) (result phonmatchnet.Result, err error) {
    engine, err := phonmatchnet.New(phonmatchnet.Config{
        ORTLibrary: "libonnxruntime.so", // OS loader name or explicit installed-library path
    })
    if err != nil { return result, err }
    defer func() { err = errors.Join(err, engine.Close()) }()

    keyword, err := engine.Keyword(ctx, "hey computer") // reuse across clips
    if err != nil { return result, err }
    return engine.Detect(ctx, phonmatchnet.Request{
        Samples: samples, SampleRate: phonmatchnet.SampleRate, Keyword: keyword,
    })
}

result.Score is the keyword-match probability (0–1) and result.Accepted reports Score >= Threshold. Reuse one engine across requests.

API and guarantees

  • Keyword normalizes the text exactly as naptrap does (accents and punctuation removed, lowercase, apostrophes inside words kept) and computes its g2p decoder features. Numbers must be spelled out ("route sixty six"). NormalizeKeyword exposes the normalization alone. A keyword may produce at most MaxKeywordPositions (128) feature rows.
  • Detect scores one clip of 0.5–2 seconds (MinSamples–MaxSamples) of finite samples in [-1, 1] at SampleRate; otherwise ErrInvalidAudio. Samples are not rescaled, clipped, trimmed, mixed or resampled, and must not be mutated during the call.
  • Result also carries the raw Logit, the operating point used and stage Timings. ModelInfo returns the validated tensor metadata of the four graphs; Initialization reports loading times and the native runtime version.
  • Errors: ErrInvalidAudio, ErrInvalidKeyword, ErrUnknownFAR, ErrClosed, ErrNotInitialized, plus context cancellation.

Thresholds

Request.FAR selects a calibrated operating point by its target per-clip false-accept rate; Engine.Thresholds lists them:

FAR Score threshold
0.01 (default) 0.7149
0.1 0.0026

These were fitted on held-out LibriSpeech development speakers. The targets do not hold on other audio: on phonetically confusable phrases the 1% threshold accepted 31% of negatives. In the test fixtures, Kokoro-synthesized "hey commuter" scores up to 0.99 for the keyword "hey computer". For a real product, record positive and negative clips in your setting and choose your own threshold on Result.Score. See the model card for the evaluation.

Listening to a microphone

The model scores single clips; it is not a streaming listener. To watch live audio, window it yourself, e.g. 1.5 s windows every 0.25 s, compute the keyword once, and debounce repeated accepts. Detection of a 1.5 s window takes about 5.5 ms and a two-word keyword about 0.7 ms on one thread of a Ryzen 5 5600X (go test -bench .).

ONNX Runtime ownership

ONNX Runtime has one environment per process, shared by every model in it.

  • Set ORTLibrary (simplest): the engine initializes the environment through github.com/androiddrew/ortenv on first use. The environment stays loaded until process exit, so closing the last engine never unloads the runtime. Every component using ortenv in the process must use the same library selector, and nothing may destroy the environment ortenv owns.
  • Leave ORTLibrary empty when your application already manages ONNX Runtime itself (directly through onnxruntime_go or through ortenv). Initialize the environment before New and destroy it only after Close; otherwise New returns ErrNotInitialized.

Config.Threads sets intra-op threads per session (default 1). With many models in one process, keep it small to avoid oversubscribing the CPU.

Runtime and behavior

Close every engine. Each engine serializes its requests and Close. Native Run cannot be interrupted mid-call; cancellation is checked between graph runs. Inference runs on CPU; the models are small enough that GPU execution is not offered. Requires Go 1.25 or newer with CGO. Tested: Linux/amd64, Go 1.25.0 and 1.26.4, ORT 1.22.0 and 1.22.2. This is a known tested configuration, not an exact native-version restriction.

Limits inherited from the model: English only, trained on read audiobook speech (expect lower accuracy with noise, far-field microphones, accents or children); no speaker verification, so anyone saying the keyword triggers it; similar-sounding phrases are often confused; not for security or safety-critical use.

Development and tools

go test ./...
go vet ./...

For native tests, set PHONMATCHNET_TEST_ORT to an ONNX Runtime library path or loader name, then run go test -race ./.... The tests compare keyword features, speech embeddings and 80 clip/keyword scores with naptrap's PyTorch outputs in testdata/reference.json. Native tests skip when the variable is unset. Tests that need ONNX Runtime uninitialized run in a fresh subprocess, because ortenv keeps the environment loaded until process exit.

tools/export regenerates the embedded models from naptrap's pinned PyTorch releases, verifies them against naptrap, and assembles the Hugging Face repositories. tools/export/fixtures.py rebuilds the test fixtures (Python, not needed to use the library).

See ASSETS.md, THIRD_PARTY.md, LICENSE and NOTICE.

Citation

If the PhonMatchNet model is useful in your research, please cite:

@inproceedings{lee23phonmatchnet,
  title     = {PhonMatchNet: Phoneme-Guided Zero-Shot Keyword Spotting for User-Defined Keywords},
  author    = {Lee, Yong-Hyeok and Cho, Namhyun},
  booktitle = {Proc. INTERSPEECH 2023},
  pages     = {3964--3968},
  year      = {2023}
}

go-phonmatchnet is not affiliated with or endorsed by NCSOFT, the PhonMatchNet authors, Google or the g2p_en authors.

Documentation

Overview

Package phonmatchnet detects text-defined wake words in 16 kHz mono PCM with naptrap's PhonMatchNet model and ONNX Runtime.

Example

Detect a keyword in one clip. The models are embedded, so no paths are needed. 

package main

import (
	"context"
	"fmt"
	"log"

	phonmatchnet "github.com/androiddrew/go-phonmatchnet"
)

// loadClip stands in for the application's audio decoding. The engine requires
// one 0.5-2 second clip of 16 kHz mono float32 PCM in [-1, 1].
func loadClip() []float32 { return make([]float32, phonmatchnet.SampleRate) }

func main() {
	engine, err := phonmatchnet.New(phonmatchnet.Config{
		ORTLibrary: "libonnxruntime.so", // OS loader name or a path to the library
	})
	if err != nil {
		log.Fatal(err)
	}
	defer engine.Close()

	ctx := context.Background()
	keyword, err := engine.Keyword(ctx, "hey computer") // reuse across clips
	if err != nil {
		log.Fatal(err)
	}
	result, err := engine.Detect(ctx, phonmatchnet.Request{
		Samples: loadClip(), SampleRate: phonmatchnet.SampleRate, Keyword: keyword,
	})
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("score %.3f accepted %v\n", result.Score, result.Accepted)
}
Example (CallerOwnedEnvironment)

An application that already initializes ONNX Runtime itself leaves ORTLibrary empty. The engine then uses that environment and never destroys it. 

package main

import (
	"fmt"
	"log"

	phonmatchnet "github.com/androiddrew/go-phonmatchnet"
	ort "github.com/yalue/onnxruntime_go"
)

func main() {
	ort.SetSharedLibraryPath("libonnxruntime.so")
	if err := ort.InitializeEnvironment(); err != nil {
		log.Fatal(err)
	}
	defer ort.DestroyEnvironment() // runs after engine.Close

	engine, err := phonmatchnet.New(phonmatchnet.Config{})
	if err != nil {
		log.Fatal(err)
	}
	defer engine.Close()
	fmt.Println(engine.Initialization().RuntimeVersion)
}

Index

Examples

Constants

View Source 
const (
	MinSamples = SampleRate / 2
	MaxSamples = 2 * SampleRate
)

Clip length bounds: one clip of 0.5 to 2 seconds.

View Source 
const DefaultFAR = 0.01

DefaultFAR selects the strictest calibrated operating point when Request.FAR is zero.

View Source 
const MaxKeywordPositions = 128

MaxKeywordPositions bounds a keyword's text features: decoder steps plus one separator between words.

View Source 
const SampleRate = 16000

SampleRate is the required input rate in Hz. Input is mono.

Variables

View Source 
var (
	// ErrNotInitialized is returned by New when ORTLibrary is empty and the
	// caller has not initialized the ONNX Runtime environment.
	ErrNotInitialized = ortruntime.ErrNotInitialized
	// ErrClosed is returned by requests on a closed engine.
	ErrClosed = errors.New("phonmatchnet engine is closed")
	// ErrInvalidAudio reports a wrong sample rate, a clip outside 0.5-2 seconds,
	// or a sample that is nonfinite or outside [-1, 1].
	ErrInvalidAudio = errors.New("invalid audio")
	// ErrInvalidKeyword reports keyword text with no English letters, with
	// digits, or longer than MaxKeywordPositions features. It is also returned
	// for a zero Keyword.
	ErrInvalidKeyword = errors.New("invalid keyword")
	// ErrUnknownFAR is returned when Request.FAR is not a calibrated operating
	// point; see Engine.Thresholds.
	ErrUnknownFAR = errors.New("FAR is not a calibrated operating point")
)

Functions

func NormalizeKeyword 

func NormalizeKeyword(text string) (string, error)

NormalizeKeyword applies the policy used during training: NFKD, accents and punctuation removed, lowercase, words joined by single spaces. Apostrophes inside words are kept. Numbers must be spelled out.

Types

type Config 

type Config struct {
	// ORTLibrary is an ONNX Runtime library path or OS-loader name. When set, the
	// engine initializes the environment through ortenv, which retains it until
	// process exit. When empty, the caller owns the environment and must
	// initialize it before New and destroy it only after Close.
	ORTLibrary string
	// ModelDir overrides the embedded models with a directory holding
	// phonmatchnet/, google/ and g2p/ laid out like the embedded models/ (the
	// three Hugging Face -onnx downloads). Usually empty.
	ModelDir string
	// Threads sets intra-op threads per session. Zero selects 1.
	Threads int
}

Config selects the models and configures the sessions.

type Engine 

type Engine struct {
	// contains filtered or unexported fields
}

Engine detects keywords. Requests and Close are serialized.

func New 

func New(c Config) (_ *Engine, err error)

New loads and validates the models before accepting requests. Close is mandatory. Concurrent calls on one engine are serialized, including Close.

func (*Engine) Close 

func (e *Engine) Close() error

Close destroys the sessions. It is idempotent. The ONNX Runtime environment stays initialized for other engines and later calls to New.

func (*Engine) Detect 

func (e *Engine) Detect(ctx context.Context, r Request) (Result, error)

Detect scores one clip against a keyword.

func (*Engine) Initialization 

func (e *Engine) Initialization() Initialization

Initialization returns how long New spent in each loading stage.

func (*Engine) Keyword 

func (e *Engine) Keyword(ctx context.Context, text string) (Keyword, error)

Keyword normalizes text and computes its g2p decoder features. Each word is encoded, then decoded greedily; every decoder state before the end token is one feature row, and a separator row sits between words.

func (*Engine) ModelInfo 

func (e *Engine) ModelInfo() ModelInfo

ModelInfo returns the validated tensor metadata of the four graphs.

func (*Engine) Thresholds 

func (e *Engine) Thresholds() map[float64]float64

Thresholds returns the calibrated operating points: target false-accept rate to score threshold. Each was fitted on held-out development speakers; the achieved rate on other audio differs, so applications should validate a threshold on their own positive and negative clips and compare it with Result.Score.

Example

Choose the more sensitive calibrated operating point, or compare Result.Score with a threshold validated on your own recordings. 

package main

import (
	"context"
	"fmt"
	"log"

	phonmatchnet "github.com/androiddrew/go-phonmatchnet"
)

// loadClip stands in for the application's audio decoding. The engine requires
// one 0.5-2 second clip of 16 kHz mono float32 PCM in [-1, 1].
func loadClip() []float32 { return make([]float32, phonmatchnet.SampleRate) }

func main() {
	engine, err := phonmatchnet.New(phonmatchnet.Config{ORTLibrary: "libonnxruntime.so"})
	if err != nil {
		log.Fatal(err)
	}
	defer engine.Close()
	for far, threshold := range engine.Thresholds() {
		fmt.Printf("target FAR %g: score >= %.4f\n", far, threshold)
	}
	keyword, err := engine.Keyword(context.Background(), "okay naptrap")
	if err != nil {
		log.Fatal(err)
	}
	result, err := engine.Detect(context.Background(), phonmatchnet.Request{
		Samples: loadClip(), SampleRate: phonmatchnet.SampleRate, Keyword: keyword, FAR: 0.1,
	})
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(result.Accepted)
}

type GraphInfo 

type GraphInfo struct {
	Inputs  []TensorInfo `json:"inputs"`
	Outputs []TensorInfo `json:"outputs"`
}

GraphInfo describes one ONNX graph's inputs and outputs.

type Initialization 

type Initialization struct {
	RuntimeVersion string  `json:"runtime_version"`
	RuntimeSeconds float64 `json:"runtime_initialization_seconds"`
	ModelSeconds   float64 `json:"model_load_seconds"`
}

Initialization is a diagnostic breakdown of New's loading time. ModelSeconds includes reading, hashing and inspecting the graphs as well as creating the sessions. RuntimeVersion is the loaded native ONNX Runtime version.

type Keyword 

type Keyword struct {
	// contains filtered or unexported fields
}

Keyword holds a keyword's precomputed text features. Create it with Engine.Keyword and reuse it across clips. The zero Keyword is invalid.

func (Keyword) Positions 

func (k Keyword) Positions() int

Positions returns the number of text feature rows.

func (Keyword) Text 

func (k Keyword) Text() string

Text returns the normalized keyword.

type ModelInfo 

type ModelInfo struct {
	Google     GraphInfo `json:"google"`
	Detector   GraphInfo `json:"detector"`
	G2PEncoder GraphInfo `json:"g2p_encoder"`
	G2PDecoder GraphInfo `json:"g2p_decoder"`
}

ModelInfo describes the four loaded graphs.

type Request 

type Request struct {
	Samples    []float32 `json:"-"`
	SampleRate int       `json:"sample_rate"`
	// Keyword comes from Engine.Keyword. Reuse it across clips.
	Keyword Keyword `json:"-"`
	// FAR selects the operating point by its target false-accept rate. Zero
	// selects DefaultFAR.
	FAR float64 `json:"far"`
}

Request scores one clip against a keyword. Samples must remain unmodified until Detect returns. No gain, clipping, trimming, channel conversion or resampling is performed.

type Result 

type Result struct {
	Keyword   string  `json:"keyword"` // normalized keyword text
	Score     float64 `json:"score"`
	Logit     float64 `json:"logit"`
	Threshold float64 `json:"threshold"`
	TargetFAR float64 `json:"target_far"`
	Accepted  bool    `json:"accepted"`
	Timings   Timings `json:"timings"`
}

Result is one detection. Score is sigmoid(Logit), the keyword-match probability. Accepted reports Score >= Threshold.

type TensorInfo 

type TensorInfo struct {
	Name  string  `json:"name"`
	Shape []int64 `json:"shape"`
	Type  string  `json:"type"`
}

TensorInfo describes one graph input or output. Dynamic dimensions are -1.

type Timings 

type Timings struct {
	EmbeddingSeconds float64 `json:"embedding_seconds"` // Google speech embedding graph
	DetectorSeconds  float64 `json:"detector_seconds"`  // PhonMatchNet graph
}

Timings is a diagnostic breakdown of one Detect call. Each stage includes tensor creation, the synchronous Run and output validation.

Source Files

View all Source files

Directories

Path Synopsis
internal
nativetest
Package nativetest runs native ONNX Runtime tests in fresh processes.
 Package nativetest runs native ONNX Runtime tests in fresh processes.
ortruntime
Package ortruntime decides who owns the process-global ONNX Runtime environment.
 Package ortruntime decides who owns the process-global ONNX Runtime environment.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.