Building a Custom Wake Word Detector for Android — Without a Cloud or API Key
“Hey Siri.” “OK Google.” “Alexa.” — What if the device woke up only for
YOUR name?
Introduction
Voice assistants are everywhere — but they all listen for a keyword chosen by a large company. What if you could build a device that wakes up only when you call it by your own name? That was the idea behind Hello Vivek: a fully custom wake word detector that runs entirely on an Android phone, with no internet connection, no cloud subscription, and no third-party API key.
This article walks through how I built it — from training a machine learning model on a laptop to shipping it as an Android SDK that any developer can drop into their app in under ten minutes.
What Is a Wake Word Detector?
A wake word detector is a program that listens to a continuous stream of audio and raises an alert only when it hears a specific phrase the wake word. Think of it as a very focused ear: it ignores everything else and only reacts to its trigger phrase.
Wake word detectors must be extremely efficient because they run non-stop, 24 hours a day, on battery-powered devices. A model that is too slow or too large would drain the battery in hours. The Hello Vivek model is only 42 KB about the same size as a small photo and runs in under 5 ms on a mid-range Android phone.
How It Works
The system has three layers: audio capture, feature extraction, and a neural network classifier.
1 — Listening to Audio
The phone’s microphone records audio in short bursts called chunks , each one just 10 milliseconds long. These chunks are assembled into a rolling 1-second window that is constantly updated. Every 20 milliseconds, the system checks whether the last second of audio contains the wake word.
2 — Feature Extraction
Raw audio waveforms are hard for a neural network to understand directly. Instead, the audio is converted into a log-mel spectrogram a compact visual representation of which frequencies are present at each moment in time, similar to a musical score. This spectrogram is what the model actually analyses.
3 — Neural Network Classification
A small neural network called a DS-CNN (Depthwise Separable Convolutional Neural Network) examines the spectrogram and outputs a confidence score between 0 and 1. A score above the threshold (0.80) means “Hello Vivek” was detected. A score below means nothing
important happened.
4 — Voice Activity Detection (VAD)
To save battery, the classifier only runs when the phone detects actual sound. A lightweight Voice Activity Detector measures the energy of each audio chunk and skips the neural network entirely when the room is quiet.
Training the Model
Training a wake word model means teaching it the difference between “Hello Vivek” (positive) and everything else (negative). The tricky part is collecting enough examples of both.
Generating Training Data
Since recording hundreds of real voice samples is impractical, the training data is generated automatically using Microsoft Edge TTS a free, high-quality text-to-speech engine with over 300 voices.
Get VoiceAIEngineer’s stories in your inbox
Join Medium for free to get updates from this writer.
The script generates “Hello Vivek” in multiple accents, speeds, and styles. To match an Indian English accent, voices from the en-IN locale are weighted three times more heavily than other accents.
Data Augmentation
To make the model robust to the user’s real voice (which sounds slightly different from TTS), each positive clip is augmented 10 times with random transformations: speed changes (0.8× — 1.2×), pitch shifts (±4 semitones), time shifts (±200 ms), room reverb simulation, and background noise mixing. This turns 560 clips into over 6 000 training examples.
Label Smoothing
One subtle but important trick: label smoothing. Without it, the model becomes overconfident scoring ambient room noise at 98–99% confidence and triggering constantly even in silence. Label smoothing teaches the model to target a maximum confidence of 95%, keeping scores realistic and dramatically reducing false positives.
The Android SDK
The model is packaged as an Android AAR library called wwd-sdk. Any Android developer can drop it into their app and start listening for “Hello Vivek” in about ten lines of code.
The SDK handles everything internally: microphone capture, VAD gating, feature extraction, TFLite inference, cooldown management, and thread safety. No TFLite dependency is needed in the consuming app , it is bundled inside the AAR.
Thread Design
Audio capture runs on a dedicated high-priority thread
(THREAD_PRIORITY_URGENT_AUDIO) to prevent gaps in the audio stream. Inference runs on a separate background thread so it never blocks audio capture. The callback is always delivered on the main UI thread, making it safe to update the interface directly.
Key Challenges
• Silent room false positives
The first trained model triggered at 98–99% confidence even in complete silence. Root cause: the model was trained without representative room noise, and per-clip normalisation was amplifying the tiny energy of ambient sound. Fixed by generating 400 synthetic background noise clips and applying label smoothing during training.
• TTS voice vs. real voice gap
A model trained only on TTS voices scores a real human voice lower (79–82%) than expected. The fix is to heavily bias training data toward Indian English TTS voices (3× more en-IN samples), use wider augmentation ranges, and add room reverb simulation to bridge the gap between studio-clean TTS and real-world audio.
• Keras version conflict
TensorFlow 2.16+ ships Keras 3 by default, but older models were saved with Keras 2. The quantisation script failed to load models saved by the wrong version. Fixed by migrating fully to Keras 3 and updating requirements to TF >= 2.16.
• INT8 quantisation accuracy
Quantising from 32-bit float to 8-bit integers risks losing accuracy. A calibration dataset of 200 samples (100 positive + 100 negative) is fed through the converter before quantisation. The final model achieves 100% accuracy on the test set at both float and INT8 precision.
What’s Next
The model performs well with TTS training data, but adding even 30–50 real voice recordings would push accuracy above 93%. Future improvements include:
• Recording personal voice samples on the Android device and adding them to training
• Connecting wake word detection to a full speech recognition pipeline
• Packaging the SDK for iOS using TFLite’s Swift bindings
• Building a browser-based demo using TFLite’s JavaScript runtime
The entire system from raw audio to wake word callback runs in
under 5 ms, fits in 42 KB, and works with no internet connection.
Technology Stack
Closing Thoughts
Building a custom wake word detector from scratch sounds daunting, but the combination of free TTS for data generation, open-source training frameworks, and TFLite’s efficient on-device runtime makes it surprisingly accessible. The whole pipeline from raw audio file to a working Android app can be completed on a standard laptop in an afternoon.
More importantly, the result is something none of the big voice assistants offer: a wake word that belongs entirely to you, runs entirely on your device, and sends your voice data exactly nowhere.
