In our recent Agora × Google Discord livestream, Thor from Google DeepMind and Hermes from Agora discussed a recurring question in voice AI: when does native audio make sense, and when is a cascaded voice pipeline still the better fit?

From an AI DJ and live transcription to reasoning, context, tool calling, and multimodal input, the conversation focused on the tradeoffs developers face when choosing how to build a voice AI application.

Native audio vs. cascaded pipelines

Thor opened with an AI DJ built on Gemini Live.

Wacth the AI DJ demo video

The model held a live conversation, switched between German, Cantonese, and Hindi, and generate music.

The demo also highlighted a core advantage of native audio: the model works directly with speech instead of first reducing it to text. That lets it work with information such as tone and delivery that may be lost in transcription — including sarcasm.

A cascaded pipeline works differently: speech is transcribed, sent to an LLM, and then converted back to audio with TTS.

The tradeoff is control. Native audio keeps more of the interaction inside one model, while a cascaded pipeline lets developers choose and tune STT, the LLM, and TTS separately.

What should transcription preserve?

Thor then showed a coffee-order demo comparing verbatim and smart transcription.

Wacth the Latte-order transcription demo video

As he changed his order mid-sentence, verbatim mode preserved the full sequence, while smart transcription resolved the corrections into the final request.

Hermes pointed out that this could also mean fewer tokens for the downstream LLM to process. Thor agreed that this could be useful, but noted that the right choice depends on the use case: an ordering system may only need the final intent, while medical or other record-sensitive scenarios may need a one-to-one transcript.

A community example is Zico’s Agora MatchCast, which combines Agora WebRTC with Gemini Transcribe Live to generate real-time captions for League of Legends commentary.

Watch: Live captioning built by Zico

Reasoning vs. latency

Native audio brings another tradeoff: more reasoning can also mean more latency.

Thor described two approaches in Gemini Live.

Interleaved reasoning keeps reasoning inside the native audio model and is suited to lower-latency tasks.

Extended thinking can offload heavier reasoning to a text-based model in the background.

To demonstrate the difference, Thor asked the model to draw an SVG of a pelican riding a bicycle.

The native model returned quickly, although the drawing was rough. With extended thinking, the intended behavior was for the background reasoning to keep working across multiple turns while the voice conversation continued.

The live demo hit a system error, but the idea was simple: heavier reasoning can run in the background without blocking the conversation.

Context, tools, and multimodal input

During the Q&A, Thor returned to the choice between native audio and a cascaded pipeline.

For applications with large amounts of context, many tools, or high requirements for tool-calling accuracy, he said a cascaded pipeline may still be the better fit.

Hermes added that, in Agora’s cascaded architecture, developers can control the LLM endpoint and apply RAG, summarization, or other processing before sending context to the model. Server-side tools and client-side actions can also follow different execution paths.

The discussion then moved into multimodal input. Thor explained that Gemini Live can accept image frames in addition to audio — for example, frames from a user’s screen — so the model can reason about what is visible in the interface.

Akshay Nandwana’s Live Lens is a community example: the Android app combines camera input with a live voice conversation, using Gemini Live for visual understanding and Agora for the real-time voice and session layer.

Watch: Live Lens, built by Akshay Nandwana (Agora)

Start with the application

The livestream did not produce a single winner between native audio and cascaded pipelines.

The more useful questions are:

• What information needs to survive the speech layer?
• Do you need a verbatim transcript or the user’s final intent?
• How much context and tool calling does the agent need?
• How much latency can the experience tolerate?

The architecture should follow those requirements — not the other way around.

Architecture Tradeoffs in Native Audio, Transcription, and Tool Calling was originally published in Agora.io on Medium, where people are continuing the conversation by highlighting and responding to this story.

Create a voice interview partner, then explore how a Go backend starts and stops the conversation.

In this tutorial, you’ll run a Go voice agent that helps you practice talking about a project. Open the app in your browser, describe something you built, and hear the agent ask a follow-up based on your answer. Your project can use any programming language.

You’ll use the existing Agora Go Quickstart Recipe, which provides a Gin backend and a Next.js client. The browser handles microphone input and playback; the Go service configures the cloud agent and starts and stops its sessions.

Before you start

• Go 1.23 or later and basic familiarity with Go functions and structs.
• Node.js 20.9 or later and pnpm. The starter pins pnpm 10.23.0.
• The Agora CLI, installed and available as agora.
• An Agora project with an App ID and App Certificate, and RTC, RTM, and Conversational AI managed-provider support enabled. You can manage your project in the Agora Console.
• A browser with microphone access. Headphones help keep the agent’s voice out of your microphone.

The project uses Make and Bash. Follow along on macOS, Linux, or Windows through WSL with those tools available.

1. Get start

Sign in and create a project from the recipe:

agora login
agora init my-go-agent --recipe golang-quickstart
cd my-go-agent
make setup

make setup downloads the Go modules, installs the frontend workspace dependencies, and prepares server/.env. The main files you'll use are:

server/
  main.go        HTTP routes and environment loading
  agent.go       Agent instructions, providers, and sessions
client/
  next.config.ts Browser API rewrites to the Go backend
Makefile         Setup, development, and verification commands

Open server/.env and replace the two example credential values with those from your Agora project. Keep the other entries supplied by setup:

AGORA_APP_ID=your_agora_app_id
AGORA_APP_CERTIFICATE=your_agora_app_certificate

Alternatively, populate the environment file through the CLI from the project root, replacing <your-project> with your project's name or ID:

agora project use <your-project>
agora quickstart env write .

Check that server/.env contains your project's values. Keep the App Certificate in the Go backend; the browser receives a signed token and does not need the certificate.

2. Configure the interview assistant

Before starting the app, open server/agent.go. Near the top, replace the existing const block containing adaPrompt and defaultGreeting with:

const (
    adaPrompt = `You are Ada, a friendly interviewer helping a developer
practice discussing a software project in English, regardless of the
programming language used.

Ask one question at a time and wait for the answer.
After each answer, point out one specific strength or one missing detail,
then ask a follow-up based on something the developer just said.
Focus on the problem, the developer's decisions, and what they learned.
Keep each turn to two or three short sentences suitable for speech.
If an answer is unclear, ask for clarification before evaluating it.
Treat this as practice, without assigning a hiring score.
`
    defaultGreeting = "Hi! Let's practice talking about a project. What did you build, and what problem did it solve?"
)

Keep the existing constant names. The starter passes adaPrompt into the agent's instructions and uses defaultGreeting for its opening message. The greeting gives the learner something specific to answer immediately; The instructions guide subsequent turns.

This exercise uses English because the supplied speech recognition configuration selects English. The recipe already configures managed Deepgram speech recognition, an OpenAI language model, and MiniMax speech output, so you don’t need to add separate provider API keys for this path.

3. Try a short interview

From the project root, start both services:

make dev

This starts Gin on port 8000 and Next.js on port 3000. It also sets AGENT_BACKEND_URL=http://localhost:8000 for the frontend's API rewrites. Keep the terminal running.

Open http://localhost:3000, then:

1. Click Start Conversation and allow microphone access.
2. Listen for the interview greeting you just configured.
3. Answer aloud. For example, “I built a small service that collects sensor readings. I used a queue so requests would not have to wait for processing.”
4. Listen for one specific strength or missing detail in your answer, followed by a related question. For the example above, the agent might ask how you handled a growing queue. Check that your speech and the agent’s response appear in the transcript.
5. Answer the follow-up, then click End Conversation.

The sensor-service answer is a practice example; use your own project if you have one. The agent’s exact response will vary. Check that its feedback refers to your answer and that it asks one question before waiting for you to speak.

Start another conversation to check that a new interview begins with the greeting. If you edit the Go instructions later, end the active conversation, stop the dev processes with Ctrl+C, run make dev again, and refresh the page. This starter runs go run . without a Go file watcher, so saving a file alone won't reload the backend.

4. How a conversation starts and ends

The browser talks to three API paths. Next.js rewrites them to handlers in server/main.go:

These requests control the conversation. Agora RTC carries the live audio between the browser and the cloud agent; RTM carries transcript and agent-state updates. Go doesn’t receive an HTTP request for each microphone audio chunk.

Starting a session

Inside server/agent.go, the service creates an authenticated SDK client using the backend credentials. It then builds an agent with your instructions, greeting, speech providers, and turn-detection settings.

The next step connects that configuration to a channel and its participants. Here is the existing session-start portion of the start method, shown for explanation; keep the surrounding implementation in place:

session := agent.CreateSession(agentkit.CreateSessionOptions{
    Channel:         channelName,
    AgentUID:        strconv.Itoa(agentUID),
    RemoteUIDs:      []string{strconv.Itoa(userUID)},
    EnableStringUID: &enableStringUID,
    IdleTimeout:     &idleTimeout,
    ExpiresIn:       expiresIn,
})

ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

agentID, err := session.Start(ctx)
if err != nil {
    return nil, fmt.Errorf("start agent: %w", err)
}

Channel names the conversation. AgentUID identifies the agent participant, and RemoteUIDs selects the user it listens to. The agentID returned by Start identifies the running agent session for later lifecycle calls.

The Go context gives the startup call a 30-second deadline. Conversation duration is controlled separately through settings such as IdleTimeout and ExpiresIn. In the referenced starter, those are 30 seconds of idle time and a one-hour session expiry.

Ending a session

After startup, the backend retains the session under its agent ID. Clicking End Conversation sends { agentId } to the stop endpoint, where Go uses that ID to stop the session. The browser also logs out of RTM and clears its conversation state. You can read the full stop handling in server/agent.go.

End the conversation through the UI before stopping the local server so the app can send that explicit stop request.

Troubleshooting

If the app doesn’t start, check your local setup from the project root:

make doctor-local

doctor-local checks local prerequisites and whether the credential fields are configured. It doesn't make a live Agora call; a browser conversation is still needed to check microphone input and spoken replies.

If you later change backend routes or frontend API handling, the repository’s local verification guide covers the Go tests and checks using a fake backend. Those checks help find local wiring errors; they don’t test a real voice conversation.

Make the next practice round more specific

Once the interview works, change one instruction to focus on error handling. Ask Ada to follow up on what happens when part of the system fails. Restart the backend, begin a new conversation, and give the same sensor-service answer.

Listen for a question tied to that answer, such as what happens if processing fails after a reading enters the queue. If Ada moves to an unrelated interview question, revise the instruction and try again. This gives you a concrete check: does each follow-up build on what you just said?

Implementation reference: this tutorial follows the Go quickstart at revision d98ba88. The CLI may fetch a newer version; compare its files with that snapshot if paths or defaults differ.

Build Your First Voice Agent in Go with the Agora Agents SDK was originally published in Agora.io on Medium, where people are continuing the conversation by highlighting and responding to this story.

You have 45 minutes before a meeting, with a pull request to review and two emails waiting. What would you do first? We can build a voice assistant to help you talk through your priorities and plan how to use that time.

The NextJS Quickstart Recipe gives you a working Next.js app with a voice assistant named Ada, which answers developers’ questions about Agora. You’ll first try that demo, then turn Ada into a planning assistant by changing its instructions and greeting. Finally, you’ll update the page text so users know to share their tasks and available time. You’ll be able to hear Ada’s suggestions and follow the exchange in a live transcript.

The app’s React components and server API routes live in one TypeScript project. Agora runs the voice agent in the cloud, while the app provides the conversation interface. Basic TypeScript, React, and terminal knowledge are enough to follow along.

Before you start

You’ll need:

• Node.js 22 or later and pnpm.
• The Agora CLI, available as agora in your terminal.
• An Agora project with an App ID and App Certificate, with RTC, RTM, and Conversational AI managed-provider support enabled. Use the Agora Console to manage the project.
• A browser with microphone access. Headphones help prevent the agent’s voice from feeding back into the microphone.

This tutorial uses English and Agora-managed providers, so you don’t need separate provider API keys.

1. Get started

Sign in, create an app from the recipe, and install its dependencies:

agora login
agora init my-ts-agent --recipe nextjs-quickstart
cd my-ts-agent
pnpm install

The initialization flow binds an Agora project and prepares its environment configuration. Open .env.local in the project root and check these two entries against your project. If the file wasn't created, copy env.local.example to .env.local and fill them in:

NEXT_PUBLIC_AGORA_APP_ID=your_agora_app_id
NEXT_AGORA_APP_CERTIFICATE=your_agora_app_certificate

Keep the certificate’s variable name exactly as shown. NEXT_PUBLIC_AGORA_APP_ID is available to the browser; NEXT_AGORA_APP_CERTIFICATE stays in server code and must not receive the NEXT_PUBLIC_ prefix.

From the project root, check the local setup and start Next.js:

pnpm run doctor
pnpm run dev

doctor checks the Node version, package-manager context, and presence of the environment fields. It doesn't verify that the credentials can start a real agent. Keep the dev terminal running and open http://localhost:3000.

First, check the original demo’s audio and transcript. Click Start Conversation, allow microphone access, and listen for Ada’s greeting. Ask a short question in English, such as “How can you help me?” Check that you hear a reply and that both speakers appear in the Transcript panel. Then click End Conversation. Resolve any startup or audio problems using the troubleshooting section before making the edits below.

2. Turn Ada into a planning assistant

Now give Ada a specific job: learn the user’s tasks and available time, then help them choose an order. The prompt sets these conversational rules, and the greeting invites the user to share their tasks.

Open app/api/invite-agent/route.ts, where the server uses the Agora Agents SDK to configure and start the cloud agent. Replace the existing ADA_PROMPT and GREETING definitions near the top with:

const ADA_PROMPT = `You are Ada, a practical planning partner.
Help the user plan their next work session. Speak in English.

Learn what tasks they need to do and how much time they have.
If an important detail is missing, ask one focused question and wait.
Once you have enough information, suggest at most three actions in order
and give a brief reason for the first priority.
If the tasks do not fit the time available, ask what can wait.
Keep each turn to two or three short sentences suitable for speech.
Do not claim to save tasks, set reminders, or update a calendar.`;

const GREETING = `Hi! Tell me what you would like to get done in your next work session.`;

Keep the constant names: the existing Agent configuration reads ADA_PROMPT as its instructions and GREETING as the opening message. These edits guide the conversation; the starter has no task-saving or calendar integration. Leave the provider and session settings in place.

3. Introduce the planner on the page

Ada now has planning instructions, but the page still introduces a general Agora voice demo. Update the start screen so someone opening the app knows to tell Ada their tasks and available time.

In components/QuickstartPreCallCard.tsx, replace the existing <h1> and the first <p> immediately below it with:

<h1 className="text-[28px] font-medium leading-[1.2] text-white">
  Plan your next work session
</h1>
<p className="mt-[14px] text-sm font-medium leading-6 text-muted-foreground">
  Tell Ada your tasks and available time, then talk through a short plan.
</p>

These elements sit above the existing start button. Keep that button, its loading state, and its onClick handler so the page still starts a conversation in the same way.

Give the live transcript a matching title. Open components/QuickstartTranscriptPanel.tsx and replace the existing <h2> containing Transcript and its Live voice turns subtitle with:

<h2 className="text-sm font-semibold text-foreground">
  Planning conversation
</h2>
<p className="text-xs text-muted-foreground">
  Your tasks and Ada&apos;s suggestions
</p>

The labels now describe this exercise. The messages will still use You and Agent to identify speakers. Changing these React labels changes what the user sees; the server-side prompt changes how Ada responds.

4. Try your planning assistant

Save all three files. With the previous conversation ended, stop the dev server with Ctrl+C and start it again from the project root:

pnpm run dev

Refresh the browser and check that the start card says Plan your next work session. A fresh conversation is important: editing a prompt doesn’t change an agent session that is already running.

1. Click Start Conversation and listen for the new planning greeting.
2. Say, “I have 45 minutes to review a pull request, reply to two emails, and prepare for a meeting. The meeting starts right after this work session.”
3. Check the Planning conversation panel. Your tasks and time should appear under You. Listen to Ada’s reply and follow its text under Agent.
4. If Ada asks for a missing detail, answer it. For example, you could say that meeting preparation needs 20 minutes. Check whether the suggested order accounts for the meeting and the time remaining. The exact wording and order may vary.
5. Click the microphone button to mute. After any reply already underway finishes, speak a short test sentence and check that it doesn’t become a new transcript turn. Unmute, speak again, and check that new speech appears.

The microphone selector beside the mute button lets you choose another input device if needed. The agent visualizer provides another cue as the conversation moves between listening, thinking, and speaking; it doesn’t replace checking the actual audio and text.

Before clicking End Conversation, copy any suggestion you want to keep. The transcript is held in the current React conversation view; This starter doesn’t save it as a task list. Ending the conversation unmounts that view, so starting again begins with a fresh transcript and greeting.

5. How conversation works

• Server: The app/api/ routes keep the App Certificate private, issue connection tokens, and start the agent with your instructions.
• Audio: Agora RTC carries your voice and the cloud agent’s spoken replies.
• Transcript: Agora RTM delivers transcript updates, which ConversationComponent.tsx stores in React state for display.

For more on how transcript updates become visible messages, see the repository’s transcript guide.

Troubleshoot setup and conversation issues

To regenerate .env.local, run the following from the project root. Replace <your-project> with your project's name or ID, then restart the dev server after running the commands:

agora project use <your-project>
agora quickstart env write .

Before sharing your changes, run pnpm run verify from the project root. It checks local configuration, lint, TypeScript types, mocked API contracts, and the production build. It doesn't exercise a real microphone or prove that generated suggestions follow the prompt; the browser exercise checks those behaviors.

Try the same tasks in less time

Start a new conversation with the same tasks, but say you have only 15 minutes. Check whether Ada asks what can wait or narrows the plan to fit. Check your time constraint in the transcript as well as listening to the reply. If the transcript shows the wrong number, correct it aloud before judging the planning advice.

If the suggestions still ignore the shorter session, refine the prompt’s time constraint and repeat the same example. Keep the screen’s invitation and the assistant’s behavior aligned as you change the task.

Source reference: the Recipe document is the source behind the recipe page. Code paths and defaults here follow quickstart revision 2437441; the CLI may fetch a newer version, so compare that snapshot if your files differ.

Build Your First Voice Agent in TypeScript with the Agora Agents SDK was originally published in Agora.io on Medium, where people are continuing the conversation by highlighting and responding to this story.

Voice AI is taking on more everyday tasks, with developers building apps for group decisions, character conversations, and hands-free computer control. In the fourth Agora Dev Digest, we explore these demos, catch up on October’s Tech Talks, and share a recipe for keeping agents responsive while they work.

Demo Spotlight

Agora Decision Rooms: talk together, decide together

@Shivam Mahendru built Agora Decision Rooms for group voice conversations with Kabir, an AI facilitator. The demo takes a familiar problem, deciding where to go, and gives everyone a chance to discuss it and vote.

GitHub

KIN: create AI characters you can talk to

@Shubham’s KIN lets users describe a character by voice, define their personality and backstory, and start a real-time voice conversation.

Built with Flutter, Agora, and Google Gemini, KIN extracts and saves details from conversations, such as user preferences, important events, and promises, to use as context in future conversations.

GitHub

Agora for Developers

Control your Mac by voice with Jev + Agora

@Akshay built a demo using TypeSafe’s Jev and Agora to control his Mac by voice. It shows voice commands for playing a video, opening folders, navigating the web, and closing tabs.

Watch the demo · GitHub

Tech Talks

Getting Started with Agora Conversational AI: An introduction to Agora’s developer ecosystem, covering the CLI, Skills for AI coding assistants, MCP, Next.js Quickstart, and Voice AI Recipes.

Watch the replay

Inside Agora Conversational AI: What Powers a Real-Time AI Agent? A look beyond STT → LLM → TTS, covering turn detection, interruptions, agent states, and how tools, memory, and multimodal inputs fit into a voice AI application.

Watch the replay

Beyond Voice Chatbots: Build AI Agents That Take Action: Explore how agents connect to APIs and services through tool calling, use RAG and memory, and support multimodal inputs, with Agora adding real-time voice interactions.

Watch the replay

Voice AI for Bharat: The Diwali Edition: Explore the challenges of building voice AI for India, from Hindi, English, and Hinglish to Indian accents, language switching, and moving from demo to production.

Watch the replay

Build This Today

Delegated Reasoning: keep talking while your agent works

Agora’s Delegated Reasoning recipe separates the conversation from the work behind an answer. GPT Live handles speaking and listening, while a second model reasons through the request and calls tools.

The example is a trip planner. Ask for somewhere sunny to visit, and the agent checks forecasts for several cities before recommending one. You can keep talking while the lookup runs. Once you choose a destination, it saves the trip; changing your mind updates the same record.

The recipe includes setup instructions and an MCP tool server for checking forecasts and saving trips. It requires access to Agora’s GPT Live preview and the OpenAI models used in the example.

Explore the recipe

Share Your Project

Building something with Agora? We’d love to see it! Share your project and tag @AgoraIO on X. We’ll be spotlighting community builds from time to time, and yours could be featured in a future edition!

Agora Dev Digest #4 was originally published in Agora.io on Medium, where people are continuing the conversation by highlighting and responding to this story.

Imagine speaking a sentence in Spanish and hearing it back in English. That’s what this tutorial gets running: a browser-based voice translator built on Agora’s Real-time Translator project, where Deepgram turns your speech into text, a managed OpenAI stage translates it, and MiniMax speaks the translation back into the same channel.

By the end, you’ll have the whole thing running locally on your own machine, holding a full speech-to-speech conversation. One caveat up front: end-to-end latency is second-scale.

This walkthrough covers the recipe’s own commands, the environment variables that control the language pair, the audio path from microphone to translated speech, and the limitations to resolve before shipping.

What You Will Get

By the end of this tutorial, you will have:

• A browser-based translator that accepts speech and plays back the translation.
• A working Spanish-to-English configuration to try first.
• A starting point you can customize with a different source language, target language, and voice.

The project uses Agora-managed OpenAI for translation, so the default setup does not require your own OpenAI API key.

Architecture at a Glance

The project combines a web interface, a small backend, and Agora’s cloud-based voice pipeline:

Browser / Next.js (:3000)
    │
    ├─ controls ──▶ FastAPI (:8000)
    │                 │
    │                 └─ starts/stops agent
    │
    └─ audio ⇄ Agora Conversational AI
                       │
                 Deepgram STT
                       │
                 OpenAI translation
                       │
                 MiniMax TTS

The project has a browser app and a small backend running locally. Together, they start and control a translation session in Agora’s cloud, where speech is recognized, translated, and spoken back to the browser.

On the credentials side, the pipeline is zero-key by default: OpenAI is Agora-managed, so no OPENAI_API_KEY is required unless your account specifically needs you to bring your own.

ServicePathPortResponsibilityWeb clientweb/3000RTC/RTM lifecycle, UI, calls /api/* onlyAgent backendserver/8000Token minting, agent session lifecycleAPI docsserver/8000Interactive reference at /docs

Get started

1. Prepare the tools

Before you begin, install:

• Python 3.10 or newer.
• Bun, which runs the project’s setup and development commands.
• Agora CLI, which scaffolds the project and can write its Agora credentials.

2. Create the project

Open a terminal and initialize the project:

agora init my-realtime-translator --recipe realtime-translator
cd my-realtime-translator

Run the remaining terminal commands from this project directory.

Install the project dependencies:

bun run setup

This installs the web dependencies, prepares the Python environment, and creates server/.env.local from the sample configuration if that file does not already exist.

3. Connect your Agora project

Log in, select your project, and write its credentials:

agora login
agora project use <your-project>
agora project env write server/.env.local

Replace <your-project> with the Agora project you want to use.

Alternatively, open server/.env.local and replace the existing credential values manually:

AGORA_APP_ID=your_agora_app_id
AGORA_APP_CERTIFICATE=your_agora_app_certificate

These are entries in the existing file, not a replacement for its entire contents. Keep the App Certificate in the backend configuration.

4. Check the language settings

For your first run, use the project’s default Spanish-to-English pair. In server/.env.local, make sure these entries have the following values, keeping the other settings in place:

SOURCE_LANG=es
TARGET_LANG=English
TTS_VOICE=English_captivating_female1

The project already supplies its model and other defaults. You can leave those settings as they are for this walkthrough.

The table above is the quickest way to sanity-check a misconfiguration before you start reading logs.

5. Start the application

From the project directory, run:

bun run dev

This starts the web app on port 3000 and the backend on port 8000. Keep the terminal running and open http://localhost:3000. You should see the translator interface with the Start Conversation button.

6.Try your first translation

With the interface open, run through one round trip:

1. Click Start Conversation.
2. Allow microphone access when your browser asks.
3. Say a short Spanish sentence, such as “Hola, ¿cómo estás?”
4. Pause and listen for an English translation, such as “Hello, how are you?”

The wording may vary. Check that the response preserves the meaning of your sentence and is spoken in English. The transcript panel also lets you inspect the recognized speech and the agent’s response.

Try a question next, such as “¿Dónde está la estación de tren?” The agent should translate the question into English rather than answer it with directions. This is a useful way to check that your demo behaves as a translator.

Click End Conversation when you finish testing.

Customize the Translation Direction

Once the default translation works, you can return to server/.env.local and try a different translation direction. The default is Spanish → English. These three settings control the language pair and the voice:

• SOURCE_LANG sets the language you speak. Use a code supported by Deepgram’s nova-3 model, such as es for Spanish. See the Models & Languages Overview for the available codes.
• TARGET_LANG sets the language the text should be translated into, such as English.
• TTS_VOICE sets the voice that reads the translation aloud. Choose a MiniMax voice that matches TARGET_LANG; the Agora MiniMax TTS guide explains the voice_id setting and links to the System Voice ID List.

After saving your changes, end any active conversation, stop the development process with Ctrl+C, and run bun run devagain. Then start a new conversation in the browser. This reloads the configuration and creates an agent session with your new settings.

Next Steps

Try the translator with the names, numbers, and questions your users are likely to ask. Check both the recognized input and spoken output before adapting the demo for a real application.

For deployment, check out the guide in the repository, it explains how to host the web app and backend and configure the connection between them. The sample backend exposes token and session controls without user authentication; add access controls before offering those endpoints to real users.

Frequently Asked Questions

1. How much does it cost to run a real-time voice translator?
   Conversational AI Engine audio tasks cost USD 0.10 per minute. The first 300 minutes are free each month. See the Conversational AI pricing page for current rates.
2. Why does the agent start but not translate anything?
   Check SOURCE_LANG. It must be a supported Deepgram language code, such as es or es-419, rather than a language name such as Spanish. Then verify that TARGET_LANG and TTS_VOICE match your intended translation direction and restart the session.
3. Where should the App Certificate live?
   Keep it in the backend environment, such as server/.env.local. Do not expose it in the browser or commit it to source control.
4. How do I add live captions?
   Enable the RTM data channel for the agent and handle transcript updates in the client. Follow Agora’s live transcript guide for the required configuration.
5. How do I deploy the backend so the web app can reach it?
   Deploy the FastAPI backend at a reachable URL, then set AGENT_BACKEND_URL in the web app’s environment. The deployment guide covers the remaining setup.

References

• Real-time Translator project repository.
• Architecture and component responsibilities.

How to Run a Real-Time Voice Translator with Agora Conversational AI was originally published in Agora.io on Medium, where people are continuing the conversation by highlighting and responding to this story.

Run a conversation with Ada, turn her into a Python tutor, and follow the path from your microphone to her spoken reply.

Open a browser, ask Ada a question, and hear her answer while the transcript appears on screen. Then change her instructions and start a new conversation: the same assistant now helps you learn Python.

This tutorial uses Agora’s Python Quickstart Recipe, which includes a web client and a Python backend built with FastAPI and the Agora Agents SDK. You’ll run the existing project, customize it, and then examine how the conversation works. Basic Python and terminal knowledge are enough.

The code examples follow quickstart revision bd1af724. The CLI may fetch a newer version; compare its files with that snapshot if names or behavior differ.

Before you start

You’ll need:

• Python 3.10 or later, available as python3 in your terminal.
• Bun and the Agora CLI.
• An Agora project with an App ID and App Certificate, with RTC, RTM, and Conversational AI enabled. RTC carries audio; RTM carries messaging updates such as transcripts.
• A browser with access to your microphone and speakers. Headphones can help prevent speaker audio from feeding back into the microphone.

The setup scripts use Bash, so use a Bash-capable environment such as macOS, Linux, or WSL on Windows.

1. Create and configure the project

Log in, create a project from the recipe, and enter its directory:

agora login
agora init my-python-agent --recipe python-quickstart
cd my-python-agent

The project has two main directories: server/ contains the Python backend, and web/ contains the browser client. Install their dependencies:

bun run setup

Before starting the app, open server/.env. Set these two entries to your project's values, leaving the other settings in place:

AGORA_APP_ID=your_agora_app_id
AGORA_APP_CERTIFICATE=your_agora_app_certificate

If you prefer to populate the file through the CLI, select your project and write its environment configuration from the project root. Replace <your-project> with your project's name or ID:

agora project use <your-project>
agora quickstart env write .

Check that server/.env now contains your credentials rather than the example values. Keep the App Certificate in the backend. The browser receives a token from the server and does not need the certificate.

2. Have your first conversation

From the project root, start the Python backend and web client:

bun run dev

Keep the terminal running. Open http://localhost:3000, click Start Conversation, and allow microphone access. Ada should introduce herself:

Hi there! I’m Ada, your virtual assistant from Agora. How can I help?

Ask a short question in English, such as “What is a voice agent?” Listen for the answer and check that your question and Ada’s reply appear in the transcript panel.

These are two useful checks. Hearing the greeting confirms playback. Receiving a spoken answer to your question exercises the full loop: microphone input, speech recognition, response generation, and speech output.

Click End Conversation before moving on. If the conversation does not work, use the troubleshooting section below before editing the agent.

3. Turn Ada into a Python tutor

Open server/src/agent.py. Find the existing ADA_PROMPT and DEFAULT_GREETING definitions near the top and replace them with:

ADA_PROMPT = """You are Ada, a patient Python tutor.
Explain beginner Python concepts using short, spoken answers.
Use one concrete example when it helps, and ask a follow-up question
when the learner's request is unclear.
"""

DEFAULT_GREETING = (
    "Hi! I'm Ada, your Python tutor. What would you like to learn?"

Keep those variable names: the existing agent wrapper reads them. The prompt guides the model’s replies; the greeting supplies the first message in a new session. Leave the rest of the file unchanged for this exercise.

Save the file. With the conversation ended, press Ctrl+C in the terminal to stop the dev processes, then restart them from the project root:

bun run dev

Refresh the browser page and start a new conversation. You should hear the Python tutor greeting. Ask “What is a Python list?” and listen for a short explanation with an example. The exact wording of the generated answer will vary.

The restart matters: the backend in the referenced version does not automatically reload your Python changes. Starting a fresh browser conversation also ensures you’re listening to a new agent session.

4. Follow a question from microphone to reply

You changed Ada’s role without changing how audio reaches her. To see why, separate the application into three parts:

• The browser captures microphone audio, plays Ada’s replies, and displays transcripts.
• The Python backend uses the Agora Agents SDK to configure, start, and stop agent sessions.
• The Agora cloud agent processes the conversation and sends spoken replies back to the browser.

The browser and cloud agent exchange audio through Agora RTC, or Real-Time Communication. Inside the agent, each turn passes through three stages:

1. Speech-to-text (STT) turns your speech into text.
2. A large language model (LLM) generates reply text using the conversation and the instructions you supplied.
3. Text-to-speech (TTS) turns that reply into audio, which returns through RTC for playback.

The referenced recipe configures Deepgram nova-3 for English speech recognition, OpenAI gpt-4o-mini for replies, and MiniMax speech_2_6_turbo for speech output. Those are this recipe's defaults, not requirements for every voice agent.

The transcript takes a separate path: Agora RTM, or Real-Time Messaging, carries text and agent-state updates to the browser. This distinction is useful when debugging: working audio with an empty transcript points you toward the messaging path.

Python controls the session lifecycle rather than receiving every audio chunk through an HTTP request. The backend supplies connection details through /get_config, starts the agent through /startAgent, and stops it through /stopAgent. The web client reaches these routes under /api/ through its development proxy.

How the Python configuration connects

Look again at server/src/agent.py. The wrapper creates an authenticated AsyncAgora client from your App ID and App Certificate. Inside Agent.start(), it passes that client and your instructions to AgoraAgent:

agora_agent = AgoraAgent(
    client=self.client,
    instructions=ADA_PROMPT,
    greeting=self.greeting,
    # Other settings are omitted from this excerpt.
)

This is an explanatory excerpt, not a replacement for the full block. The wrapper sets self.greeting from DEFAULT_GREETING, which connects your edit to the session's opening message.

The speech and language providers are then attached to the agent:

agora_agent = (
    agora_agent
    .with_stt(stt)
    .with_llm(llm)
    .with_tts(tts)
)

Finally, create_async_session(...) associates the agent with the conversation's channel and user. Calling await session.start() returns an agent ID that the backend retains for stopping the session later.

The full configuration also includes voice activity detection (VAD), which detects speech activity and helps determine pauses and interruptions. Keep those defaults while learning the basic flow; prompt changes and turn-timing changes are easier to evaluate separately.

Troubleshooting

If the conversation does not start, run this from the project root:

agora project doctor --deep

Follow the reported remediation. A passing diagnostic checks project readiness; the browser conversation remains the check for the complete audio loop.

Give Ada a focused next task

Try narrowing the tutor to one topic, such as Python lists, and ask the same few questions after each prompt change. Listen for whether the answers stay short, use examples, and ask for clarification when needed. Change one instruction at a time so you can judge what it did.

Voice can carry a guided explanation while the transcript provides something to revisit. For exact code that a learner needs to copy or compare, keep a readable on-screen version rather than relying on speech alone.

The quickstart repository contains the full implementation. With the first conversation and role change working, you have a concrete place to explore the next layer: the prompt, a speech provider, or the timing of each turn.

Build Your First Voice Agent in Python with the Agora Agents SDK was originally published in Agora.io on Medium, where people are continuing the conversation by highlighting and responding to this story.

Try saying this out loud:

कच्चा पापड़, पक्का पापड़।

Now say it faster.

For English speakers, try:

Red lorry, yellow lorry.

These sentences look harmless on a screen. Put someone on a microphone, add a playful AI host, and suddenly you have a game.

That’s what we’re building: Zubaan Race, a Hindi and English tongue twister challenge powered by Agora Conversational AI and NVIDIA Nemotron 3.5 Transcribe.

The host introduces each challenge, listens to your voice commands, and talks through your results. NVIDIA transcribes each recorded attempt. Our application compares that transcript with the challenge and calculates the score.

You can run the web app on your laptop and use a free Hugging Face GPU demo for NVIDIA inference. You won’t need to download the model or configure a GPU server for this tutorial.

Source code: AgoraIO-Community/agora-nemotron-transcribe

What we’re building

Zubaan Race is a solo browser game with ten challenges in each language.

Choose हिंदी or English at the top right, start the game, and meet Chulbul, your AI host.

Here’s the game loop:

1. Chulbul introduces a tongue twister.
2. You say the phrase.
3. A short pause ends your recording automatically.
4. NVIDIA transcribes what it heard.
5. The game calculates word match and a speed bonus.
6. Chulbul explains the result.
7. Say “try again” or “next” to continue.

After the final round, ask the host which words matched, which ones differed, and what to practice. You can also review individual rounds, replay your best recordings, and download a score card.

The interface stays within the screen. Results use previous and next controls rather than a scrolling report.

The first Start action and microphone permission require a click. After that, voice commands control the main game flow, with buttons available as fallbacks.

How Agora and NVIDIA work together

A talking game needs two kinds of speech handling.

The host needs a live conversation loop: hear a command, respond naturally, and speak to the player. Scoring needs a completed recording that we can inspect and compare with a known phrase.

Each part has a specific responsibility:

• Agora RTC carries the player’s microphone audio and the host’s spoken responses.
• Agora Conversational AI runs the host’s speech recognition, language model, and speech generation.
• Agora RTM and Client Toolkit carry host context, captions, and conversation events.
• NVIDIA Nemotron 3.5 Transcribe transcribes completed player attempts.
• The game’s scoring engine calculates word differences, match percentage, and points.

NVIDIA describes Nemotron 3.5 Transcribe as an instruction-following model for recorded speech. Its promptable transcription is useful here: we want repetitions and mistakes preserved rather than rewritten into a cleaner sentence. NVIDIA’s introduction

Nemotron is not the host’s live streaming recognizer in this application. It evaluates completed attempts separately.

The host’s live recognition handles conversation and commands. Only the NVIDIA transcript supplies the words used for scoring.

Before you start

You’ll need:

• Node.js 22 or newer.
• pnpm.
• An Agora account and project with an App Certificate, RTC, RTM, and Conversational AI available.
• Access to the managed providers used by the host.
• A free Hugging Face account and access token for your account’s GPU allowance.
• A browser with microphone access.

The host configuration uses Agora-managed Deepgram Nova-3, OpenAI GPT-4o-mini, and MiniMax TTS. Availability and usage charges depend on your Agora project.

The free part of this setup is the limited Hugging Face inference route. It does not make every service in the application unlimited or free.

Use headphones so the host’s voice is less likely to enter your microphone.

Step 1: Install Agora Skills and the CLI

The Agora CLI handles account login, project discovery, configuration, and diagnostics.

For macOS or Linux:

curl -fsSL https://dl.agora.io/cli/install.sh | sh

Open a new terminal if necessary, then check the installation:

agora version
agora doctor --json

Use a current CLI release. This workflow requires version 0.2.1 or newer.

For Windows, use the installer documented in the CLI repository.

If you’re working with an AI coding assistant, install Agora Skills too:

npx skills add github:AgoraIO/skills

Skills give the assistant Agora-specific implementation guidance and references. They don’t replace account configuration or runtime testing.

The CLI also exposes its built-in workflow catalog:

agora skills list --json

Then sign in and find your project:

agora login
agora project list

Use your project ID or exact project name in the following steps.

Step 2: Clone the game and configure Agora

Clone the repository:

git clone https://github.com/AgoraIO-Community/agora-nemotron-transcribe.git
cd agora-nemotron-transcribe

Install dependencies and create the local configuration file:

pnpm install --frozen-lockfile
cp env.local.example .env.local

Bind the existing checkout to your Agora project:

agora quickstart env write . \
  --template nextjs \
  --scenario voice-agent \
  --project YOUR_PROJECT

Replace YOUR_PROJECT with your project ID or exact name.

The template-aware writer populates the names this Next.js application expects:

NEXT_PUBLIC_AGORA_APP_ID=your_app_id
NEXT_AGORA_APP_CERTIFICATE=your_app_certificate

The App ID is used by the browser. The App Certificate stays on the server.

Select the project and check Conversational AI readiness:

agora project use YOUR_PROJECT
agora project doctor --feature convoai --json

Resolve any reported project or feature requirements before continuing. A successful diagnostic checks configuration; an actual spoken conversation confirms the runtime connection.

This game was adapted from the Agora Next.js agent quickstart. It preserves the quickstart’s RTC/RTM lifecycle and token patterns.

Use quickstart env write for this existing checkout. Running agora init creates a separate starter rather than installing the game’s custom UI and scoring logic.

Step 3: Connect free NVIDIA inference

The default provider uses the public Nemotron 3.5 Transcribe Hugging Face Space.

It runs the exact model used in this tutorial:

nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B

Open .env.local and set:

NEMOTRON_PROVIDER=hf-space
HF_TOKEN=hf_your_token_here

You do not need NEMOTRON_BASE_URL for this provider.

To create your token:

1. Sign in to Hugging Face.
2. Open Settings → Access Tokens.
3. Create a Read token named zubaan-race.
4. Copy it into HF_TOKEN in .env.local.

Read access is sufficient for this inference workflow; write access is unnecessary. Keep the token server-side and out of Git. Hugging Face token documentation

Understand the free quota

At the time of writing:

• Anonymous users receive 2 minutes of GPU time daily.
• Free accounts receive 5 minutes of GPU time daily.
• This Space uses xlarge GPU allocation, which consumes quota at twice the standard rate.
• GPU time is different from recording duration.
• Players using the same backend token share its allowance.
• Anonymous backend requests share the server’s anonymous allowance.

Check the current ZeroGPU limits before relying on these numbers.

This is suitable for experimentation, but a ten-round run with retries may hit the quota.

The application never switches to a paid inference provider automatically. If the free GPU is unavailable or its quota is exhausted, the attempt remains unscored.

Step 4: Check the connection and launch

Check local prerequisites:

pnpm run doctor

Optionally, run one inference connectivity test:

pnpm run smoke:inference

This sends the bundled, attributed English sample to your selected provider. It uses GPU quota, so there’s no need to repeat it after a successful result.

A successful response includes a transcript beginning with:

The birch canoe slid on the smooth planks.

Now start the web app:

pnpm dev

Open the Local URL printed in your terminal.

Choose a language, accept audio processing, and start the game. In English, tell Chulbul:

Start.

In Hindi:

शुरू करो।

Choose Peek at the challenges if you want to inspect the phrases without starting the host or using inference.

Restart the server whenever you change .env.local.

Step 5: Configure the conversational host

The host is configured in app/api/invite-agent/route.ts.

Its provider chain uses the selected language:

.withStt(new DeepgramSTT({
  model: "nova-3",
  language,
}))
.withLlm(new OpenAI({
  model: "gpt-4o-mini",
  greetingMessage: greeting,
  failureMessage,
  maxHistory: 30,
  params: {
    max_tokens: 300,
    temperature: 0.6,
  },
}))
.withTts(new MiniMaxTTS({
  model: "speech_2_6_turbo",
  voiceId:
    language === "en"
      ? process.env.AGORA_HOST_VOICE_EN || "English_Aussie_Bloke"
      : process.env.AGORA_HOST_VOICE_HI ||
        process.env.AGORA_HOST_VOICE ||
        "hindi_female_1_v2",
}))

This is an excerpt from the existing server route. Its surrounding code validates requests, starts the agent session, and returns a signed capability used for cleanup.

The host’s personality is playful, but its responsibilities are narrow:

• Read the supplied challenge.
• Wait for the player.
• Explain results supplied by the application.
• Answer questions about those results.

The host does not invent a score or grade an attempt from its live recognition.

The application sends three main context messages:

• GAME_LEVEL supplies the current phrase and instructions to read it once.
• GAME_RESULT supplies the measured transcript, match percentage, score, and word differences.
• GAME_SUMMARY supplies the completed run’s results.

These are application-defined messages sent through the toolkit, not built-in Agora game events.

Step 6: Coordinate the host and the recording

The application waits until the host finishes introducing a challenge before recording the player’s attempt.

The browser watches the microphone’s audio level to identify speech. About 1.15 seconds of silence after speech ends the attempt automatically, with a maximum attempt window of 27 seconds.

The implementation lives in:

• lib/attempt-capture.ts
• lib/capture.ts
• public/pcm-recorder.js

The recording becomes mono, 16 kHz PCM WAV audio.

Speaking duration is measured locally from detected speech onset to the last detected speech. Pauses within the utterance count; upload and GPU processing time do not.

A player should not lose speed points because a GPU request waited in a queue.

The scored recording comes from the player’s local microphone track rather than the combined conversation. Headphones still matter because a microphone can physically pick up the host’s voice from speakers.

Step 7: Ask NVIDIA to preserve the attempt

The transcription instruction in lib/game.ts is deliberately specific:

export const transcriptionInstruction = (language: GameLanguage) =>
  `Transcribe the ${
    language === "en"
      ? "English speech in English"
      : "Hindi speech in Devanagari"
  } exactly as spoken. Preserve repetitions, false starts, omissions and incorrect words. Do not correct tongue twisters or infer missing words. Output only the transcript.`;

The expected tongue twister is never included in the NVIDIA request.

Supplying the answer as a hint could encourage the recognizer to reconstruct the intended phrase and hide the differences the game is trying to show.

The browser sends the recording to:

POST /api/evaluate?level=0&language=en&seconds=4.200
Content-Type: audio/wav

The route validates:

• The selected language.
• The level number.
• The WAV format and upload size.
• The recording length.
• The supplied speaking duration against the clip’s duration.

It then calls the selected transcription provider:

const result = await transcribe(
  bytes,
  duration,
  "verbatim",
  "",
  true,
  request.signal,
  fetch,
  transcriptionInstruction(language),
);

For the free Space, lib/hf-space.ts:

1. Uploads the recording through the Gradio upload API.
2. Queues a /transcribe request with the neutral instruction.
3. Reads the server-sent event stream.
4. Waits for the final completed response.
5. Removes processing statistics from the returned text.
6. Rejects incomplete, empty, or output-limited responses.

Partial text is never scored.

The free Space returns transcript text without word timestamps. The game uses its locally measured speaking duration and transcript comparison. The self-hosted provider has a separate optional alignment path.

Step 8: Calculate the score

The scoring engine lives in lib/game.ts.

It normalizes Unicode, ignores punctuation and invisible joiners, and compares words using Levenshtein alignment. Hindi vowel marks, repeated words, and word order remain significant.

The comparison identifies:

• Matching words.
• Substitutions.
• Missing words.
• Extra words.

For example:

Expected: कच्चा पापड़ पक्का पापड़
Heard:    कच्चा पापड़ पापड़

One of four expected words is missing:

Word match = 75%
Accuracy points = 75 × 8 = 600
Speed bonus = 0

The formulas are:

word match =
  max(0, round(100 × (1 − word edit distance / expected word count)))
accuracy points =
  word match × 8

A speed bonus unlocks only at 90% word match or higher:

speed bonus =
  round(200 × min(1, target seconds / speaking seconds))

Each round offers up to 1,000 points, for a maximum run score of 10,000. Retrying keeps the highest-scoring attempt for that level.

The server calculates these values. It does not accept a score generated by the model.

Word match is not a pronunciation assessment. Speech recognition can mishear a correct attempt or infer an intended word. The visible transcript and recorded replay let players inspect the result themselves.

Step 9: Turn the score into a conversation

After scoring, the application sends the host the actual result:

{
  level: currentLevel + 1,
  expected: TWISTERS[currentLevel].text,
  heard: result.transcript,
  wordMatch: result.match,
  points: result.score,
  seconds: result.speakingSeconds,
  differences: result.differences.filter(
    word => word.kind !== "match"
  )
}

The expected phrase can be shared with the host at this stage because transcription has already completed.

The host can explain that a word was missing or that the transcript contained a different word. Its instructions prohibit diagnosing tongue placement, accent quality, or pronunciation disorders.

At the end, GAME_SUMMARY supplies the best recorded result for each completed round.

The player can then ask:

Which words should I practice?

The host has concrete results to refer to rather than a score it guessed.

Step 10: Support Hindi and English

The language switch changes:

• The challenge catalog.
• Interface copy.
• Host greeting and instructions.
• Live recognition language.
• Host voice.
• NVIDIA transcription instruction.
• Expected words used for scoring.
• Result labels and score-card headline.

English mode has its own ten phrases, including Peter Piper and the woodchuck challenge. It is not a translation of the Hindi phrases.

The switch is disabled during an active session. Exit the game before changing languages so recordings and results from different catalogs cannot mix.

Test the implementation

Run the repository checks:

pnpm run verify

The current suite contains 26 tests covering:

• Hindi text normalization and English challenge targets.
• Word alignment and deterministic scoring.
• Speed bonuses and best-attempt selection.
• Request validation and token behavior.
• Recording finalization.
• Chunked transcription responses.
• Quota errors and cancellation.
• Rejection of incomplete output.
• Isolation of expected phrases from inference prompts.

The integration was also checked against real NVIDIA inference through the free Space:

• A bundled 15-second speech sample completed through the adapter in 12.4 seconds.
• A four-second sample completed through /api/evaluate in approximately 9.7 seconds.
• The free-provider readiness endpoint returned ready.

These are individual connectivity measurements, not latency guarantees or Hindi accuracy benchmarks.

Before sharing your own deployment, test the complete microphone experience: host speech, voice commands, all ten rounds, retries, replay, final spoken review, and session cleanup.

Troubleshooting

“NVIDIA server is not reachable”

Set the free provider explicitly:

NEMOTRON_PROVIDER=hf-space

Restart the application. The vllm provider expects your own GPU server.

“Free GPU quota is exhausted”

Check that HF_TOKEN is configured and restart the server after adding it.

A token uses your account’s allowance; it does not remove daily limits. Wait for the quota to reset if the account allowance is also exhausted.

Avoid repeating connectivity tests unnecessarily.

The host cannot join

Check:

• Agora project readiness.
• RTM availability.
• App ID and App Certificate.
• Access to the configured managed providers.

No speech is detected

Check:

• Browser microphone permission.
• The selected input device.
• Microphone distance and input level.

The host misses “next” or “repeat”

Use the visible fallback control and try again in a quieter environment.

The transcript looks unexpectedly correct

Listen to the replay. ASR can reconstruct intended speech even without an answer hint.

GPU processing takes a long time

The free provider may be queued. Processing time does not affect the speed score.

The readiness endpoint checks whether the provider’s API is reachable. It cannot reserve GPU time or guarantee sufficient remaining quota.

Taking it beyond a local demo

For limited experimentation, the free Space keeps setup small: a laptop, an Agora project, and a Hugging Face token.

For a public deployment:

• Add application access controls.
• Limit request frequency and concurrent inference.
• Account for all visitors sharing a backend token’s quota.
• Review recording handling with your inference provider.
• Configure host and inference usage limits.

The repository also includes an optional self-hosted NeMo/vLLM route:

NEMOTRON_PROVIDER=vllm
NEMOTRON_BASE_URL=http://127.0.0.1:8000/v1
NEMOTRON_MODEL=nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B

Its setup scripts target NVIDIA’s documented BF16 H100 80 GB configuration. Check the current model card for supported hardware and serving instructions.

Granary v2 and NVIDIA’s adaptation recipes offer another direction if you later want to investigate domain-specific transcription. They are not required to run this game. NVIDIA’s model and dataset introduction

For now, start with the first phrase.

कच्चा पापड़, पक्का पापड़।

Your microphone is ready. Is your tongue?

Build a Talking Tongue Twister Game with Agora and NVIDIA Nemotron was originally published in Agora.io on Medium, where people are continuing the conversation by highlighting and responding to this story.

A useful voice interface needs to do several things at once: capture speech, deliver the agent’s response, show what was said, and keep the user informed while the conversation connects or ends. Those details shape the experience as much as the model behind it.

The Agora Conversational AI Flutter Quickstart brings those pieces together in an app you can run and explore. It pairs a Flutter client with a small Node.js backend and Agora’s Conversational AI Engine.

In this tutorial, we’ll start a conversation in Chrome, trace it through the code, and customize the agent for a different use case. Then we’ll explore the web, Android, iOS, and macOS targets. The repository shares its Flutter UI and session controller across these platforms, with platform-specific services underneath.

Watch the conversation demo.

What you’ll build

The sample starts a voice conversation with Ada, an assistant configured to help developers with Agora. You can speak into your microphone, hear the agent’s replies, read the transcript, mute yourself, and end the session.

The interface also exposes the channel, user IDs, agent ID, and connection state. Those details make it easier to understand what is happening while you develop your own experience.

You can adapt this foundation into a language practice partner, an onboarding assistant, a product guide, or another voice experience with your own prompt and interface.

One Flutter interface, several places to talk. The platform section below explains what each target currently supports.

How the pieces fit together

The app has three main parts:

• Flutter client: Renders the interface, requests microphone access, joins audio, and displays conversation updates.
• Node.js backend: Generates tokens and starts or stops agent sessions using server-side credentials.
• Agora services: Carry RTC audio, run the configured conversational AI pipeline, and deliver RTM events.

RTC carries the live audio. RTM carries messages such as transcript and agent-state updates. The backend keeps the Agora App Certificate out of the Flutter application.

That separation gives you clear places to work: change the interface in Flutter, adjust the agent configuration on the server, and keep credentials on the server throughout.

1. Get the quickstart

You’ll need Flutter, its bundled Dart SDK, Node.js 22 or newer, npm, Chrome, and an Agora project with Conversational AI and RTM enabled. You’ll also need the project’s App ID and App Certificate from the Agora Console.

This walkthrough uses the dependency-upgrade snapshot at commit d859829, which requires Flutter 3.47.7 and Dart 3.13.5 or newer. Checking out that revision keeps the code aligned with the examples here:

git clone https://github.com/AgoraIO-Conversational-AI/agent-quickstart-flutter.git
cd agent-quickstart-flutter
git checkout d859829
flutter --version
node --version

To make your own changes on this snapshot, create a branch:

git switch -c my-voice-agent

2. Configure the backend

Create a file named .env.local in the repository root, beside pubspec.yaml:

NEXT_PUBLIC_AGORA_APP_ID=your_agora_app_id
NEXT_AGORA_APP_CERTIFICATE=your_agora_app_certificate
NEXT_PUBLIC_AGENT_UID=123456
NEXT_AGENT_GREETING="Hi! I'm Ada. What would you like to build today?"

The backend reads this file and exposes the public client configuration through /api/client-config. The NEXT_ prefixes are the environment-variable names used by this sample; the companion server is a Node.js application.

The App ID identifies your project. The App Certificate is a secret used by the backend for authentication and token generation. Keep the certificate in this server-side file and out of Dart code, browser JavaScript, and --dart-define values. The repository ignores .env.local in Git.

The sample configures its speech recognition, language model, and speech synthesis through Agora-managed inference, so its base configuration does not require separate provider API keys. Your Agora project must have access to the configured services.

Start the backend from the repository root:

cd backend
npm ci
npm run dev

It listens on port 3001 by default. In another terminal, check that it is running:

curl http://localhost:3001/health

The response should be:

{"ok":true}

That confirms the server is reachable. The first conversation will exercise the Agora credentials and agent configuration.

3. Start the Flutter app

In a second terminal, navigate to the repository root and run:

flutter pub get
flutter run -d chrome

Click Start Conversation and allow microphone access when Chrome asks. Once the agent joins, speak a short question such as “What can I build with Agora?”

Look for the agent’s audio, transcript updates, and connection information. Try Mute mic, unmute, and then choose End Conversation.

You now have a concrete flow to modify: start a session, exchange speech, observe its state, and clean it up.

4. Follow a conversation through the code

The main session logic lives in ConversationSessionController. It extends ChangeNotifier, allowing the interface to rebuild as session state changes.

When you start a conversation, the controller requests microphone access, obtains a token, and asks the backend to invite the agent. This excerpt shows the token-to-agent handoff:

final tokenData = await _backendApi.generateToken();
final agentResponse = await _backendApi.inviteAgent(
  ClientStartRequest(
    requesterId: tokenData.uid,
    channelName: tokenData.channel,
  ),
);

The backend routes provide a small API surface:

• GET /api/client-config: Returns the public App ID, agent UID, and greeting.
• GET /api/generate-agora-token: Returns a token, client UID, and channel name.
• POST /api/invite-agent: Starts the agent for the requested channel and returns its agent ID.
• POST /api/stop-conversation: Stops the agent identified by that agent ID.

The controller then initializes RTM with the returned client identity and subscribes to the conversation channel. It initializes RTC and joins that same channel for audio.

The RTC service joins with these media options:

options: const ChannelMediaOptions(
  clientRoleType: ClientRoleType.clientRoleBroadcaster,
  publishMicrophoneTrack: true,
  autoSubscribeAudio: true,
  autoSubscribeVideo: false,
),

This publishes microphone audio and subscribes to incoming audio. The sample does not subscribe to video.

Keep the identities consistent when extending the app: the token’s client UID is also the RTM login identity, and the agent, RTC client, and RTM subscription use the same conversation channel.

The client joining RTC and the agent joining RTC are separate events. A successful client connection is useful progress; the remote-user join event tells you that the agent’s audio participant has arrived.

Here is the successful session path, from the first click to cleanup:

The channel ties the audio and events together. The agent ID identifies the server-side session to stop; the client UID identifies the person joining it.

5. Understand the agent configuration

The backend configures the agent in backend/src/agent.js.

At this snapshot, the pipeline uses Deepgram nova-3 for speech recognition, OpenAI gpt-4o-mini for language responses, and MiniMax speech_2_6_turbo for speech synthesis. These are the sample’s configured choices; they are not requirements for every app you build.

A voice turn takes a trip through the pipeline. RTM separately brings transcript and state updates back to the interface.

The agent configuration contains its instructions, greeting, turn-detection settings, and RTM options. The following excerpt enables the RTM data channel and requests metrics and error messages:

advancedFeatures: { enable_rtm: true, enable_tools: true },
parameters: {
  data_channel: 'rtm',
  enable_error_message: true,
  enable_metrics: true,
},

The server creates a session for the client’s channel and starts it:

const session = agent.createSession({
  channel: channel_name,
  agentUid: process.env.NEXT_PUBLIC_AGENT_UID ?? String(DEFAULT_AGENT_UID),
  remoteUids: [requester_id],
  idleTimeout: 30,
  expiresIn: ExpiresIn.hours(1),
  debug: false,
});
const agentId = await session.start();

Flutter stores the returned agent ID so it can ask the backend to stop that specific session later.

6. Make the agent your own

Start with a focused change to the agent’s purpose. For example, replace the ADA_PROMPT text in backend/src/agent.js with instructions for a language practice partner:

const ADA_PROMPT = `You are a patient English conversation partner.
Help the learner practice everyday situations.
Keep each response to one or two short sentences.
Ask one question at a time.
When correcting a mistake, offer one natural alternative and continue the conversation.`;

Update the greeting in .env.local to match:

NEXT_AGENT_GREETING="Let's practice ordering at a café. What would you like to order?"

Restart the backend and start a fresh conversation. The existing RTC, RTM, and session flow can support the new persona without changes to the transport code.

Next, adapt the Flutter interface in lib/main.dart. You might add a scenario selector, a lesson objective, or a completion screen. Transcript formatting lives under lib/utils/, while the session controller owns lifecycle state.

If you add scenarios that affect the server prompt, send an allowed scenario identifier to the backend and map it to server-owned instructions. That gives your app a controlled configuration surface as it grows.

The scenario selector is an extension you can add; the current sample starts with Ada’s configured persona. A possible extension would look like this:

7. Take the same app to web, mobile, and desktop

Flutter lets this sample share its interface and session logic across targets. The RTC and RTM implementations still depend on the platform, so the current quickstart has the following scope:

• Web / Chrome: Run flutter run -d chrome for browser RTC audio and RTM transcripts/state through the JavaScript bridge.
• Android: Run flutter run -d <android-device-id> for native RTC audio and RTM transcripts/state.
• iOS: Run flutter run -d <ios-device-id> for native RTC audio and RTM transcripts/state. Requires Xcode, CocoaPods, and device setup.
• macOS: Run flutter run -d macos for RTC audio and microphone capture. RTM is stubbed, so live transcripts and RTM agent-state updates are unavailable.

This repository includes host projects for these targets. Windows and Linux would require additional host setup, compatible SDKs, and validation; this article does not treat them as supported quickstart targets.

Web: share a browser experience

Keep the Node.js companion running and launch Chrome:

flutter run -d chrome

The browser target defaults to http://localhost:3001. To generate the web assets for hosting, run:

flutter build web

The output goes into build/web. A hosted version also needs a reachable backend, HTTPS for microphone access, and the public backend URL supplied when building:

flutter build web \
  --dart-define=BACKEND_BASE_URL=https://your-backend.example.com

The backend address is embedded in that build. Supply the real address before deploying the web assets, and add the backend access controls described at the end of this tutorial.

Android: emulator or physical device

For an Android emulator, keep the backend running and use:

flutter devices
flutter run -d <android-device-id>

The app defaults to http://10.0.2.2:3001 on Android, which lets an emulator reach the host computer. The updated project builds against Android SDK 37 and requires API 24 or newer at runtime. For a physical Android phone, use the LAN override below.

iOS: simulator or physical device

On a Mac with the required Xcode components installed, select an iOS simulator or connected device from flutter devices:

flutter run -d <ios-device-id>

The iOS default is http://127.0.0.1:3001, suitable for a simulator reaching the backend on the same Mac. The iOS project requires iOS 15 or newer, Xcode with the appropriate platform components, and CocoaPods for the RTM plugin. Physical-device builds also need signing configured and a reachable backend URL.

Physical phones: use a reachable backend

For a physical Android or iOS device, point Flutter at a backend address the device can reach. For example, with the phone and computer on the same network:

flutter run -d <device-id> \
  --dart-define=BACKEND_BASE_URL=http://192.168.1.10:3001

Replace the example address with your computer’s LAN IP. Allow local-network access if prompted and make sure the computer’s firewall allows the backend connection. Use HTTPS for a deployed backend.

macOS: explore a desktop voice interface

With the backend running on the same Mac:

flutter run -d macos

The app defaults to http://127.0.0.1:3001. You can explore RTC audio and microphone capture in a native desktop window. The current macOS RTM service is a stub, so transcript and RTM agent-state events do not reach the UI. Adding a compatible macOS event transport is one way to extend this quickstart.

Choose a target for the experience you want to explore: Chrome offers the tutorial’s initial browser path, Android and iOS take the conversation onto phones, and macOS provides a starting point for a desktop voice interface.

Where to look when something goes wrong

If the app cannot load configuration, check the backend’s /health endpoint and the client’s backend URL. On a physical phone, localhost points to the phone itself.

If a conversation fails to start, check the backend terminal, the App ID and certificate, and whether your Agora project has access to Conversational AI and RTM. If you already use the Agora CLI, agora project doctor --deep provides additional project diagnostics.

If audio works but transcripts do not appear, inspect the RTM connection and channel identity. On web, web/index.html loads the browser SDKs, and web/rtm_bridge.js connects the RTM JavaScript SDK to Flutter.

When you change SDK versions, keep the Flutter RTC package and its documented Iris browser script compatible. The snapshot above pins a matching combination.

Build on top of the quickstart

Once your first conversation works, choose one useful behavior to add: a different agent persona, a guided practice session, a product-specific interface, or a conversation history view.

The session controller already provides places to work with microphone controls, token renewal, transcript updates, and ending a conversation. When End Conversation succeeds, it stops the backend agent, leaves RTM and RTC, disposes the services, and resets the client state.

For a deployed product, extend the companion backend with user authentication and session ownership checks before exposing its agent-control routes publicly. Its current routes are designed for a local quickstart. Add recovery for partial startup and cleanup failures as you develop the complete session experience.

Clone the Flutter quickstart, run the sample conversation, and give Ada a new purpose. A working voice session is a useful starting point for discovering what your own app should do next.

Building a Conversational AI App in Flutter with Agora was originally published in Agora.io on Medium, where people are continuing the conversation by highlighting and responding to this story.

Agora Skill now supports setting up a one-to-one voice and video calling demo for the web, extending its guided setup workflow from Voice Agent to RTC. Once Agora Skill and the Agora CLI are installed, you can ask your AI coding assistant to set up and run the demo.

The Skill gives your assistant the instructions to follow, while the Agora CLI handles project setup and credentials. Together, they help your assistant get the demo running with less manual work from you.

How it works

The CLI now includes a one-to-one calling scenario. Following the Skill’s instructions, your assistant uses the CLI to guide you through signing in and choosing an Agora project, or creating one if needed. It then retrieves the App ID and project credentials, prepares the demo, and runs it.

Once the demo is running, turn on your camera for a video call, or leave it off for an audio-only conversation. You have a Web calling demo and its source code to explore and extend. The interface follows the same design as the Voice Agent Quickstart, giving both scenarios a consistent look and feel.

A faster path from prompt to prototype

Less manual setup. The CLI retrieves the credentials the demo needs, so you don’t have to copy values from the console and pass them back to your assistant. Your main setup interactions are signing in once and choosing a project.

A more predictable workflow. Your assistant starts with a complete calling example and a defined setup sequence. That reduces the implementation decisions it needs to make before you can try the demo, while the Skill and CLI connect project preparation with running the sample.

Up and running sooner. In early trials, our engineering team reported completing setup in under 10 minutes. Actual time will vary with your development environment, network, and coding assistant.

Get started

Run the demo locally and try a one-to-one video call between two browser windows.

Prerequisites

• An AI coding tool that supports Skills, such as Codex or Claude Code
• Node.js 22 and npm or pnpm installed locally

1. Install Agora Skill and the CLI

In your terminal, install Agora Skill for your coding assistant:

npx skills add github:AgoraIO/skills

Then install the Agora CLI. On macOS or Linux:

curl -fsSL https://dl.agora.io/cli/install.sh | sh

On Windows, use PowerShell 7+:

irm https://dl.agora.io/cli/install.ps1 | iex

Make sure Agora Skill is loaded in your coding tool. Tool-specific setup instructions are in the Agora Skill repository listed below.

Agora Skill installed in Codex.

2. Run the demo with a prompt

With Agora Skill loaded, send this request to your coding assistant:

I'd like to create an Agora video call demo, could you help me?

Ask your coding assistant to create an Agora video call demo.

Follow the sign-in prompts, then select an Agora project or create one. If the project has no primary App Certificate, enable one in Agora Console or choose another project.

When your assistant returns the local URL, open it in your browser. You should see the Quickstart home page with a Create Channel action.

The assistant returns the local URL for the running RTC demo.

3. Make your first call

Select Create Channel, enter your display name, and copy the invite link. Select Join Call and allow camera and microphone access when prompted.

Enter your name, copy the invite link, and join the call.

Open the invite link in a second browser window on the same computer. Enter a different display name and join the channel. Keep both microphones and cameras on, and check that each window shows the other participant’s video and receives their audio. Use headphones to avoid feedback.

You now have a local copy of the Quickstart to explore and extend. See the RTC Quickstart repository below for configuration and deployment instructions.

We plan to add more scenarios and extend support to additional platforms and programming languages.

We’re also refreshing the Voice Agent and RTC experiences in the Agora Console. You can try both scenarios more easily and copy sample code to continue building.

GitHub resources

• Agora Skill — installation and tool setup
• Agora CLI — installation and troubleshooting
• RTC Quickstart — source code, npm setup, and deployment

Agora Skill Now Supports RTC: From Prompt to Video Call was originally published in Agora.io on Medium, where people are continuing the conversation by highlighting and responding to this story.

Splitting turn-taking in an Agora Conversational AI agent between a fast acoustic gate and TypeSafe’s Jev, starting from a quickstart recipe.

Voice agents are very good at noticing silence. They’re much worse at knowing what the silence means.

Say “So what I want to know is…” and pause while you look for the next word. In a silence-driven turn detector, that pause can be enough to close the turn and send an unfinished thought to the agent. In a basic VAD-driven setup the language model may never see the problem: the turn was already closed because the audio pipeline interpreted the pause as end of speech.

Silence doesn’t tell you whether someone has finished their turn. A speaker may be searching for a word. An “uh-huh” during the agent’s response may only be a backchannel, and speech picked up by the microphone might not be directed at the agent at all. Audio activity can tell you that something happened; it gives you very little information about the speaker’s intent.

What I built

Instead of asking one detector to solve the entire problem, I split turn-taking into two jobs. Audio still detects that someone has started speaking. Once text begins arriving from speech recognition, Jev decides whether the speaker has actually yielded the floor.

The project is called Jev VAD, although Jev itself isn’t doing voice activity detection. The VAD-like part is a lightweight acoustic gate in the browser; Jev handles the semantic decisions that conventional VAD can’t make. It runs on Agora’s Conversational AI Engine, and it’s open source.

Jev is TypeSafe’s System One model. Instead of generating text, it takes application state and focused questions and returns typed probabilistic decisions. That makes it useful here because I don’t need another model to write anything. I need a few numbers the application can branch on.

Architecture at a glance

mic
 ↓
acoustic gate (browser)
 ↓
manualSOS
 ↓
Agora ASR → streaming transcript → /judgeTurn → Jev
                                                 ↓
                         turn_complete · holding_floor ·
                         addressed_to_agent · taking_floor
                                                 ↓
                         browser: manualEOS, interrupt(), or keep listening

The short version of the design is open fast, decide late.

Speech onset and end-of-turn detection have very different latency requirements. For onset, I care more about speed than intelligence, because waiting too long risks clipping the first word. The browser watches microphone energy against a moving noise floor and opens the turn after roughly 110 ms of sustained activity. It also drops obvious impulsive bursts, like a single click or thump, before they open anything.

Some non-speech sounds can still get through the gate and open a turn. That is intentionally cheap: if speech recognition produces no text, the controller never closes the turn or sends anything to the agent.

End of turn is where Jev comes in. As the streaming transcript evolves, the browser sends it to Jev, which returns four scores in one call:

• turn_complete: has the speaker actually handed over the floor? "So what I want to know is…" hasn't.
• holding_floor: are they explicitly asking for more time, as in "hold on, let me type something"?
• addressed_to_agent: is the latest speech intended for the assistant? "Hey, can you hold my coffee?" usually isn't.
• taking_floor: while the agent is talking, is this a real interruption or just a backchannel like "right"?

In my test sessions the Jev call usually came back in roughly 300 ms. That is too slow to sit in front of speech onset, but perfectly usable once speech recognition is already streaming words, which is why the two jobs are split the way they are.

The client still measures silence, but it no longer treats a fixed silence duration as the decision. Instead, silence gradually relaxes the score Jev needs to reach before the client closes the turn.

Start from the recipe with the Agora CLI

Jev VAD is available in the Agora recipes catalog. Once the CLI is authenticated, agora init creates a local copy and binds it to one of your Agora projects.

1. Install the CLI and log in.

curl -fsSL https://dl.agora.io/cli/install.sh | sh
agora login

2. Find the recipe.

agora recipes list --type ai
agora recipes show jev-vad

3. Create your copy.

agora init my-jev-vad --recipe jev-vad

It can create or reuse an Agora project, clone the recipe, and write the project credentials into the local environment file (server/.env). Then it prints the next steps.

4. Add your TypeSafe key and start it.

cd my-jev-vad
echo 'TYPESAFE_API_KEY=<your-key>' >> server/.env
bun run setup
bun run dev

Open http://localhost:3000, click Start Conversation, and start talking. Jev's scores move in the side panel as you speak.

Already have a ConvoAI app?

The recipe also has a smaller drop-in path for an existing Conversational AI app. It keeps the two decisions that matter most for turn-taking, completion and interruption, and leaves out the demo’s side-talk handling, patient mode, telemetry and tuning UI.

There’s also a copyable coding-agent prompt at the top of the recipe guide, so the same instructions can be applied to an existing project instead of starting from the demo. The rest of this post describes the full demo.

What changed from the Agora quickstart

I started from the Agora Python quickstart rather than building the agent stack from scratch. It already provides the Next.js client, the FastAPI backend, the transcription flow and the agent lifecycle. Jev VAD changes the parts responsible for deciding when a user turn opens, closes, or interrupts the agent.

1. Switch Agora’s turn detection to manual (agent config)

In the backend’s agent configuration, start of speech and end of speech are set to manual, and automatic interruption is disabled:

turn_detection={
    "mode": "default",
    "config": {
        "start_of_speech": {"mode": "manual"},
        "end_of_speech":   {"mode": "manual"},
    },
},
interruption={"enable": False, "disabled_config": {"strategy": "append"}},

Agora still handles the realtime session, transcription, agent execution and speech output. What it no longer owns in this demo is turn control: the browser explicitly sends manualSOS, manualEOS and interrupt() through the Agora client toolkit.

With interruption disabled, strategy: "append" keeps speech captured while the agent is talking instead of throwing it away before the client has decided what it means.

2. Add one judgment route (API routes)

/judgeTurn receives the current transcript along with enough conversational state to interpret it: recent turns, the agent's last message and current state, mic silence, and some information about how the partial transcript has changed. Jev evaluates all four questions against that state in one call. The browser reaches it through the same /api/* path as the rest of the app, so the TypeSafe key stays on the server.

Normal transcript changes are debounced by about 150 ms before calling /judgeTurn. During a potential interruption the controller evaluates partials immediately, which is why a real "wait, wait" can stop the agent quickly.

The Jev questions define what the model judges; thresholds and timing determine how aggressively the client acts on those judgments.

3. The browser state machine (conversation UI)

The browser owns the state machine. Jev returns probabilities; it never calls Agora directly. Stripped down, the branches look like this:

complete + addressed + quiet   → manualEOS
side-talk or backchannel       → keep listening
taking_floor                   → interrupt()

The completion threshold starts at 0.72 and relaxes toward 0.55 as mic silence approaches three seconds. A separate quiet-confirmation timer prevents a high semantic score from ending the turn while the user is still audibly speaking, and five seconds remains the hard fallback. When Jev says the user is holding the floor, the threshold doesn’t relax and the fallback stretches to twelve seconds.

In practice a user turn produces several judgments rather than one, usually three to six. The partial transcript changes as the user speaks, and the client may ask again during silence before closing the turn.

I also added a panel that shows the four scores live, and a Tune drawer for changing thresholds and timers mid-call without restarting anything. Most of my tuning happened there, watching the numbers move while I talked.

What happened in actual calls

These are scores from my test sessions, not fixed outputs you should expect from Jev.

The test I cared about most was a long pause in the middle of an unfinished sentence. I said “So what I want to know is…”, stayed silent for three seconds, and finished with “…did you jump in?” During the pause turn_complete sat around 0.05, so the turn stayed open even as the threshold relaxed. The agent answered about a second after the question.

I also wanted to see what happened when I explicitly asked for time. “Hold on, let me type something” scored 0.8 to 0.9 on holding_floor, so the agent stayed quiet while I typed and then answered the question I asked afterwards.

For backchannels, I asked the agent to walk me through how a call works and reacted the way I normally would while it talked. “Right.” scored 0.03 on taking_floor and "Okay." scored 0.04; both were held and the agent kept going. "Wait, wait. Let me stop you there." scored 0.97, and the agent stopped within about a second.

Side-talk was the last case. I turned away and said “Hey, can you hold my coffee, please?”, which scored 0.09 on addressed_to_agent, so nothing was sent to the agent. "Sorry, that wasn't for you." scored 0.79, and the agent replied that it had noticed.

Wrapping up

The useful part of this experiment wasn’t replacing VAD with another model. I still use audio to open the turn. What changed was treating end of turn as a judgment about the conversation rather than a timer attached to silence.

If you want to experiment with the same approach, the full demo and the smaller drop-in recipe are both available below.

• The recipe: recipes.agora.io/recipes/jev-vad, or agora init my-jev-vad --recipe jev-vad
• The demo source: convoai-jev-vad
• The Agora CLI: AgoraIO/cli
• The base recipe: agent-quickstart-python
• Agora Conversational AI: docs.agora.io
• TypeSafe Jev: typesafe.ai

Jev VAD: Turn Detection That Understands What You Say was originally published in Agora.io on Medium, where people are continuing the conversation by highlighting and responding to this story.