Process Management in Hermes Agent governs the spawning, tracking, and lifecycle management of background processes. This is crucial for managing long-running tasks such as servers, watchers, or continuous builds without blocking the agent's core conversation or tool execution loop.
The heart of this system is the ProcessRegistry, an in-memory, thread-safe registry of background processes that tracks their status, buffers output logs, supports blocking waits with interrupt support, process killing, and crash recovery through checkpointing [tools/process_registry.py:2-10].
Background processes execute entirely within the selected environment backend (local, Docker, Modal, SSH, Singularity, Daytona) ensuring process isolation and sandboxing where applicable [tools/process_registry.py:12-14].
The terminal tool [tools/terminal_tool.py:1-32] (and its underlying environment implementations) supports two modes of command execution, controlled by the background parameter:
| Mode | Behavior | Use Cases | Return Value |
|---|---|---|---|
| Foreground | Blocks until the command finishes or times out | Build scripts, installs, CLI cmds | Command output + exit code |
| Background | Spawns process and returns immediately with a unique session ID | Servers, watchers, daemons | Session ID, for status checks |
Background processes are tracked via ProcessRegistry, while foreground commands synchronously return results to the caller. Foreground commands are subject to a hard timeout cap (default 600s) [tools/terminal_tool.py:107-112].
The ProcessRegistry [tools/process_registry.py:142-150] maintains two dictionaries keyed by unique session IDs:
_running: currently active background processes [tools/process_registry.py:159]._finished: recently completed processes retained for a time-to-live period [tools/process_registry.py:160].It also features a pending_watchers list and a completion_queue to signal process completions and watch pattern matches to the gateway [tools/process_registry.py:164-167].
The diagram below associates the natural language concepts of background process management with the specific classes and methods found in the codebase.
Title: Process Management Code Association
Sources: [tools/process_registry.py:91-140], [tools/process_registry.py:142-168], [tools/process_registry.py:176-228], [tools/process_registry.py:418-440]
Each background process is encapsulated as a ProcessSession dataclass storing lifecycle state [tools/process_registry.py:91-140]:
| Field | Type | Description |
|---|---|---|
id | str | Unique session identifier, e.g., proc_xxxxxxxxxxxx [tools/process_registry.py:93] |
command | str | The original shell command line [tools/process_registry.py:94] |
task_id | str | Sandbox/terminal environment isolation key [tools/process_registry.py:95] |
session_key | str | Gateway session key for reset isolation [tools/process_registry.py:96] |
pid | Optional[int] | OS process ID if known (local) [tools/process_registry.py:97] |
process | Optional[Popen] | Local subprocess handle (only local env) [tools/process_registry.py:98] |
env_ref | Any | Reference to the environment object for remote processes [tools/process_registry.py:99] |
cwd | Optional[str] | Working directory of the process [tools/process_registry.py:100] |
started_at | float | Spawn timestamp (epoch) [tools/process_registry.py:101] |
exited | bool | Whether process has exited [tools/process_registry.py:103] |
exit_code | Optional[int] | Process exit code if exited [tools/process_registry.py:104] |
output_buffer | str | Rolling buffer of last ~200KB output [tools/process_registry.py:107] |
max_output_chars | int | Maximum size of output buffer (default 200,000 chars) [tools/process_registry.py:108] |
detached | bool | True if recovered from crash (no pipe) [tools/process_registry.py:109] |
pid_scope | str | "host" for local/PTY PIDs, "sandbox" for env-local PIDs [tools/process_registry.py:110] |
watch_patterns | List[str] | Regex/string patterns to watch for in output [tools/process_registry.py:121] |
notify_on_complete | bool | Whether to send a notification on process completion [tools/process_registry.py:119] |
When running locally, background processes are spawned as subprocesses or with pseudo-terminals (PTY) for interactive behaviors.
Lifecycle:
_find_shell() to locate the shell binary [tools/environments/local.py:1120-1165] (referenced from [tools/process_registry.py:44])._sanitize_subprocess_env [tools/environments/local.py:1168-1240].subprocess.Popen or ptyprocess.PtyProcess if use_pty=True [tools/process_registry.py:176-228].For non-local backends, background commands execute inside isolated sandboxes.
execute_code can spawn child processes that run Python scripts. These scripts communicate back to the parent via Unix Domain Sockets (local) or file-based RPC (remote) [tools/code_execution_tool.py:8-22].bash -c process. A session snapshot (env vars, functions, aliases) is captured and re-sourced before each command [tools/environments/base.py:3-7].Title: Background Process Execution Flow
Sources: [tools/process_registry.py:176-228], [tools/process_registry.py:258-337], [tools/process_registry.py:339-416], [tools/environments/base.py:3-7]
Background processes support "watch patterns": regex/string patterns matched against new output. When a pattern matches, a notification event is enqueued. The system enforces strict rate-limiting [tools/process_registry.py:63-76]:
WATCH_MIN_INTERVAL_SECONDS) [tools/process_registry.py:69].If notify_on_complete is set, a notification is enqueued in completion_queue when the process exits [tools/process_registry.py:119]. These synthetic completion events are used by the gateway to trigger agent callbacks [tools/process_registry.py:166-167].
The process tool (handled via ProcessRegistry methods) provides a procedural interface:
| Action | Description |
|---|---|
poll | Returns status (running, exited), exit code, and output preview [tools/process_registry.py:418-440]. |
wait | Blocks until process exit or timeout. Supports is_interrupted polling [tools/process_registry.py:442-493]. |
read_log | Paginated retrieval of the 200KB rolling output buffer [tools/process_registry.py:495-517]. |
kill | Terminates process. Uses os.kill for local or environment-specific kill commands [tools/process_registry.py:519-553]. |
write | Writes data to process stdin [tools/process_registry.py:555-591]. On Windows, pywinpty expects str while POSIX expects bytes [tests/tools/test_process_registry.py:68-104]. |
MAX_OUTPUT_CHARS) [tools/process_registry.py:58].FINISHED_TTL_SECONDS) [tools/process_registry.py:59].The registry maintains a checkpoint file at {HERMES_HOME}/processes.json [tools/process_registry.py:55]. On startup, it attempts to recover sessions via _load_checkpoint(). Recovered sessions are marked as detached because their I/O pipes are lost [tools/process_registry.py:775-837].
To prevent credential leaks, Hermes scrubs environment variables before spawning child processes.
tirith binary before execution [tools/tirith_security.py:1-21]._SECRET_SUBSTRINGS (e.g., "KEY", "TOKEN", "SECRET") are filtered out [tools/code_execution_tool.py:148-158].OPENAI_API_KEY, ANTHROPIC_TOKEN) are explicitly stripped from local subprocesses [tools/environments/local.py:1168-1240].Sources: [tools/process_registry.py:54-76], [tools/process_registry.py:418-591], [tools/process_registry.py:775-837], [tools/code_execution_tool.py:8-22], [tools/code_execution_tool.py:148-190], [tools/tirith_security.py:1-21], [tools/environments/local.py:1168-1240], [tools/environments/base.py:3-7]
Refresh this wiki
This wiki was recently refreshed. Please wait 2 days to refresh again.