This document describes the logging infrastructure and panic recovery mechanisms used throughout the firestack codebase. The system provides multi-level logging with spam filtering, external console integration (including zero-copy shared memory support on Linux), stack trace collection with ring buffer support, and panic recovery with flight recording capabilities. For runtime configuration of logging levels and other system settings, see Runtime Configuration and Settings.
The logging system is built around the Logger interface and its implementation simpleLogger, which provides structured logging with multiple output destinations and intelligent rate limiting.
The Logger interface defines the contract for all logging operations:
Sources: intra/log/logger.go52-73 intra/log/logger.go75-102 intra/log/logger.go173
The system supports nine log levels with increasing severity:
| Level | Symbol | Description | Default Threshold |
|---|---|---|---|
VVERBOSE | Y | Most verbose tracing | Not shown by default |
VERBOSE | V | Verbose debugging | Not shown by default |
DEBUG | D | Debug information | Not shown by default |
INFO | I | Informational messages | Default internal level (INFO) |
WARN | W | Warning messages | Always shown |
ERROR | E | Error messages | Always shown |
STACKTRACE | F | Fatal with stack trace | Default console level |
USR | U | User-facing messages | Always shown |
NONE | | Disables logging | N/A |
Sources: intra/log/logger.go175-188 intra/log/logger.go214-215 intra/log/log.go82-86
The package exports a global Glogger instance and convenience functions. Most functions use a standard callerat depth of 2 to correctly identify the caller in the log output.
Sources: intra/log/log.go44-51 intra/log/log.go148-270
The simpleLogger writes to multiple destinations and utilizes a slab-based memory pool for efficiency:
o *golog.Logger) - For normal logs via internal output methods intra/log/logger.go90e *golog.Logger) - For errors and stack traces intra/log/logger.go91x *xlog) - Direct output via NDK __android_log_write when built for Android intra/log/xlog_android.go77q *ring[*[]byte]) - Keeps recent logs as pooled byte slabs for post-mortem diagnostics intra/log/logger.go94LB512 and LB1024 slabs to avoid allocations during formatting intra/log/logpool.go22-33Sources: intra/log/logger.go89-94 intra/log/xlog_android.go37-50 intra/log/logpool.go22-33
The system supports external consoles for redirecting logs to Android logcat, pipes, or shared memory regions.
Sources: intra/log/log.go56-71 intra/log/fconsole.go13-18 intra/log/memconsole.go112-130
On Linux platforms, Memconsole provides a zero-copy, double-buffered logging mechanism using memfd_create.
mcbuf) are created using unix.MemfdCreate and mapped via unix.Mmap intra/log/memconsole.go156-174 While the writer (firestack) fills the active buffer, the reader (Console) can drain the other intra/log/memconsole.go97-102memSlotSize = 801 bytes) intra/log/memconsole.go51-55Memconsole can either wrap around (overwriting old slots) or wait briefly for a drain to complete, controlled by overwriteOnFull and waitOnFull intra/log/memconsole.go41-43memFlushInterval (2500ms) even if the buffer isn't full intra/log/memconsole.go33-34Sources: intra/log/memconsole.go25-55 intra/log/memconsole.go97-127 intra/log/memconsole.go148-181
The logger implements a sophisticated clock-based spam filter to prevent log flooding.
The spam filter uses a two-level clock mechanism to detect when a specific code location (identified by pc) is logging too frequently:
Sources: intra/log/logger.go139-147 intra/log/logger.go149
The system tracks dropped messages via the skips array intra/log/logger.go149 Statistics for these drops are captured in LogStat for monitoring the health of the logging system intra/log/logstat.go27-28
Sources: intra/log/logger.go149 intra/log/logstat.go27-28
The system provides methods for collecting and reporting stack traces with intelligent suppression of duplicate traces.
The ring[T] buffer stores the most recent log messages for inclusion in crash reports. It uses a background goroutine process() to handle input from a channel inC.
| Aspect | Detail |
|---|---|
| Type | ring[*[]byte] - Generic circular buffer of pooled byte slabs |
| Capacity | 512 messages (qSize) |
| Backpressure | Non-blocking Push; drops messages if channel is full |
| Management | onEvict callback recycles slabs back to logpool |
Sources: intra/log/ring.go15-24 intra/log/ring.go48-60 intra/log/logger.go94 intra/log/logger.go228
The system provides comprehensive panic recovery with flight recording for diagnostics.
Recover and RecoverFn are used to catch panics. RecoverFn is designed for background goroutines and allows for a Finally cleanup function. core.Go and its variants automatically wrap execution in Recover to prevent the entire process from crashing on routine errors intra/core/async.go31-40
Sources: intra/core/dontpanic.go73-88 intra/core/dontpanic.go166-179 intra/core/async.go31-40
A trace.FlightRecorder is integrated to capture runtime traces for post-mortem diagnostics.
captureRecorderOutput writes the flight recording to a pre-configured file using WriteRecordingTo intra/core/dontpanic.go123-148log.C and log.T are used to dump goroutine stacks to the console or log file during critical failures intra/log/log.go214-241Sources: intra/core/dontpanic.go60-68 intra/core/dontpanic.go123-148 intra/core/dontpanic.go207-232
The panic recovery system uses a global sync.RWMutex named _pmu to ensure that only one goroutine is responsible for printing the final panic message and exiting the process.
DontExit, the process exits with Exit11 (int 11) after logging intra/core/dontpanic.go39-43 intra/core/dontpanic.go191applog ensures that even during a panic, the final logs are flushed to the console before the process terminates intra/core/dontpanic.go181-205Sources: intra/core/dontpanic.go36-47 intra/core/dontpanic.go181-205
Sources: intra/log/log.go167-206 intra/core/dontpanic.go73-88 intra/core/dontpanic.go166-179
Refresh this wiki