Documentation
¶
Overview ¶
Package storage is the mAPI-ng data plane: the ClickHouse-backed writer and query layer for Summary rows. Percentiles are computed at query time from the merged DDSketch bucket maps, matching the frozen gamma/offset contract shared with the client (ADR-0001, ADR-0003).
Index ¶
- Constants
- Variables
- func ApplyMigrations(ctx context.Context, conn driver.Conn, log *slog.Logger) error
- func HasAnySummary(ctx context.Context, conn driver.Conn, tenantID tenant.ID) (bool, error)
- func LastIngestByTenant(ctx context.Context, conn driver.Conn) (map[string]time.Time, error)
- func LatencyByStatusClass(ctx context.Context, conn driver.Conn, tenantID tenant.ID, ...) (map[string]ClassLatency, error)
- func QuantileFromBuckets(buckets map[int32]uint64, q float64) float64
- type ClassLatency
- type Config
- type DownstreamStat
- type EndpointDetail
- type EndpointStat
- type ErrorClass
- type ErrorClassStat
- type Exemplar
- type ExemplarRow
- type HistogramBar
- type InstanceResourceStat
- type InstanceStat
- type InstanceWindowRow
- type MemoryTrendPoint
- type NoStatusReason
- type NoStatusReasonStat
- type OperatorQuery
- type PerformanceStat
- type QueryService
- type Row
- type ServiceStat
- type SketchBucket
- type StatusClassCount
- type StatusCode
- type TenantQuery
- func (q TenantQuery) DownstreamForEndpoint(ctx context.Context, service, method, route string, from, to time.Time) (DownstreamStat, error)
- func (q TenantQuery) EndpointDetail(ctx context.Context, service, method, route string, from, to time.Time) (EndpointDetail, error)
- func (q TenantQuery) Endpoints(ctx context.Context, service string, from, to time.Time) ([]EndpointStat, error)
- func (q TenantQuery) ErrorClassesForEndpoint(ctx context.Context, service, method, route string, from, to time.Time) ([]ErrorClassStat, error)
- func (q TenantQuery) ExemplarsForEndpoint(ctx context.Context, service, method, route string, from, to time.Time) ([]ExemplarRow, error)
- func (q TenantQuery) HasAnySummary(ctx context.Context) (bool, error)
- func (q TenantQuery) InstanceResourcesForService(ctx context.Context, service string, from, to time.Time) ([]InstanceResourceStat, error)
- func (q TenantQuery) InstancesForEndpoint(ctx context.Context, service, method, route string, from, to time.Time) ([]InstanceStat, error)
- func (q TenantQuery) LatencyByStatusClass(ctx context.Context, service, method, route string, from, to time.Time) (map[string]ClassLatency, error)
- func (q TenantQuery) MemoryTrendForService(ctx context.Context, service string, from, to time.Time, step time.Duration) ([]MemoryTrendPoint, error)
- func (q TenantQuery) NoStatusReasonsForEndpoint(ctx context.Context, service, method, route string, from, to time.Time) ([]NoStatusReasonStat, error)
- func (q TenantQuery) PerformanceStats(ctx context.Context, from, to time.Time) (PerformanceStat, error)
- func (q TenantQuery) SeriesOverTime(ctx context.Context, service, method, route string, from, to time.Time, ...) ([]TimePoint, error)
- func (q TenantQuery) Services(ctx context.Context, from, to time.Time) ([]ServiceStat, error)
- func (q TenantQuery) Usage(ctx context.Context, now time.Time) (TenantUsage, error)
- func (q TenantQuery) VersionsForEndpoint(ctx context.Context, service, method, route string, from, to time.Time) ([]VersionStat, error)
- type TenantUsage
- type TimePoint
- type VersionStat
- type Writer
Constants ¶
const ( // SketchGamma is the DDSketch relative-error base (gamma). SketchGamma = 1.01 // SketchGammaPlusOne is gamma+1, precomputed for value(i). SketchGammaPlusOne = SketchGamma + 1.0 )
Sketch mapping constants. These are the FROZEN DDSketch contract shared with the client and encoded in query.go's percentile SQL. gamma = 1.01; a bucket index i represents latency value(i) = 2*pow(gamma, i)/(gamma+1) seconds. Changing these is a breaking migration.
const ( // DefaultFlushInterval bounds how long a row waits before being inserted. DefaultFlushInterval = 2 * time.Second // DefaultFlushRows triggers a flush once this many rows are buffered. DefaultFlushRows = 100_000 // DefaultInsertTimeout bounds a single batch insert so a ClickHouse // connection that stalls mid-insert cannot block the sole batcher goroutine // indefinitely (and, through it, wedge all ingest). It is a hang guard, not // an SLA: it must exceed a healthy insert's latency by a wide margin. On // expiry the steady-state flush drops the batch and the final drain retries. DefaultInsertTimeout = 30 * time.Second )
Batcher flush thresholds: app-side batcher, no ClickHouse Buffer engine.
Variables ¶
var ErrEmptyTenant = errors.New("storage: row has empty tenant")
ErrEmptyTenant is returned by the enqueue methods when a row carries the zero-value tenant.ID. Rows are normally built from a resolved tenant (see ingest.summaryToRow), so the type already keeps an unvalidated tenant out of a query; this guards the one hole the type leaves open — a row struct-literal'd with the zero ID — so an un-tenanted write fails closed instead of silently persisting under an empty tenant. It is the write-path half of ADR-0010.
var ErrWriterClosed = errors.New("storage: writer closed")
ErrWriterClosed is returned by Enqueue after Close has been called.
Functions ¶
func ApplyMigrations ¶
ApplyMigrations runs every embedded ClickHouse migration in lexical order. Every statement is idempotent (CREATE ... IF NOT EXISTS, ALTER ... MODIFY TTL), so applying them on every startup is safe — this mirrors the control plane's applyMigrations and keeps mAPI-ng zero-config: a fresh ClickHouse gets its schema at boot rather than needing an out-of-band migrate step.
func HasAnySummary ¶
HasAnySummary reports whether tenant has at least one summary row in the raw tier. It drives the dashboard's onboarding gate (step 4: first data received) and the "show dashboard vs. onboarding panel" decision — cheap existence check, not a full aggregate. Uses the raw tier because onboarding cares about "any data ever ingested", and the raw tier is where fresh summaries land.
func LastIngestByTenant ¶ added in v0.11.0
LastIngestByTenant returns the most recent ingest time for every tenant that has ever ingested, read in one scan of the 1d tier. It is a deliberate CROSS-TENANT read: unlike the tenant-scoped dashboard API (reached only through Tenant), this powers the operator console's account list and is exposed through the dedicated Operator handle, so cross-tenant reads stay explicit and greppable rather than smuggled into the scoped path. It returns only timestamps, never tenant metrics.
func LatencyByStatusClass ¶
func LatencyByStatusClass( ctx context.Context, conn driver.Conn, tenantID tenant.ID, service, method, route string, from, to time.Time, ) (map[string]ClassLatency, error)
LatencyByStatusClass returns the per-status-class latency split for one endpoint for tenant over [from, to). The returned map is keyed on the status_class Enum8 values (STATUS_CLASS_2XX, _3XX, _4XX, _5XX, _NO_STATUS) and always contains every class: classes with no traffic in the window map to a zero-valued ClassLatency, so the caller sees a stable, complete shape. It answers whether a latency rise is on failures or also on successes. method and route may be empty to aggregate across all methods / all route templates.
func QuantileFromBuckets ¶
QuantileFromBuckets computes a quantile from a merged DDSketch bucket map using the FROZEN convention that MUST match both the client and query.go's SQL:
total = sum of counts rank = clamp(ceil(q * total), 1, total) walk buckets in ASCENDING index order accumulating counts; the first bucket whose cumulative count >= rank gives the answer; return value(index).
It returns 0 for an empty sketch. This is the Go oracle the integration test asserts the SQL against, and it keeps the storage package meaningfully covered without a live ClickHouse.
Types ¶
type ClassLatency ¶
ClassLatency is the per-status-class latency split for an endpoint: the merged request count in the class and its p50/p95/p99 over that class's own sketch.
type Config ¶
type Config struct {
// DSN is the ClickHouse native-protocol connection string.
DSN string
// FlushInterval bounds batcher latency.
FlushInterval time.Duration
// FlushRows triggers a flush once buffered rows reach this count.
FlushRows int
// InsertTimeout bounds a single batch insert so a stalled connection cannot
// block the batcher goroutine indefinitely. Defaults to DefaultInsertTimeout.
InsertTimeout time.Duration
}
Config holds the ClickHouse connection and batcher settings.
func ConfigFromEnv ¶
func ConfigFromEnv() Config
ConfigFromEnv builds a Config from MAPING_CLICKHOUSE_DSN, falling back to the local dev default.
type DownstreamStat ¶
DownstreamStat is the self-vs-downstream time split for an endpoint over the window: the request count, the total request time, and the total time spent waiting on downstream calls. Self time is SumDurationNs - SumDownstreamNs. A zero SumDownstreamNs means no downstream timing was reported (the RoundTripper is unwired), which the UI uses to feature-gate the panel.
func DownstreamForEndpoint ¶
func DownstreamForEndpoint( ctx context.Context, conn driver.Conn, tenantID tenant.ID, service, method, route string, from, to time.Time, ) (DownstreamStat, error)
DownstreamForEndpoint returns the self-vs-downstream time split for the given endpoint for tenant over [from, to). method and route may be empty to aggregate across all methods / all route templates of the service. It answers "how much of this endpoint's latency is its own work vs waiting on a dependency?".
type EndpointDetail ¶
type EndpointDetail struct {
Count uint64
ErrorRate float64
P50 float64
P95 float64
P99 float64
Histogram []HistogramBar
StatusClasses []StatusClassCount
StatusCodes map[uint32]uint64
}
EndpointDetail is the endpoint-detail level's aggregate: headline RED numbers, the DDSketch-derived latency histogram, the per-class breakdown, and the exact status-code map. All computed over the whole [from, to) window.
func QueryEndpointDetail ¶
func QueryEndpointDetail( ctx context.Context, conn driver.Conn, tenantID tenant.ID, service, method, route string, from, to time.Time, ) (EndpointDetail, error)
QueryEndpointDetail returns the full aggregate for one endpoint over [from, to): the merged sketch as a latency histogram, the per-class breakdown, the exact status-code map, and p50/p95/p99 computed in Go from the same merged sketch via QuantileFromBuckets — so the percentiles and the histogram can never disagree. Returns a zero-count EndpointDetail (no error) when the endpoint has no data in the window. Named QueryEndpointDetail (not EndpointDetail) because the return type already owns that identifier; the QueryService method is EndpointDetail, since methods have their own namespace.
type EndpointStat ¶
type EndpointStat struct {
Method string
Route string
Count uint64
ErrorRate float64
Client4xxRate float64
P50 float64
P95 float64
P99 float64
ReqBytesAvg float64
RespBytesAvg float64
}
EndpointStat is one row of the endpoint-table level: aggregate RED metrics for one (method, route_template) of a service over the query window. The ReqBytesAvg / RespBytesAvg fields are the per-request average payload sizes (sum(req_bytes)/sum(count), sum(resp_bytes)/sum(count)) for the bytes-symmetry view; they are appended so existing scanners and callers stay valid.
type ErrorClass ¶
ErrorClass is one normalized error-label count for the error_classes map column. Class is an uppercase [A-Z0-9_] token bounded by the client.
type ErrorClassStat ¶
ErrorClassStat is one row of the error-class breakdown for an endpoint: a normalized error label and how many requests carried it in the window. It answers "5xx up because of what?" ("DB_POOL_EXHAUSTED" vs "UPSTREAM_TIMEOUT").
func ErrorClassesForEndpoint ¶
func ErrorClassesForEndpoint( ctx context.Context, conn driver.Conn, tenantID tenant.ID, service, method, route string, from, to time.Time, ) ([]ErrorClassStat, error)
ErrorClassesForEndpoint returns the top error-class labels attached to the given endpoint for tenant over [from, to), ordered by request count descending and capped at maxErrorClassesPerEndpoint. method and route may be empty to aggregate across all methods / all route templates of the service.
type Exemplar ¶
type Exemplar struct {
At time.Time
DurationNs uint64
StatusCode uint32
TraceID string
SpanID string
RequestID string
}
Exemplar is one real-request breadcrumb attached to a Row. It is stored only in the raw summaries tier (rollups drop it) and lets a user pivot from a p99 / error spike to an actual request. TraceID/SpanID/RequestID are best-effort and may be empty. The tuple column order in ClickHouse is (at, duration_ns, status_code, trace_id, span_id, request_id).
type ExemplarRow ¶
type ExemplarRow struct {
At time.Time
DurationNs uint64
StatusCode uint32
TraceID string
SpanID string
RequestID string
}
ExemplarRow is one real-request breadcrumb read back for the endpoint-detail view: the flattened exemplar plus its status class (derived at query time from the status code) so the caller can label errors without a second lookup.
func ExemplarsForEndpoint ¶
func ExemplarsForEndpoint( ctx context.Context, conn driver.Conn, tenantID tenant.ID, service, method, route string, from, to time.Time, ) ([]ExemplarRow, error)
ExemplarsForEndpoint returns up to maxExemplarsPerEndpoint real-request breadcrumbs for the given endpoint for tenant over [from, to), ordered by latency descending, so a user can pivot from a p99 / error spike to an actual request to open in a tracing tool or logs. method and route may be empty to aggregate across all methods / all route templates of the service.
It reads the RAW summaries tier directly (NOT via selectTier): exemplars live only in the finest tier under its short TTL, so this query is only meaningful for recent windows within the raw retention. Older windows return no exemplars even though the RED aggregates remain available on the rollups.
type HistogramBar ¶
HistogramBar is one bar of the latency histogram: the bucket's latency value in seconds (from the frozen value(i) mapping) and the merged count in it.
type InstanceResourceStat ¶
type InstanceResourceStat struct {
Instance string
CPUNs uint64
RSSBytes uint64
HeapAllocBytes uint64
GCPauseNs uint64
Goroutines uint64
NumGC uint64 // completed GC cycles over the window (delta sum)
TotalAllocBytes uint64 // heap bytes allocated over the window (delta sum)
Mallocs uint64 // heap objects allocated over the window (delta sum)
GCCPUFraction float64 // average GC CPU fraction over the window (gauge avg)
HeapInuseBytes uint64 // peak in-use heap bytes over the window (gauge max)
GOMAXPROCS uint32 // peak GOMAXPROCS over the window (gauge max)
PostGCHeapBytes uint64 // peak post-GC heap baseline over the window (gauge max)
RSSTrueBytes uint64 // peak true resident set size over the window (gauge max)
OpenFDs uint64 // peak open file-descriptor count over the window (gauge max)
FDLimit uint64 // peak soft RLIMIT_NOFILE ceiling over the window (gauge max)
InFlight uint64 // peak in-flight request concurrency over the window (gauge max)
}
InstanceResourceStat is the per-instance resource summary over the query window: CPU and GC-pause time summed (they are per-window deltas), and the peak memory / goroutine gauges (max over the window). It answers "which replica is saturated, and how?".
func InstanceResourcesForService ¶
func InstanceResourcesForService( ctx context.Context, conn driver.Conn, tenantID tenant.ID, service string, from, to time.Time, ) ([]InstanceResourceStat, error)
InstanceResourcesForService returns one InstanceResourceStat per instance of the service for tenant over [from, to), ordered by instance. It is the USE side of the debug view: saturation gauges to sit next to the RED metrics so a latency rise can be attributed to GC or goroutine growth without a release.
type InstanceStat ¶
type InstanceStat struct {
Instance string
Count uint64
ErrorRate float64
P50 float64
P95 float64
P99 float64
ReqBytesAvg float64
RespBytesAvg float64
}
InstanceStat is one row of the instance-outlier breakdown: aggregate RED metrics plus the average payload sizes for a single instance (replica) of an endpoint over the query window. ErrorRate is a fraction in [0,1]; the ...Avg fields are per-request byte averages (sum(bytes)/count).
func InstancesForEndpoint ¶
func InstancesForEndpoint( ctx context.Context, conn driver.Conn, tenantID tenant.ID, service, method, route string, from, to time.Time, ) ([]InstanceStat, error)
InstancesForEndpoint returns one InstanceStat per instance (replica) serving the given endpoint for tenant over [from, to), ordered by instance. method and route may be empty to aggregate across all methods / all route templates of the service. This is the flagship outlier query: it answers whether a degradation is confined to one replica or is fleet-wide. Percentiles use the frozen DDSketch convention; the error rate is (4xx+5xx+no_status)/total.
type InstanceWindowRow ¶
type InstanceWindowRow struct {
Tenant tenant.ID
Service string
Instance string
WindowStart time.Time
WindowEnd time.Time
CPUNs uint64
RSSBytes uint64
HeapAllocBytes uint64
GCPauseNs uint64
Goroutines uint64
NumGC uint64 // completed GC cycles during the window (delta)
TotalAllocBytes uint64 // heap bytes allocated during the window (delta)
Mallocs uint64 // heap objects allocated during the window (delta)
GCCPUFraction float64 // fraction of CPU time in GC since start (gauge, 0..1)
HeapInuseBytes uint64 // in-use heap bytes at sample time (gauge)
GOMAXPROCS uint32 // GOMAXPROCS at sample time (gauge)
PostGCHeapBytes uint64 // live heap as of the last GC mark, post-GC baseline (gauge)
RSSTrueBytes uint64 // true resident set size from the OS, 0 if unavailable (gauge)
OpenFDs uint64 // open file-descriptor count, 0 if unavailable (gauge)
FDLimit uint64 // soft RLIMIT_NOFILE ceiling, 0 if unavailable (gauge)
InFlight uint64 // peak in-flight request concurrency during the window (gauge)
}
InstanceWindowRow is one process resource snapshot (USE gauges) ready for insertion into the instance_windows table: the server-resolved tenant plus service/instance and the per-window gauges. cpu_ns and gc_pause_ns are per-window deltas; the rest are point-in-time reads. It is a separate stream from Row (per-endpoint summaries).
type MemoryTrendPoint ¶
MemoryTrendPoint is one time-bucket of the service's fleet memory: the peak post-GC live heap (the leak-vs-burst signal) and peak in-use heap over the bucket, across every instance/sample. Memory is a per-process property of the instances serving the service (instance_windows has no endpoint dimension), so this trend is service-scoped, not per-endpoint.
func MemoryTrendForService ¶
func MemoryTrendForService( ctx context.Context, conn driver.Conn, tenantID tenant.ID, service string, from, to time.Time, step time.Duration, ) ([]MemoryTrendPoint, error)
MemoryTrendForService returns the per-bucket fleet memory trend for a service over [from, to), bucketed by step, ordered by time. It reads the same [from, to, step] window the detail-page timeline uses so heap and latency line up. step is floored to one second; the caller passes the timeline step.
type NoStatusReason ¶
NoStatusReason is one reason count for the no_status_reasons map column. Reason is the proto NoStatusReason enum value, stored as UInt8 (the domain is a tiny fixed enum).
type NoStatusReasonStat ¶
NoStatusReasonStat is one row of the NO_STATUS reason breakdown: the proto NoStatusReason enum value (as UInt8) and how many aborted requests it explains, telling apart timing-out vs canceling vs crashing.
func NoStatusReasonsForEndpoint ¶
func NoStatusReasonsForEndpoint( ctx context.Context, conn driver.Conn, tenantID tenant.ID, service, method, route string, from, to time.Time, ) ([]NoStatusReasonStat, error)
NoStatusReasonsForEndpoint returns the NO_STATUS reason breakdown for the given endpoint for tenant over [from, to), ordered by reason. method and route may be empty to aggregate across all methods / all route templates of the service.
type OperatorQuery ¶ added in v0.11.0
type OperatorQuery struct {
// contains filtered or unexported fields
}
OperatorQuery is the cross-tenant read surface for the operator console. It is the deliberate counterpart to TenantQuery: where Tenant(id) scopes every read to one validated tenant, Operator() reaches reads that span all tenants. Keeping it a distinct handle (rather than adding cross-tenant methods to the scoped path) makes every cross-tenant read explicit at the call site and greppable in review.
func (OperatorQuery) LastIngestByTenant ¶ added in v0.11.0
LastIngestByTenant forwards to the package-level cross-tenant last-ingest scan.
type PerformanceStat ¶
PerformanceStat is the real, tenant-scoped basis for the performance page: how many requests the tenant's stored summaries represent, how many summary rows were actually shipped to carry them, and an estimate of the on-disk bytes those summaries occupy. Requests/Summaries are measured exactly; SummaryDiskBytes multiplies the tenant's summary-row count by the measured average on-disk size of a summary row (falling back to a constant when system.parts is unavailable).
func PerformanceStats ¶
func PerformanceStats( ctx context.Context, conn driver.Conn, tenantID tenant.ID, from, to time.Time, ) (PerformanceStat, error)
PerformanceStats measures, for tenant over [from, to), the total requests the stored summaries represent (sum of the per-summary count) and the number of shipped summary rows, both from the raw summaries tier. It then estimates the on-disk size of those summaries from the measured average summary-row size (system.parts, best-effort — a failure falls back to a constant rather than failing the page). Their ratio is the compression the page reports; the caller projects the raw-event-pipeline size from Requests and a documented per-event assumption (mAPI-ng stores no raw events, so that side cannot be measured).
type QueryService ¶
type QueryService struct {
// contains filtered or unexported fields
}
QueryService wraps a ClickHouse connection to expose the read API as methods, so callers (the web layer) can depend on a small interface rather than a free function plus a raw driver.Conn.
func NewQueryService ¶
func NewQueryService(conn driver.Conn) *QueryService
NewQueryService builds a QueryService over an open connection.
func (*QueryService) Operator ¶ added in v0.11.0
func (s *QueryService) Operator() OperatorQuery
Operator returns the cross-tenant operator read handle. Reads issued through it are not tenant-scoped and are intended only for the allowlisted operator console.
func (*QueryService) Tenant ¶
func (s *QueryService) Tenant(id tenant.ID) TenantQuery
Tenant binds the service to a tenant, returning the scoped query handle through which every data-plane read is issued. The tenant is a validated tenant.ID, so a query can never be scoped to an unvalidated identity.
type Row ¶
type Row struct {
Tenant tenant.ID
Service string
Instance string
Method string
RouteTemplate string
StatusClass string // Enum8 string value, e.g. "STATUS_CLASS_2XX".
WindowStart time.Time
WindowEnd time.Time
Count uint64
SumDurationNs uint64
ReqBytes uint64
RespBytes uint64
Sketch []SketchBucket // sorted ascending by Index.
StatusCodes []StatusCode // sorted ascending by Code.
MaxDurationNs uint64 // exact slowest request in the window; merges via max.
Exemplars []Exemplar // bounded breadcrumbs; raw tier only.
ErrorClasses []ErrorClass // sorted ascending by Class; merges via sumMap.
NoStatusReasons []NoStatusReason // sorted ascending by Reason; merges via sumMap.
SumDownstreamNs uint64 // summed downstream wait time; merges via sum.
// Deploy identity: stored low-cardinality dimensions carried from the
// Envelope. DeployVersion is a sort-key dimension (rows from different
// versions never collapse); the rest are non-key stored columns.
// InstanceStart is the process boot wall-clock (zero when the client did not
// report one).
DeployVersion string
DeployID string
Environment string
Region string
InstanceStart time.Time
}
Row is one Summary ready for insertion: the server-resolved tenant plus the full series key, RED counters, and the merged-at-query DDSketch buckets. The sketch keys are pre-sorted (see NewRow) for deterministic map ordering.
func NewRow ¶
func NewRow( tenantID tenant.ID, service, instance, method, routeTemplate, statusClass string, windowStart, windowEnd time.Time, count, sumDurationNs, reqBytes, respBytes uint64, sketch map[int32]uint64, statusCodes map[uint32]uint64, deployVersion, deployID, environment, region string, instanceStart time.Time, maxDurationNs uint64, exemplars []Exemplar, errorClasses map[string]uint64, noStatusReasons map[uint32]uint64, sumDownstreamNs uint64, ) Row
NewRow builds a Row from an already-mapped sketch and status-code maps, sorting both by key so the ClickHouse map column ordering is deterministic across inserts.
type ServiceStat ¶
type ServiceStat struct {
Service string
Count uint64
ErrorRate float64
Client4xxRate float64
P50 float64
P95 float64
P99 float64
}
ServiceStat is one row of the service-overview level: aggregate RED metrics for a whole service over the query window. ErrorRate is a fraction in [0,1].
func Services ¶
func Services( ctx context.Context, conn driver.Conn, tenantID tenant.ID, from, to time.Time, ) ([]ServiceStat, error)
Services returns one aggregate ServiceStat per service for tenant over [from, to), ordered by Count descending. Percentiles use the frozen DDSketch convention; the error rate is the CONTEXT (4xx+5xx+no_status)/total.
type SketchBucket ¶
SketchBucket is one DDSketch bucket: an index and its count. Rows carry the sketch as a sorted slice (not a map) so the map column insert order into ClickHouse is deterministic.
type StatusClassCount ¶
StatusClassCount is the merged request count for one status class label (2xx/3xx/4xx/5xx/no_status), used for the breakdown beside the error rate.
type StatusCode ¶
StatusCode is one exact-code count for the status_codes map column.
type TenantQuery ¶
type TenantQuery struct {
// contains filtered or unexported fields
}
TenantQuery is a QueryService bound to a single tenant. It is the ONLY way to reach the data-plane read API: QueryService exposes no tenant-taking methods, so a cross-tenant read is unrepresentable — callers must go through Tenant(tenant) and the bound tenant threads into every query.
func (TenantQuery) DownstreamForEndpoint ¶
func (q TenantQuery) DownstreamForEndpoint( ctx context.Context, service, method, route string, from, to time.Time, ) (DownstreamStat, error)
DownstreamForEndpoint forwards to the package-level self-vs-downstream time split for the bound tenant.
func (TenantQuery) EndpointDetail ¶
func (q TenantQuery) EndpointDetail( ctx context.Context, service, method, route string, from, to time.Time, ) (EndpointDetail, error)
EndpointDetail forwards to the package-level EndpointDetail aggregate for the bound tenant.
func (TenantQuery) Endpoints ¶
func (q TenantQuery) Endpoints( ctx context.Context, service string, from, to time.Time, ) ([]EndpointStat, error)
Endpoints forwards to the package-level Endpoints aggregate for the bound tenant.
func (TenantQuery) ErrorClassesForEndpoint ¶
func (q TenantQuery) ErrorClassesForEndpoint( ctx context.Context, service, method, route string, from, to time.Time, ) ([]ErrorClassStat, error)
ErrorClassesForEndpoint forwards to the package-level error-class breakdown for the bound tenant.
func (TenantQuery) ExemplarsForEndpoint ¶
func (q TenantQuery) ExemplarsForEndpoint( ctx context.Context, service, method, route string, from, to time.Time, ) ([]ExemplarRow, error)
ExemplarsForEndpoint forwards to the package-level raw-tier exemplar breadcrumb query for the bound tenant.
func (TenantQuery) HasAnySummary ¶
func (q TenantQuery) HasAnySummary(ctx context.Context) (bool, error)
HasAnySummary forwards to the package-level existence check for the bound tenant.
func (TenantQuery) InstanceResourcesForService ¶
func (q TenantQuery) InstanceResourcesForService( ctx context.Context, service string, from, to time.Time, ) ([]InstanceResourceStat, error)
InstanceResourcesForService forwards to the package-level per-instance USE gauge breakdown for the bound tenant.
func (TenantQuery) InstancesForEndpoint ¶
func (q TenantQuery) InstancesForEndpoint( ctx context.Context, service, method, route string, from, to time.Time, ) ([]InstanceStat, error)
InstancesForEndpoint forwards to the package-level instance-outlier breakdown for the bound tenant.
func (TenantQuery) LatencyByStatusClass ¶
func (q TenantQuery) LatencyByStatusClass( ctx context.Context, service, method, route string, from, to time.Time, ) (map[string]ClassLatency, error)
LatencyByStatusClass forwards to the package-level per-class latency split for the bound tenant.
func (TenantQuery) MemoryTrendForService ¶
func (q TenantQuery) MemoryTrendForService( ctx context.Context, service string, from, to time.Time, step time.Duration, ) ([]MemoryTrendPoint, error)
MemoryTrendForService forwards to the package-level per-window memory trend for the bound tenant.
func (TenantQuery) NoStatusReasonsForEndpoint ¶
func (q TenantQuery) NoStatusReasonsForEndpoint( ctx context.Context, service, method, route string, from, to time.Time, ) ([]NoStatusReasonStat, error)
NoStatusReasonsForEndpoint forwards to the package-level NO_STATUS reason breakdown for the bound tenant.
func (TenantQuery) PerformanceStats ¶
func (q TenantQuery) PerformanceStats( ctx context.Context, from, to time.Time, ) (PerformanceStat, error)
PerformanceStats forwards to the package-level performance basis (represented requests, shipped summaries, estimated summary disk) for the bound tenant.
func (TenantQuery) SeriesOverTime ¶
func (q TenantQuery) SeriesOverTime( ctx context.Context, service, method, route string, from, to time.Time, step time.Duration, ) ([]TimePoint, error)
SeriesOverTime forwards to the package-level query using the wrapped connection and the bound tenant.
func (TenantQuery) Services ¶
func (q TenantQuery) Services( ctx context.Context, from, to time.Time, ) ([]ServiceStat, error)
Services forwards to the package-level Services aggregate using the wrapped connection and the bound tenant.
func (TenantQuery) Usage ¶ added in v0.11.0
func (q TenantQuery) Usage(ctx context.Context, now time.Time) (TenantUsage, error)
Usage forwards to the package-level operator volumetry for the bound tenant.
func (TenantQuery) VersionsForEndpoint ¶
func (q TenantQuery) VersionsForEndpoint( ctx context.Context, service, method, route string, from, to time.Time, ) ([]VersionStat, error)
VersionsForEndpoint forwards to the package-level per-deploy-version breakdown for the bound tenant.
type TenantUsage ¶ added in v0.11.0
type TenantUsage struct {
FirstIngest time.Time
LastIngest time.Time
Endpoints uint64
Series uint64
Services uint64
Instances uint64
Requests30d uint64
DiskBytes uint64
}
TenantUsage is the operator-facing volumetry for one tenant: liveness (First/LastIngest), current cardinality (Endpoints/Series/Services/Instances), recent activity (Requests over the last 30 days), and an estimate of the on-disk bytes the tenant's raw summaries occupy. Series counts distinct (method, route_template, status_class) — the exact key the cardinality guardrail meters (guardrail.SeriesKey), so a caller can honestly render "series vs cap". A never-ingested tenant yields the zero value (zero times, zero counts).
func Usage ¶ added in v0.11.0
func Usage(ctx context.Context, conn driver.Conn, tenantID tenant.ID, now time.Time) (TenantUsage, error)
Usage assembles the operator volumetry for tenantID as of now. It runs three cheap aggregates: liveness from the 1d tier (longest retention, so a churned tenant still shows its last-ingest date), cardinality and 30-day requests from the 1m tier, and the disk estimate from the raw tier via PerformanceStats. now is a parameter (not time.Now()) so the windows are deterministic in tests.
type TimePoint ¶
type TimePoint struct {
TS time.Time
Count uint64
ErrorRate float64
P50 float64
P95 float64
P99 float64
}
TimePoint is one time-bucket of the RED distribution: request rate inputs (Count over the step), the derived error rate, and p50/p95/p99 latency in seconds computed from the merged DDSketch buckets.
func SeriesOverTime ¶
func SeriesOverTime( ctx context.Context, conn driver.Conn, tenantID tenant.ID, service, method, route string, from, to time.Time, step time.Duration, ) ([]TimePoint, error)
SeriesOverTime returns the per-time-bucket RED distribution for a series over [from, to), bucketed by step. method and route may be empty to aggregate across all methods / all route templates of the service. Percentiles use the frozen DDSketch convention above; the same math runs client-side, which is the whole correctness point.
type VersionStat ¶
type VersionStat struct {
Version string
Count uint64
ErrorRate float64
P50 float64
P95 float64
P99 float64
}
VersionStat is one row of the per-deploy-version breakdown: aggregate RED metrics for a single deploy_version of an endpoint over the query window. ErrorRate is a fraction in [0,1]. It answers "did release X regress this endpoint?" by comparing metrics across versions.
func VersionsForEndpoint ¶
func VersionsForEndpoint( ctx context.Context, conn driver.Conn, tenantID tenant.ID, service, method, route string, from, to time.Time, ) ([]VersionStat, error)
VersionsForEndpoint returns one VersionStat per deploy_version serving the given endpoint for tenant over [from, to), ordered by deploy_version. method and route may be empty to aggregate across all methods / all route templates of the service. It answers whether a release regressed an endpoint by putting each version's RED metrics side by side. Percentiles use the frozen DDSketch convention; the error rate is (4xx+5xx+no_status)/total.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer batches Row inserts into ClickHouse. A single background goroutine drains an unbounded-intent buffered channel and flushes on either the flush interval or the row threshold, whichever comes first. Close drains and flushes remaining rows within the caller's context deadline.
func NewWriter ¶
NewWriter opens a ClickHouse connection from cfg and starts the batcher goroutine. The caller owns the returned Writer and must call Close.
func (*Writer) Close ¶
Close signals the batcher to drain the buffered rows and stop, then waits for the run loop to exit or ctx to expire.
func (*Writer) Enqueue ¶
Enqueue hands a row to the batcher. It returns ErrEmptyTenant if the row has no tenant (fail-closed, never persisted) and ErrWriterClosed once Close has been called. It never blocks indefinitely: if the buffer is full it blocks only until the batcher drains, which is bounded by the flush cadence.
func (*Writer) EnqueueInstanceWindow ¶
func (w *Writer) EnqueueInstanceWindow(row InstanceWindowRow) error
EnqueueInstanceWindow hands one USE-gauge row to the batcher's instance-window stream. Like Enqueue it returns ErrWriterClosed after Close and never blocks indefinitely (bounded by the flush cadence).
func (*Writer) Migrate ¶
Migrate applies the embedded ClickHouse schema on the writer's connection. Keeping the raw connection inside the storage package means callers cannot run un-scoped SQL against the data plane: reads go through QueryService.Tenant and writes through Enqueue.
func (*Writer) QueryService ¶
func (w *Writer) QueryService() *QueryService
QueryService returns a read handle over the writer's connection. Reads remain tenant-scoped — callers reach a query only via QueryService.Tenant(tenant) — so the raw connection never leaves the storage package.