prom-forge

prom-forge

Config-driven synthetic metrics generator for Prometheus using remote_write.

Quick Start

Setup Prometheus Locally

1. Download the Prometheus Agent Binary and move it to your path.

2. Run the prometheus agent that we downloaded earlier adding the --web.enable-remote-write-receiver flag in order to enable the /api/v1/write endpoint that accepts pushed metrics.

   prometheus --config.file examples/configs/prometheus.yaml --web.enable-remote-write-receiver --web.external-url=http://localhost:9090/
   

Running Prom-forge

CLI

1. Edit the example config in examples/configs/example.yaml to your liking.

2. Open a separate terminal in order to run the cli to write metrics to the local prometheus server.

   go run . --config examples/configs/example.yaml
   

API

1. Navigate to Go examples in this repository.

2. Run an example.

   go run config/example.go --config ../configs/example.yaml
   

3. (Optionally) Import prom-forge into an existing project.

   go get github.com/spectrocloud-labs/prom-forge@latest
   

Generating Data

Generating Data in the Past

You can generate data in the past. Below is a configuration to generate a steady gpu utilization metric for the past 15m at an interval of 15s plus a random jitter between 0-2s.

prometheus:
  remote_write_url: "http://localhost:9090/"
  insecure_skip_verify: true
metrics:
- name: "gpu_utilization"
  type: "gauge"
  utilization_pattern:
    steady:
      slope: 0.0
      offset: 7.0
  labels:
    node: edge-ffa238429efe572a777ef4a17e4fd9b7
  tick: false
  interval_duration: 15s
  jitter_duration: 2s
  time_machine_duration: 15m

Generating Data in the Present

You can generate data in the present. Below is a configuration to generate a steady gpu utilization metric at an interval of 5s plus a random jitter between 0-2s.

prometheus:
  remote_write_url: "http://localhost:9090/"
  insecure_skip_verify: true
metrics:
- name: "gpu_utilization"
  type: "gauge"
  utilization_pattern:
    steady:
      slope: 0.0
      offset: 7.0
  labels:
    node: edge-ffa238429efe572a777ef4a17e4fd9b7
  interval_duration: 5s
  jitter_duration: 2s

Generating Data in the Past and the Present

You can generate data in the past and the present.

Below is a configuration to generate a steady gpu utilization metric for the past 15m and the present at an interval of 5s plus a random jitter between 0-2s.

prometheus:
  remote_write_url: "http://localhost:9090/"
  insecure_skip_verify: true
metrics:
- name: "gpu_utilization"
  type: "gauge"
  utilization_pattern:
    steady:
      slope: 0.0
      offset: 7.0
  labels:
    node: edge-ffa238429efe572a777ef4a17e4fd9b7
  interval_duration: 5s
  jitter_duration: 2s
  time_machine_duration: 15m

Note, you can also generate data in the past that has a different utilization pattern than data in the present. This can be helpful for mocking CPU, GPU, etc. signal behaviors. Below is a configuration to generate a steady gpu utilization metric for the past 15m at an interval of 5s plus a random jitter between 0-2s, then generate a random gpu utilization metric for the present at an interval of 5s plus a random jitter between 0-2s.

prometheus:
  remote_write_url: "http://localhost:9090/"
  insecure_skip_verify: true
metrics:
- name: "gpu_utilization"
  type: "gauge"
  utilization_pattern:
    steady:
      slope: 0.0
      offset: 7.0
  labels:
    node: edge-ffa238429efe572a777ef4a17e4fd9b7
  tick: false
  interval_duration: 5s
  jitter_duration: 2s
  time_machine_duration: 15m
- name: "gpu_utilization"
  type: "gauge"
  utilization_pattern:
    random:
      max: 100.0
      min: 50.0
  labels:
    node: edge-ffa238429efe572a777ef4a17e4fd9b7
  interval_duration: 2s
  jitter_duration: 5s

Config

# Note: be sure to add the following flag when running the prometheus server: --web.enable-remote-write-receiver
prometheus:
  # the remote write url to write the metrics to (required)
  remote_write_url: string
  # certificate check switch (optional)
  insecure_skip_verify: bool
  # certificate file to use for the remote write url (optional)
  ca_file: string
  # basic authentication configuration (optional)
  basic_auth:
    username: string
    password: string
metrics:
  # metric name (required)
- name: string
  # the metric type (required, "gauge" or "counter")
  type: string
  # utilization pattern for metric (required)
  utilization_pattern:
    # write a value defined by a linear function: value(t) = offset + slope * elapsed_seconds.
    # for gauges, slope = units/second of change; slope: 0.0 yields a constant value equal to offset.
    # for counters, slope = units/second of growth (must be >= 0); slope: 0.0 yields a flat counter at offset.
    steady:
      slope: float64    # rate of change per second; 0.0 = constant; counters require >= 0
      offset: float64   # starting value at t=0
    # write a uniform random value in [min, max) at each sample (counter rate stays in [min, max))
    random:
      max: float64
      min: float64
    # oscillate between 2 values
    oscillating:
      phase_a:
        value: float64
        hold_count: int
        ramp_steps: int
      phase_b:
        value: float64
        hold_count: int
        ramp_steps: int
  # labels for metric (optional)
  labels:
    string: string
  # generate metrics in the present (optional, default: true)
  tick: bool
  # writes metric at the given interval duration (required)
  interval_duration: time.Duration
  # add jitter between 0 and the given value to the interval duration (default: 0s)
  jitter_duration: time.Duration
  # generates metrics in the past for the time machine duration (default: 0s)
  time_machine_duration: time.Duration