Centralized Logging: Spring Boot Meets Elasticsearch, Logstash, and Kibana
Build a production-grade ELK observability stack for Spring Boot, end-to-end — with structured JSON logs, distributed trace IDs, and ECS-compliant fields ready for any backend.
In this post, we cover how to implement structured logging for Spring Boot using the ELK stack — Elasticsearch, Logstash, and Kibana — with logs shaped to the Elastic Common Schema (ECS).
Prerequisites
This is the list of all the prerequisites:
- Spring Boot 3 or later
- Maven 3.6.3 or later
- Java 21 or later
- Docker / Docker Compose installed
- At least 4 GB of free RAM for the ELK containers
- Postman or any other API testing tool
- IntelliJ IDEA, or another IDE
Overview
Modern applications generate an overwhelming amount of logs. When you’re running multiple services — especially in a microservices architecture — tracking down issues without a centralized logging system quickly becomes a nightmare.
The ELK stack has been the de facto standard for centralized logging in the Java ecosystem for over a decade. It’s deployed across banking, telco, e-commerce, and pretty much every other enterprise space you can name. If you’ve worked on an enterprise Spring Boot project, you’ve almost certainly shipped logs to Elasticsearch at some point.
Today, we’ll build the modern version of that stack: Elasticsearch for storage and search, Logstash for the ingestion and transformation pipeline, and Kibana for visualization — with our application logs shaped to the Elastic Common Schema so they’re portable across vendors.
What is Elasticsearch?
Elasticsearch is a distributed, RESTful search and analytics engine built on Apache Lucene. Unlike systems that only index metadata, Elasticsearch indexes the full content of every document by default — every field, every word, every value. That’s what makes it fast for arbitrary queries like “find every log line containing this stack trace fragment from last month.” The trade-off is higher storage and memory cost compared to label-indexed systems.
For logs, Elasticsearch typically stores documents in time-based indices (e.g. spring-logs-2026.03.23), which makes retention, sharding, and lifecycle management straightforward.
📚 https://www.elastic.co/elasticsearch
What is Logstash?
Logstash is the transformation engine of the ELK stack — a server-side data processing pipeline that ingests data from many sources, transforms it, and ships it to a destination (typically Elasticsearch).
A Logstash pipeline has three stages:
- Input — where logs come from (files, TCP, Beats, Kafka, etc.)
- Filter — parse, enrich, rename, drop, mutate
- Output — where transformed logs go (Elasticsearch, S3, Kafka, etc.)
Logstash is heavier than lightweight collectors like Filebeat, but its filter stage is where it earns its keep: complex parsing, conditional logic, field renaming, GeoIP enrichment, anonymization, and more. For this article, we’ll use it to convert our raw log fields into ECS-compliant documents before they hit Elasticsearch.
📚 https://www.elastic.co/logstash
What is Kibana?
Kibana is the visualization and exploration UI for Elasticsearch. It lets you query, filter, chart, alert on, and dashboard your logs, metrics, and traces. For day-to-day log investigation, the Discover view is what you’ll live in — a column-pickable, filterable, time-bucketed log explorer.
📚 https://www.elastic.co/kibana
What is ECS?
The Elastic Common Schema (ECS) is an open specification for a consistent set of field names across all observability data. Instead of every team and every service inventing its own field names (appName, app_name, application.name, serviceName…), ECS standardizes on service.name, http.request.method, event.duration, trace.id, and so on.
Why this matters: ECS-shaped logs are portable. The same JSON document you ship to Elasticsearch can be ingested by Loki, Splunk, Datadog, or any modern observability backend with minimal rework. Adopt it from day one — your future self will thank you when a client switches vendors.
📚 https://www.elastic.co/guide/en/ecs/current/index.html
Why Centralized Logging Matters
Imagine debugging a production issue where requests pass through multiple services. Without centralized logging, you would:
- SSH into multiple servers
- Manually search log files
- Correlate timestamps across systems
This is slow, error-prone, and frustrating.
Centralized logging solves this by:
- Aggregating logs in one place
- Making logs searchable and filterable across services
- Enabling real-time monitoring and alerting
- Letting you follow a single
trace.idacross every microservice it touched
Architecture Overview
Our logging pipeline will look like this:
Spring Boot App
│
│ (Writes structured JSON logs to file)
▼
./logs/*.log
│
│ (Logstash tails log files via file input plugin)
▼
Logstash
├─ input : file (json codec)
├─ filter : rename → ECS, drop, mutate, enrich
└─ output : elasticsearch
│
▼
Elasticsearch
│
│ (Indexes documents into time-based indices)
▼
Kibana
│
│ (Discover, dashboards, alerts via KQL)
▼
DeveloperEach component has a focused responsibility:
- Spring Boot — generates structured JSON logs with trace IDs via Logback + the
logstash-logback-encoder - Logstash — tails the log files, parses each line, renames fields to ECS, and ships to Elasticsearch
- Elasticsearch — indexes and stores logs for fast querying
- Kibana — visualizes and queries logs via KQL (Kibana Query Language)
A note on collectors. In production, the canonical ELK setup is Filebeat → Logstash → Elasticsearch — Filebeat is a lightweight collector that sits on each host, and Logstash runs centrally.
Project Setup
We’ll create a simple Spring Boot project from start.spring.io, with the following dependencies: Spring Web, Lombok, Spring Boot Actuator, H2, Spring Data JPA, and Validation.
The base project is a REST API exposing /books and /authors endpoints, backed by an H2 database. The application code itself is incidental — what matters for this post is the logging configuration that sits underneath it.
Configuring Spring Boot for Structured Logging
To make logs machine-parseable and easy to query in Elasticsearch, we need JSON output. The dependency to add is logstash-logback-encoder — and yes, the name is no coincidence. This encoder was originally built for the ELK stack, so we're using it in its native habitat.
<!-- pom.xml -->
<properties>
<logstash-logback-encoder.version>8.0</logstash-logback-encoder.version>
</properties>
<dependencies>
<dependency>
<groupId>net.logstash.logback</groupId>
<artifactId>logstash-logback-encoder</artifactId>
<version>${logstash-logback-encoder.version}</version>
</dependency>
</dependencies>That dependency turns every log event into a JSON object. But a flat JSON message isn’t enough — we need each log line to carry request context: which request was being served, who the caller was, which trace it belongs to. That’s where MDC comes in.
Understanding MDC: The Hidden Engine of Contextual Logging
MDC stands for Mapped Diagnostic Context. It’s a tiny but powerful concept that does a lot of heavy lifting in any production logging setup — and it’s worth taking a moment to understand before we use it.
The problem MDC solves
Imagine a typical request flow:
HTTP request → AuthFilter → BookController → BookService → BookRepositoryA single request can produce dozens of log lines from different classes:
[BookController] Received GET /books/42
[BookService] Looking up book by id 42
[BookRepository] SELECT * FROM books WHERE id = ?
[BookService] Cache miss, fetching from DB
[BookController] Returning 200 OKNow imagine 500 requests per second hitting the service in parallel. Those log lines from one request are scattered across thousands of other log lines from concurrent requests. How do you reconstruct what happened to request #42?
You could pass a traceId parameter into every method:
bookService.findById(id, traceId);
log.info("Looking up book {} for trace {}", id, traceId);…but that’s awful. Every method in our codebase would need an extra parameter. Every log statement would need extra interpolation.
What MDC actually is
MDC is essentially a ThreadLocal<Map<String, String>> that lives inside Logback (technically SLF4J). When we do this at the start of a request:
MDC.put("traceId", "abc-123");…we’ve stored that key-value pair for the current thread only. Then, every subsequent call to log.info(...), log.error(...), etc. — from any class, anywhere in the call chain, as long as it runs on the same thread — automatically picks up that traceId and attaches it to the emitted log event.
The BookRepository doesn't have to know anything about tracing. It just calls log.info(...) and the trace ID magically appears in the output:
{ "@timestamp": "...", "logger": "BookRepository", "message": "SELECT * FROM books...", "traceId": "abc-123" }That’s the whole trick. MDC is the bridge that lets the filter at the front of the request enrich every log line that gets produced during that request — without touching any of the code in between.
How MDC interacts with the JSON encoder
This is the nice part: we don’t have to do anything special. The logstash-logback-encoder we just added automatically reads the current thread's MDC map and includes every entry as a top-level field in the JSON output. Put traceId in MDC → "traceId" shows up in the JSON. Put userId in MDC → "userId" shows up in the JSON. No further configuration needed.
The one rule you must never break
try {
MDC.put("traceId", traceId);
// ... do work ...
} finally {
MDC.remove("traceId"); // ← MANDATORY
}Because MDC is ThreadLocal and modern web servers reuse threads (Tomcat has a thread pool), if you forget to clear MDC at the end of the request, the next request that lands on the same thread will inherit the previous request's traceId. Always clean up in a finally block.
With that understanding, the rest of the article will make a lot more sense — particularly the HTTP filter, which is essentially “the thing that puts stuff into MDC at the start of every request and takes it out at the end.”
Logging Configuration Properties
Rather than scattering values across the codebase, all tuning knobs live in one @ConfigurationProperties class bound to the logging.http.* namespace.sd
// config/LoggingProperties.java
@Getter
@Setter
@ConfigurationProperties(prefix = "logging.http")
public class LoggingProperties {
/** URI prefixes that bypass the filter entirely. */
private Set<String> excludedPaths = Set.of("/actuator", "/health");
/**
* When true, request and response bodies are included in the log event.
* Keep false in production to avoid logging sensitive payloads.
* See the "Optional: Capturing Request and Response Bodies" section below.
*/
private boolean logBody = false;
/** Cap on request/response body size when logBody is true. Larger bodies are truncated. */
private int maxBodySize = 10_000;
}// config/LoggingConfig.java
@Configuration
@EnableConfigurationProperties(LoggingProperties.class)
public class LoggingConfig {
}Inbound HTTP Logging Filter
The HTTP filter is where the theory from the MDC section becomes practice. Its job is dead simple:
- Before the request reaches the controller: put a
traceIdinto MDC. - Let the request run.
- After the request finishes: log a one-line summary (method, URL, status, duration) and clean up MDC.
The filter extends Spring’s OncePerRequestFilter — guaranteed to run exactly once per request, regardless of servlet dispatch type — and the @Order(Ordered.HIGHEST_PRECEDENCE) annotation puts it first in the filter chain so the MDC is set up before anything else runs.
// filter/HttpLoggingFilter.java
@Component
@Order(Ordered.HIGHEST_PRECEDENCE)
public class HttpLoggingFilter extends OncePerRequestFilter {
private static final Logger log = LoggerFactory.getLogger(HttpLoggingFilter.class);
private static final String MDC_TRACE_ID = "traceId";
private final LoggingProperties props;
public HttpLoggingFilter(LoggingProperties props) {
this.props = props;
}
@Override
protected boolean shouldNotFilter(HttpServletRequest request) {
// Skip health and actuator endpoints — they'd otherwise flood the logs
String path = request.getRequestURI();
return props.getExcludedPaths().stream().anyMatch(path::startsWith);
}
@Override
protected void doFilterInternal(HttpServletRequest request,
HttpServletResponse response,
FilterChain filterChain)
throws ServletException, IOException {
// Reuse the upstream trace ID if there is one (distributed tracing),
// otherwise generate a fresh one for this request
String traceId = Optional.ofNullable(request.getHeader("X-Trace-Id"))
.filter(s -> !s.isBlank())
.orElseGet(() -> UUID.randomUUID().toString());
MDC.put(MDC_TRACE_ID, traceId);
response.addHeader("X-Trace-Id", traceId); // propagate to the caller
long startTime = System.currentTimeMillis();
try {
filterChain.doFilter(request, response);
} finally {
logExchange(request, response, System.currentTimeMillis() - startTime);
MDC.remove(MDC_TRACE_ID);
}
}
private void logExchange(HttpServletRequest request,
HttpServletResponse response,
long durationMs) {
Map<String, Object> fields = new LinkedHashMap<>();
fields.put("event.kind", "event");
fields.put("event.category", "web");
fields.put("event.action", "http-request");
fields.put("http.method", request.getMethod());
fields.put("http.url", request.getRequestURI());
fields.put("http.query", request.getQueryString() != null ? request.getQueryString() : "");
fields.put("http.status_code", response.getStatus());
// ECS expects event.duration in nanoseconds — convert from ms here
fields.put("event.duration", durationMs * 1_000_000L);
fields.put("client.ip", resolveClientIp(request));
log.info(appendEntries(fields),
"HTTP {} {} -> {} ({}ms)",
request.getMethod(), request.getRequestURI(),
response.getStatus(), durationMs);
}
private String resolveClientIp(HttpServletRequest request) {
String xff = request.getHeader("X-Forwarded-For");
return (xff != null && !xff.isBlank())
? xff.split(",")[0].trim()
: request.getRemoteAddr();
}
}That’s the entire filter — about 50 lines, no magic. The try/finally is the critical part: it guarantees that even if the controller throws an exception, we still log the exchange and clean up MDC.
Get Lampis Papantoniou’s stories in your inbox
Join Medium for free to get updates from this writer.
Notice what we’re not doing here: we’re not reading the request or response body. That’s a different problem with its own complications, so we treat it separately.
Optional: Capturing Request and Response Bodies
Sometimes you want the full payload in the log — typically for debugging integrations, never in production. Adding body logging needs a little extra plumbing because of how the Servlet API works.
Why bodies are tricky
HttpServletRequest.getInputStream() returns a one-shot stream. You can read it once. If your filter reads the body to log it, the controller downstream gets an empty stream — broken. If you instead try to read the body after the controller, it's already been consumed.
You’d be stuck: log the body, or deliver it to the controller, never both.
Spring’s ContentCachingRequestWrapper solves this. It wraps the real request and transparently copies bytes into an internal buffer as the controller reads them. So:
Filter wraps the request
→ calls filterChain.doFilter(wrappedRequest, ...)
→ controller reads body ← bytes flow through wrapper, get cached
→ filter resumes
→ wrappedRequest.getContentAsByteArray() ← cached bytes now available!ContentCachingResponseWrapper is the same trick in reverse — it caches the response body as your controller writes it. The catch: you must call copyBodyToResponse() at the end to flush the cached bytes back out to the actual response, otherwise the client receives an empty body.
How to enable body logging
If logBody=true in properties, wrap the request and response. Otherwise, don't — wrapping has a measurable performance cost and isn't free.
Here’s the relevant change in doFilterInternal:
boolean cacheBodies = props.isLogBody();
HttpServletRequest reqToUse = cacheBodies
? new ContentCachingRequestWrapper(request, props.getMaxBodySize())
: request;
HttpServletResponse respToUse = cacheBodies
? new ContentCachingResponseWrapper(response)
: response;
try {
filterChain.doFilter(reqToUse, respToUse);
} finally {
logExchange(reqToUse, respToUse, System.currentTimeMillis() - startTime);
// Critical: log FIRST, THEN copyBodyToResponse.
// copyBodyToResponse() drains the cached buffer back to the real response,
// so if you call it before logging, the cached body is gone.
if (respToUse instanceof ContentCachingResponseWrapper w) {
w.copyBodyToResponse();
}
MDC.remove(MDC_TRACE_ID);
}Then in logExchange, when bodies are cached, append them:
if (request instanceof ContentCachingRequestWrapper req && props.isLogBody()) {
byte[] body = req.getContentAsByteArray();
if (body.length > 0) {
fields.put("http.request.body", new String(body, StandardCharsets.UTF_8));
}
}
if (response instanceof ContentCachingResponseWrapper resp && props.isLogBody()) {
byte[] body = resp.getContentAsByteArray();
if (body.length > 0) {
fields.put("http.response.body", new String(body, StandardCharsets.UTF_8));
}
}That’s it. With logging.http.log-body=true, every exchange logs the bodies; with the default false, the wrappers are skipped entirely.
Treat body logging as a debugging tool, not a production default. Bodies typically contain PII, credentials, and tokens. If you must enable it, pair it with a scrubbing step in the Logstash pipeline (out of scope for this article).
Logback JSON Configuration
logback-spring.xml replaces Spring Boot's default text-based logging with LogstashEncoder — the JSON serializer from logstash-logback-encoder. It outputs every log event as a single-line JSON object, writes it to a rolling file, and mirrors it to the console outside of production.
<configuration>
<include resource="org/springframework/boot/logging/logback/defaults.xml"/>
<springProperty name="APP_NAME" source="spring.application.name" defaultValue="application"/>
<springProperty name="APP_VERSION" source="spring.application.version" defaultValue="unknown"/>
<springProperty name="ENVIRONMENT" source="spring.profiles.active" defaultValue="default"/>
<appender name="JSON_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>./logs/${APP_NAME}.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>./logs/${APP_NAME}.%d{yyyy-MM-dd}.%i.log.gz</fileNamePattern>
<maxFileSize>100MB</maxFileSize>
<maxHistory>30</maxHistory>
<totalSizeCap>3GB</totalSizeCap>
</rollingPolicy>
<encoder class="net.logstash.logback.encoder.LogstashEncoder">
<!-- Static fields stamped onto every log event -->
<customFields>{"service":{"name":"${APP_NAME}","version":"${APP_VERSION}","environment":"${ENVIRONMENT}"}}</customFields>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="JSON_FILE"/>
</root>
</configuration>About 25 lines, doing exactly four things:
- Reads Spring properties (
spring.application.name, etc.) so we don't hardcode values. - Rotates the log file daily and whenever it hits 100 MB, gzipping old segments, keeping 30 days of history and capping total disk usage at 3 GB.
- Serializes log events as JSON via
LogstashEncoder. - Stamps every event with
service.name,service.version, andservice.environment.
A sample log event from this configuration looks like this:
{
"@timestamp": "2026-03-23T10:14:22.481923100+01:00",
"@version": "1",
"message": "HTTP GET /books -> 200 (42ms)",
"logger_name": "com.example.demo.filter.HttpLoggingFilter",
"thread_name": "http-nio-8080-exec-1",
"level": "INFO",
"level_value": 20000,
"service": { "name": "spring-boot-elk-demo", "version": "0.0.1-SNAPSHOT", "environment": "dev" },
"traceId": "9a3b1f...",
"event.kind": "event",
"event.category": "web",
"event.action": "http-request",
"http.method": "GET",
"http.url": "/books",
"http.status_code": 200,
"event.duration": 42000000,
"client.ip": "172.18.0.1"
}That’s not yet ECS-compliant: traceId should be trace.id, http.method should be http.request.method, level should be log.level, and so on. Logstash is where we'll finish the job.
Setting Up the ELK Stack
Docker Compose Stack
This Docker Compose file brings up Elasticsearch, Logstash, and Kibana. The Spring Boot service joins via a shared elk network and a volume that maps ./logs (where the app writes) to /var/log/spring (where Logstash reads).
# docker-compose.yml
version: "3.8"
services:
# Uncomment and adjust once the image is built
# api:
# build: .
# ports:
# - "8080:8080"
# volumes:
# - ./logs:/app/logs # app writes here; Logstash reads /var/log/spring
# networks:
# - elk
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:8.15.0
environment:
- discovery.type=single-node
- xpack.security.enabled=false # demo only — enable in production
- ES_JAVA_OPTS=-Xms1g -Xmx1g
ports:
- "9200:9200"
volumes:
- es-data:/usr/share/elasticsearch/data
networks:
- elk
healthcheck:
test: ["CMD-SHELL", "curl -s http://localhost:9200/_cluster/health | grep -q '\"status\":\"\\(green\\|yellow\\)\"'"]
interval: 10s
timeout: 5s
retries: 10
logstash:
image: docker.elastic.co/logstash/logstash:8.15.0
environment:
- LS_JAVA_OPTS=-Xms512m -Xmx512m
volumes:
- ./logs:/var/log/spring # ← Spring app log directory
- ./docker/logstash/pipeline:/usr/share/logstash/pipeline:ro # pipeline definitions
- ./docker/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml:ro
- logstash-data:/usr/share/logstash/data # for sincedb persistence
depends_on:
elasticsearch:
condition: service_healthy
networks:
- elk
kibana:
image: docker.elastic.co/kibana/kibana:8.15.0
environment:
- ELASTICSEARCH_HOSTS=http://elasticsearch:9200
ports:
- "5601:5601"
depends_on:
elasticsearch:
condition: service_healthy
networks:
- elk
volumes:
es-data:
logstash-data:
networks:
elk:
driver: bridgeA minimal docker/logstash/config/logstash.yml:
http.host: "0.0.0.0"
xpack.monitoring.enabled: falseResource note. Elasticsearch’s JVM heap (
-Xms1g -Xmx1g) plus Logstash's (-Xms512m -Xmx512m) plus Kibana puts you around 3 GB of container memory just for the stack.
Logstash Pipeline
This is where we take the raw JSON our Spring Boot app emits and reshape it into ECS-compliant documents. The pipeline has three sections — input, filter, output — and it reads top-to-bottom like a Unix pipeline.
Create docker/logstash/pipeline/logstash.conf:
# docker/logstash/pipeline/logstash.conf
input {
file {
path => "/var/log/spring/*.log"
start_position => "beginning"
sincedb_path => "/usr/share/logstash/data/sincedb_spring"
codec => "json" # parse each line as JSON automatically
}
}
filter {
# Use the application's own timestamp, not the time Logstash ingested the line.
# Without this, Kibana would order logs by ingestion time — misleading once Logstash buffers.
date {
match => ["@timestamp", "ISO8601"]
target => "@timestamp"
}
# Rename our raw fields to their ECS counterparts.
# The left side is the *flat field name* the app emitted (with literal dots in the name).
# The right side is a *nested ECS path* — square-bracket notation is Logstash's way of
# saying "create a nested object". So "[http][request][method]" becomes { "http": { "request": { "method": ... } } }.
mutate {
rename => {
"traceId" => "[trace][id]"
"logger_name" => "[log][logger]"
"thread_name" => "[process][thread][name]"
"level" => "[log][level]"
"http.method" => "[http][request][method]"
"http.url" => "[url][path]"
"http.query" => "[url][query]"
"http.status_code" => "[http][response][status_code]"
"event.kind" => "[event][kind]"
"event.category" => "[event][category]"
"event.action" => "[event][action]"
"event.duration" => "[event][duration]"
"client.ip" => "[client][ip]"
}
}
# Ensure numeric fields are stored as numbers, not strings.
# Critical for range queries and aggregations later.
mutate {
convert => {
"[http][response][status_code]" => "integer"
"[event][duration]" => "integer"
}
}
# Drop the leftover noise we don't need in Elasticsearch.
mutate {
remove_field => ["@version", "level_value", "host", "path"]
}
}
output {
elasticsearch {
hosts => ["http://elasticsearch:9200"]
index => "spring-logs-%{+YYYY.MM.dd}"
}
# Echoes every transformed event to the Logstash container's stdout.
# Invaluable while developing the pipeline — comment out once stable.
stdout {
codec => rubydebug { metadata => false }
}
}A few things worth calling out:
codec => "json"on the input parses each line as JSON automatically. Nogrok, no regex, no fuss.mutate renameis where the ECS magic happens. Read each line as: "take the field named on the left, and move it to the nested location on the right."datefilter sets@timestampfrom the app's clock. Without it, you'd be looking at ingestion time, which drifts whenever Logstash buffers.convertmatters more than you'd think —status_codestored as a string can't be range-queried (>= 500), andevent.durationas a string can't be aggregated for latency percentiles.stdoutoutput is your friend during development. Taildocker compose logs -f logstashand you'll see exactly what hit Elasticsearch.
Elasticsearch Index Template
By default, Elasticsearch will guess field types from the first document it sees. That’s fine for a demo, but in real life you want an index template that locks in mappings — including making text fields searchable as full-text and as exact-match keywords.
Apply this once after Elasticsearch starts (Kibana → Dev Tools, or via curl):
PUT _index_template/spring-logs
{
"index_patterns": ["spring-logs-*"],
"template": {
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0
},
"mappings": {
"dynamic_templates": [
{
"strings_as_keyword_and_text": {
"match_mapping_type": "string",
"mapping": {
"type": "text",
"fields": { "keyword": { "type": "keyword", "ignore_above": 1024 } }
}
}
}
],
"properties": {
"@timestamp": { "type": "date" },
"log.level": { "type": "keyword" },
"service.name": { "type": "keyword" },
"service.environment": { "type": "keyword" },
"trace.id": { "type": "keyword" },
"http.request.method": { "type": "keyword" },
"http.response.status_code": { "type": "integer" },
"url.path": { "type": "keyword" },
"event.duration": { "type": "long" },
"client.ip": { "type": "ip" }
}
}
}
}This gives you sensible defaults — keyword fields for filtering, full-text indexing where it matters, and a proper ip type for client.ip.
Running the Stack
- Start the logging stack:
docker compose up -d - Wait for Elasticsearch to be healthy:
curl http://localhost:9200/_cluster/healthshould return"status":"green"or"yellow". - Apply the index template via Kibana Dev Tools or
curl. - Start your Spring Boot app:
./mvnw spring-boot:run - Generate some traffic — hit
/books,/authors, throw in some 4xx and 5xx for variety.
Verify Logstash is consuming logs
Tail the Logstash container logs:
docker compose logs -f logstashYou should see your events being printed by the stdout output we configured — that's your real-time confirmation that the pipeline is working end to end. If you see your event with properly nested trace.id, http.request.method, etc., you're golden.
You can also query Elasticsearch directly:
curl -s "http://localhost:9200/spring-logs-*/_search?pretty&size=1"Explore logs in Kibana
Open Kibana at http://localhost:5601.
- Create a data view — go to Stack Management → Data Views → Create, name it
spring-logs-*, and pick@timestampas the time field. - Open Discover — left sidebar → Discover. Select the
spring-logs-*data view. - Filter with KQL — try a few:
service.name : "spring-boot-elk-demo" and log.level : "ERROR"http.response.status_code >= 500trace.id : "9a3b1f..."event.duration > 1000000000 // requests slower than 1s- Pick columns — in the left panel, add
@timestamp,log.level,service.name,http.request.method,url.path,http.response.status_code,event.duration,trace.idas columns. This is the layout you'll use 90% of the time. - Save a search — give your column layout + KQL filter a name. Now you have a reusable view for “errors in service X over the last 24h.”
From here, the natural next steps are Dashboards (charts and metrics over time) and Alerting Rules (page someone when log.level : ERROR count spikes). Those are full topics in their own right, but you have everything you need to start.
Conclusion
Well done! You now have a working ELK pipeline that:
- Generates structured JSON logs from a Spring Boot service
- Uses MDC to attach a trace ID to every log line produced during a request — without your service code knowing anything about tracing
- Captures inbound HTTP exchanges with method, URL, status, duration, and trace ID
- Transforms raw application fields into ECS-compliant documents in Logstash
- Stores and indexes those documents in Elasticsearch with proper field types
- Queries them in Kibana with KQL, by trace ID, status code, latency, or anything else
The really important takeaway, though, isn’t the stack itself — it’s that your Spring Boot application code is essentially identical to what you’d write for Loki, Splunk, Datadog, or any other modern backend. The HttpLoggingFilter, the MDC trace IDs, the logstash-logback-encoder, the JSON output — all of it stays the same. Only the bottom half of the pipeline (collector → backend → UI) changes.
That portability is what ECS buys you. Adopt it once, and you can swap observability backends without rewriting your services. In an industry where your client might be on Splunk this year and Datadog next year and Grafana Cloud the year after, that’s worth more than any one tool.
Where to go next
- Add OpenTelemetry / Micrometer Tracing — replace the hand-rolled
X-Trace-IdUUID with W3Ctraceparentpropagation, automatic across HTTP/Kafka/JMS - Switch to Filebeat → Logstash — for production-grade collection with backpressure and per-host shipping
- Sensitive-data scrubbing — mask
Authorizationheaders and PII inside the Logstash filter stage - Dashboards and alerts — error-rate alerts, latency percentiles, traffic-by-endpoint charts
The complete source code is available on GitHub https://github.com/l-papantoniou/spring-boot-elk-logging.git.
Thank you for reading!
