Spring AI 2.0 in 10 Minutes: Structured Output, Tools and RAG With Real Code

Vighneshwar Bhat

6 min read

·

Oct 14, 2026

--

Listen

Share

Three features you’ll use in almost every AI feature you build, and the one mistake in RAG that most tutorials skip.

Press enter or click to view image in full size

Spring AI 2.0 went GA in June 2026. It runs on Spring Boot 4 and Spring Framework 7, so if you’re still on Boot 3, that’s the first upgrade on your list.

This isn’t a tour of every module. It’s the three things you’ll reach for on day one: getting typed objects back from a model, letting the model call your Java code, and answering questions from your own documents. Every snippet follows the Spring AI 2.0 reference docs. Check the exact names for your patch version, because the project moves quickly.

AI Engineer Interview Guide for Java Developers.

Setup

Import the Spring AI BOM, then add the model starter for your provider and the modules you need:

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.springframework.ai</groupId>
      <artifactId>spring-ai-bom</artifactId>
      <version>2.0.1</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

Put your API key in configuration that comes from a secret store, never in code, and pin the exact model version per environment. Changing the model should be a reviewed config change, not a code release.

Everything below starts from a ChatClient:

@RestController
class AskController {
    private final ChatClient chatClient;

    AskController(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }
}

1. Structured output: get a Java object, not a string

Say you’re extracting fields from supplier invoices. You want an Invoice, not a paragraph you have to parse.

record Invoice(@NotBlank String supplier,
               @PositiveOrZero BigDecimal total,
               @NotNull LocalDate date) {}

Invoice invoice = chatClient.prompt()
        .user("Extract the invoice fields from this text:\n" + text)
        .call()
        .entity(Invoice.class, spec -> spec
                .useProviderStructuredOutput()   // provider enforces the schema, if it can
                .validateSchema());              // re-ask the model if the reply breaks it

What’s happening here:

• .entity(Invoice.class) generates a JSON schema from your record and maps the reply into it.
• By default, the schema is only instructions in the prompt. The model is asked to follow it, but it can’t be forced to.
• useProviderStructuredOutput() asks providers that support it to enforce the schema at the API level. If yours doesn't, Spring AI quietly falls back to the prompt approach.
• validateSchema() checks the reply against the schema. If it fails, it adds the error to the prompt and tries again, up to three attempts by default.

Here’s the part tutorials skip: a valid shape is not a true answer. A total of -500 is perfectly valid JSON. So check business rules yourself:

Set<ConstraintViolation<Invoice>> errors = validator.validate(invoice);
if (!errors.isEmpty()) {
    // re-ask once with the errors, or send the invoice to a person
}

One more limit: typed parsing needs the complete reply, so it doesn’t work on a streaming response.

2. Tools: let the model ask your code to do things

A customer asks your support bot “Where’s my order?” The model doesn’t know. Your order service does. Tool calling connects the two.

enum Period { LAST_7_DAYS, LAST_30_DAYS }

class OrderTools {
    @Tool(description = "List the signed-in customer's recent orders. "
            + "Use for 'where is my order' questions.")
    List<OrderSummary> recentOrders(
            @ToolParam(description = "Time window") Period period,
            ToolContext ctx) {
        String customerId = (String) ctx.getContext().get("customerId");
        return orders.recent(customerId, period);
    }
}String answer = chatClient.prompt()
        .user(question)
        .tools(new OrderTools())
        .toolContext(Map.of("customerId", currentCustomerId()))
        .call()
        .content();

The flow is simple. Spring AI sends the tool’s name, description and parameter schema to the model. The model replies “call recentOrders with LAST_7_DAYS". Spring AI runs your method, sends the result back, and the model writes the final answer.

The important detail is ToolContext. The model never runs your code, and it should never decide who the user is. If customerId were a normal parameter, the model would fill it in, and a message like "show me customer 7's orders" could steer it. With toolContext, your code passes the id from the logged-in session, and it never goes through the model.

A few habits that make tools work well:

• Write descriptions that say when to use the tool and when not to. The model chooses tools by reading them.
• Use typed parameters. An enum beats free text.
• Return short, structured results and error messages the model can act on, like “Order not found, ask the customer to check the number”.
• Keep write actions, like cancelling an order, behind a confirmation step, and make them safe to call twice.

3. RAG: answer from your own documents

Now the HR assistant. An employee asks “How many days of parental leave do I get?” and the answer is in a policy PDF.

Ingestion (ahead of time): read the documents, split them into chunks, store them in a vector store. Here we also put the document id in the text, so the model can cite it, and the access group in the metadata, so we can filter by it.

List<Document> chunks = new TokenTextSplitter()
        .apply(new TikaDocumentReader(resource).get())
        .stream()
        .map(c -> new Document("[" + docId + "] " + c.getText(),
                Map.of("docId", docId, "accessGroup", accessGroup)))
        .toList();
vectorStore.add(chunks);

Query time: the QuestionAnswerAdvisor searches the vector store and adds the best chunks to the prompt.

ChatClient rag = ChatClient.builder(chatModel)
        .defaultAdvisors(QuestionAnswerAdvisor.builder(vectorStore)
                .searchRequest(SearchRequest.builder().topK(6).build())
                .build())
        .build();

The mistake most tutorials skip

That advisor, as written, searches every document in the store. If the HR folder has restricted files, such as salary bands or a disciplinary policy, anyone can get answers from them.

The fix is a filter on every request, built from the user’s verified groups:

String groups = String.join("', '", currentUser.groups());   // from the login token

String answer = rag.prompt()
        .system("Answer only from the context. After each fact, cite its [docId]. "
              + "If the answer is not in the context, say you don't know.")
        .user(question)
        .advisors(a -> a.param(QuestionAnswerAdvisor.FILTER_EXPRESSION,
                "accessGroup in ['" + groups + "']"))
        .call()
        .content();

Two things matter here. First, filter inside the search, not after it. If you filter afterwards, the top six results might all be documents the user can’t see, and you’re left with nothing useful. Second, the groups come from the verified login token, never from anything the user typed.

The system prompt does the other half of the work: it tells the model to stick to the retrieved text, cite the [docId], and say "I don't know" instead of guessing. That last instruction prevents more wrong answers than any clever prompt.

Bonus: streaming in one line

Long answers feel faster when words appear as they’re generated:

@GetMapping(value = "/ask", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
Flux<String> ask(@RequestParam String q) {
    return chatClient.prompt().user(q).stream().content()
            .timeout(Duration.ofSeconds(60));
}

If the browser disconnects, WebFlux cancels the Flux, which should also cancel the provider request so you stop paying for tokens nobody reads. Check that your provider client actually does this.

Before you ship

What I’d check before putting any of this in front of users:

• Hide it behind your own interface. Business code shouldn’t import Spring AI directly. Then switching frameworks or providers later means changing one adapter, not the whole codebase.
• Pin versions. That means the model version, the Spring AI version and the embedding model. A silent change in any of them can change your answers.
• Write an evaluation set. Collect 50 real questions with what a good answer must contain, and run them on every prompt or model change.
• Log IDs, not content. Log the prompt version, model version, chunk ids, token counts and latency. Full prompts and answers can contain personal data.
• Set timeouts and retries. Use timeouts on every call, and retry only transient errors such as 429s, 5xx and timeouts. Most providers bill retries again.

The short version

• .entity() gets you typed objects. Add useProviderStructuredOutput() and validateSchema(), then still validate business rules.
• @Tool lets the model call your code. Pass identity through ToolContext, never as a model-filled argument.
• QuestionAnswerAdvisor gives you RAG in a few lines, but it searches everything unless you pass a filter. Filter by the user's groups on every request.

This article comes from my AI Engineer Interview Guide for Java Developers: about 100 interview questions with answers, 26 production incident scenarios, 9 system designs, and a companion Maven project with every Java snippet. If you’re preparing for AI engineering interviews, you can find it here: AI Engineer Interview Guide for Java Developers.