Introduction

Azure DevOps is a robust platform for source code management, CI/CD pipelines, and project planning. In this article, we’ll show you how to connect your Azure DevOps repository to Choreo, build a service from that repository, and deploy it to the cloud using automated workflows.

Whether you are onboarding an existing application or deploying a new service, this integration streamlines deployments directly from your Azure DevOps Git repository.

Prerequisites

Before you begin, make sure you have:

1. A Choreo account
2. An Azure DevOps organization and project
3. A repository in Azure DevOps containing your application source code

Step 1 — Generate a Personal Access Token (PAT) in Azure DevOps

• Sign in to Azure DevOps (dev.azure.com).
• In the top-right corner, next to your profile picture → User Settings.
• Select Personal access tokens → New Token.
• Give the token a name like ChoreoAccessToken.
• Select Scopes: Code (Read), Project (Read) and Team (Read)
• Set the expiration and create the token.
• Copy the generated token and store it securely, as it cannot be viewed again.

Step 2 — Add Azure DevOps Credentials in Choreo

1. Log in to the Choreo Console.
2. Open your organization from the sidebar.
3. Navigate to Settings → Credentials.
4. Click + Add Credentials.
5. Fill in:

• Credential Name: e.g., AzureDevOpsCred
• Service Provider: Azure DevOps
• Access Token: Paste the PAT generated earlier

6. Click Save.

You should now see your Azure DevOps credential listed in Choreo.

Step 3 — Create a New Component Using Your Azure DevOps Repository

• Navigate back to your Choreo Organization homepage.
• Click + Create Project and enter a project name.
• Within the project, select a Component Type card.
• Select ‘Try a Different Git Provider’ -> Select ‘Authorize with Azure DevOps’ and select the previously created credential.

• Select: Project, Repository and Branch
• Enter the directory path
• Choose the corresponding build preset
• Click on Create and Deploy

Step 4 — Build the Component

• When the component creation is successful, you will be redirected into the Component Overview page.
• Navigate to the Build page from the left navigation menu.

• Once you are in the Build page, Click on Build Latest (to build the latest commit) or select Show Commits -> Select a commit from the commit list to start building your source code.
• You will be able to see the status of the build once it is in-progress.

Once the build is successful, navigate to the ‘Deploy’ page from the left menu.

Step 5 — Deploy to Production

• Click on ‘Configure & Deploy’ in the Set Up card to deploy to your first environment.
• To promote this service to the production environment, click ‘Promote’ and select the ‘Use Development configuration’ or ‘Add New Configurations’ option from the right drawer.

Your source code should now be in production!

To test the application, navigate to the Test -> Console page from the left menu.

Summary

This article explains how to take your code from Azure DevOps to the cloud using Choreo. It covers generating a PAT, adding Azure DevOps credentials in Choreo, creating a component from your repo, building the service, and deploying it to production — all in a streamlined, automated workflow.

Sign up to Choreo for free to explore more cool features.

From Azure DevOps to Cloud with Choreo: 5 Steps was originally published in Choreo Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Blazor has changed the game for web developers who prefer C# over JavaScript. If you’re looking to get your Blazor app to the cloud, Choreo’s Bring Your Own Container (BYOC) feature makes the process straightforward.

In this tutorial, you’ll learn how to set up a sample Blazor project and prepare it for deployment on Choreo.

What You’ll Learn

By the end of this guide, you’ll learn to:

• Set up a sample Blazor Web App project
• Create a production-ready Dockerfile for your Blazor application
• Prepare and deploy the application on Choreo

Prerequisites

Before you begin, make sure you have:

• A Choreo Account (Sign up for free at https://console.choreo.dev/signup)
• .NET 9.0 SDK installed
• Docker installed (for local testing)
• A GitHub account
• Basic familiarity with Blazor and containerization concepts

Let’s write some code!

Step 1: Setting Up the Sample Project

We’ll use the official Blazor Web App Movies sample from Microsoft’s blazor-samples repository. This sample contains a full-featured Blazor web application with database connectivity and CRUD operations.

Fork the Repository

First, navigate to the official Blazor samples repository and fork it to your GitHub account. This will allow you to make modifications and connect it to Choreo later.

Clone Your Fork

Once you’ve forked the repository, clone it to your local machine:

git clone https://github.com/<YOUR_USERNAME>/blazor-samples.git
cd blazor-samples/9.0/BlazorWebAppMovies

Test the Application Locally

Before containerizing the app, verify that it runs correctly on your local machine. This app requires an MSSQL database to run properly. If you don’t have one set up locally, you can either:

• Install MSSQL Server locally
• Use a Docker container for MSSQL

Once your database is available, update the connection string in appsettings.json or set it as an environment variable, then run:

dotnet restore
dotnet run

Open your browser and navigate to the URL displayed in the terminal (typically http://localhost:5216). You should see the Movies application, allowing you to create, edit, and delete movie records.

Step 2: Creating the Dockerfile

This is the most important step — creating a Dockerfile that packages your Blazor app for deployment on Choreo.

Create a new file called Dockerfile in the root of your BlazorWebAppMovies project directory.

Here’s what your Dockerfile should look like:

# Build stage - Use the .NET SDK to build the application
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
ARG BUILD_CONFIGURATION=Release
WORKDIR /src

# IMPORTANT FOR CHOREO: Create non-root user
# Choreo requires containers to run as non-root users for security
RUN groupadd -g 10014 choreo && \
    useradd --no-create-home --uid 10014 --gid 10014 choreouser

# Copy project file and restore dependencies
COPY . .
RUN dotnet restore "BlazorWebAppMovies.csproj"

# Build the project
RUN dotnet build "BlazorWebAppMovies.csproj" -c $BUILD_CONFIGURATION -o /app/build

# Publish stage - Create the final published output
FROM build AS publish
ARG BUILD_CONFIGURATION=Release
RUN dotnet publish "BlazorWebAppMovies.csproj" -c $BUILD_CONFIGURATION -o /app/publish /p:UseAppHost=false

# Runtime stage - Use the ASP.NET Core runtime for the final image
FROM mcr.microsoft.com/dotnet/aspnet:9.0 AS final
WORKDIR /app

# Use the non-root user ID
USER 10014

# Copy the published application from the publish stage
COPY --from=publish /app/publish .

# Expose ports
EXPOSE 8080
EXPOSE 8081

# Configure ASP.NET Core to listen on port 8080
ENV ASPNETCORE_URLS=http://+:8080

# Set the entry point to run the application
ENTRYPOINT ["dotnet", "BlazorWebAppMovies.dll"]

Let’s break down what each section does:

Build Stage:

• Uses the official .NET 9.0 SDK image as the build environment
• Copies the project file first to leverage Docker layer caching
• Restores NuGet packages
• Copies all source files and builds the application in Release mode

Publish Stage:

• Publishes the application, creating optimized assemblies for production
• Removes unnecessary build artifacts and prepares the application for deployment

Runtime Stage:

• Uses the lightweight ASP.NET Core runtime image
• Sets USER 10014. This is critical for Choreo deployment as it requires non-root users for security
• Configures the application to listen on port 8080/81
• Sets up the entry point to run your Blazor application

Step 3: Testing Your Docker Image Locally

Before deploying to Choreo, test the container locally.

# Build the Docker image
docker build -t blazor-movies-app .

# Run the container with your database connection string
docker run -p 8080:8080 \
  -e ConnectionStrings__BlazorWebAppMoviesContext="Host=your-db-host;Port=1433;Database=BlazorWebAppMovies;Username=sa;Password=YourPassword;TrustServerCertificate=true" \
  blazor-movies-app

Make sure to replace the connection string with your actual MSSQL database details. If you don’t have a database set up for testing, you can skip this step and test once deployed on Choreo.

Open your browser and navigate to http://localhost:8080 to verify that your containerized application works correctly.

Step 4: Commit and Push Your Changes

Once you’ve created and tested your Dockerfile, commit it to your forked repository.

Step 6: Deploying to Choreo

Now that your application is containerized and pushed to GitHub, you’re ready to deploy it on Choreo using the BYOC (Bring Your Own Container) approach.

[Important] Database Setup

This app needs an MSSQL database. You’ll need to add the connection string as an environment variable in Choreo. It should look something like this:

ConnectionStrings__BlazorWebAppMoviesContext=Host=mssql-host.com;Port=16638;Database=BlazorWebAppMovies;Username=sa;Password=super$tr0ngPassword;SslMode=Require;TrustServerCertificate=true

Refer to Choreo’s Manage Configurations and Secrets documentation on how to add environment variables.

The Deployment Process

Here’s what you’ll do in Choreo:

• Create a Web Application component
• Connect your GitHub repository
• Select “Dockerfile” as the build preset
• Set the Dockerfile path (./Dockerfile)
• Configure the build context (usually the project root)
• Set your environment variables (like that database connection string)
• Build and deploy to your selected environment

Choreo handles the heavy lifting automatically! Choreo will:

• Build your Docker image from the Dockerfile
• Scan for security vulnerabilities
• Deploy to Kubernetes
• Set up CI/CD pipelines for automatic deployments on code changes

For detailed instructions, check out the Choreo container deployment documentation.

Wrapping Up

And that’s it! You’ve successfully containerized a Blazor application and deployed it on Choreo using BYOC.

Happy Coding!

Deploy Blazor Web Apps on Choreo was originally published in Choreo Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

That experience sparked a question: could we keep the creative, vibe-coding flow while automating all the plumbing? We set out to find the answer by rebuilding MenuGen, combining a simple product brief from a large model with an implementation agent in VS Code. But we almost hit the same wall Karpathy did. Our agent could dream up the code, but it was like an architect who’d never seen the building site, it was completely blind to our actual infrastructure.

The breakthrough for us was finding a way to give the agent a pair of eyes. We used WSO2’s Choreo platform, whose Model Context Protocol (MCP) server acted as that crucial link, giving our AI a real-time “cheat sheet” to our services, APIs, and deployment targets.

From Idea to Local App

Like the original project, ours began with a clear idea. We used Gemini 2.5 Pro within VSCode to produce a concise Software Design Document (SDD) outlining the core components:

• An OCR step to extract menu items.
• An image generation step to visualize them.
• A simple web UI.

With that Software Design Document in hand, we handed it to a Claude Sonnet 4.0 agent in VS Code with a single, crucial instruction.

This prompt contains two key instructions. The first, “implement this app” is the straightforward task of code generation. The second part, however, is what sets this process apart and directly addresses the “painful slog” of productionizing a prototype: “using the Choreo Marketplace.”

By instructing the agent to use the marketplace as its source of truth for APIs, we preemptively solve the problem of hunting for endpoints, wrestling with deprecated docs, and manually managing secret keys. This is the bridge between intent and infrastructure that will be critical later on.

This automated correction is a key part of the process. Armed with the correct-by-definition API specs from the marketplace, the agent didn’t just write code, it wrote the right code. It then proceeded to translate the entire design document into a working local repository.

The agent delivered a complete repository with a React frontend and a Go backend. It included a detailed README.md with clear instructions for local setup, from the docker run commands to the required .env configuration. This provided all the basic plumbing to iterate quickly on a laptop. This brings you to that exhilarating ‘80% done’ feeling that Karpathy described, the fun part of the project.

Automating the “Painful Slog” of Integration

The initial prototype is the fun part where you feel 80% done. The remaining 20%, as Karpathy noted, is often a “painful slog” of wrestling with deprecated APIs and out-of-date docs. This is where our path diverged sharply from the original. For us, this entire step was automated because we instructed the agent to use the Choreo Marketplace.

An AI agent’s code is only as reliable as its knowledge, which can be months or years out of date. This is why it might generate calls to endpoints that no longer exist. The marketplace solves this by acting as a live, guaranteed source of truth.

It contains connectors for common public APIs like OpenAI, but critically, it also allows you to register your own internal or third-party services. We had already registered Replicate this way, making its API discoverable and standardized for our agent.

Instead of hallucinating API calls, the agent queried this marketplace. It received the up-to-date OpenAPI specs and direct references to secrets managed within Choreo, and used this correct-by-definition information to automatically wire both integrations into the Go backend. No more hunting for endpoints or pasting stale code snippets.

From Laptop to Live in One Go

With the code working locally, we arrived at the final headache: deployment. For this, we gave the agent a simple, conversational prompt: ”Can you deploy this on Choreo?” What followed was an intelligent, multi-step process orchestrated entirely by the agent and Choreo’s MCP server. It was like watching an expert DevOps engineer work at machine speed.

1. Analyze and Plan
First, the agent analyzed the entire project repository, the Go backend and the React frontend to understand the application’s structure and dependencies. This initial reconnaissance was crucial for formulating a deployment strategy.

2. Deploy the Backend

Based on its analysis, the agent generated a component.yaml manifest file defining a backend component. It then built and deployed the initial version of the backend service to Choreo. Getting the API live first is a critical step in a modern deployment workflow.

3. Provision and Publish the Database

Next, the agent requested a managed PostgreSQL database from Choreo. Once provisioned, it immediately published the new database as a service in the Choreo marketplace, making it a discoverable and connectable resource for other components.

4. Wire Up the Backend Connections

With the database and third-party services available in the marketplace, the agent updated the component.yaml manifest. It added “Choreo Connections” to formally link the backend to the database, OpenAI, and Replicate services, then redeployed the backend with this new, complete configuration.

5. Implement and Deploy the Frontend with Auth

The agent then turned its attention to the frontend. It scaffolded sign-in and sign-out buttons in the React UI and wired them to Choreo’s managed authentication endpoints. After generating the frontend component definition, it built and deployed the frontend.

6. Secure the Final Connection

With both frontend and backend live, the agent made the final crucial link- connecting the web app to the backend. This ensures all communication flows through Choreo’s secure gateway, protecting the API and managing user sessions.

7. Create Test Users

As a final touch, the agent used Choreo’s Secure Token Service (STS) to automatically create a handful of test users, allowing us to immediately sign in and test the end-to-end flow of the live application.

In just a few minutes and from a single conversational prompt, the process took the project from a local repository to a fully wired, secure, and live cloud architecture. Suddenly, the abstract components from the initial SDD became tangible, live services, fully interconnected and managed by the Choreo platform.

This entire system was automatically assembled and brought to life, culminating in the final, user-facing application.

What would normally be a multi-hour, error-prone manual task was reduced to an automated, repeatable workflow, orchestrated entirely by the agent.

Solving the Hard Problems Automatically

This automated flow highlights a fundamental shift. The innovation isn’t just that an AI can write code, it’s that a tight integration between an implementation agent and a deployment platform can automate the vast majority of platform engineering friction. This approach didn’t just save time, it elegantly solved the very problems that made the original MenuGen journey a “painful slog”:

• The Database Problem: What Karpathy called a source of “more suffering” was reduced to a single request. The agent asked for a database, and Choreo provisioned and connected it, enabling persistence and a robust processing queue.
• The Authentication Maze: Instead of manual OAuth configuration, the agent leveraged Choreo’s managed auth, injecting credentials at runtime so the same build works across any environment.
• The Secret Key Scavenger Hunt: The .env.local headache disappeared. All keys were stored in Choreo’s secure vault and injected automatically at runtime. The plumbing just worked.

This entire end-to-end process, from idea to a live, secure application, started with the same conversational prompt we used to generate the initial design document:

I want to create POC for a Menu Visualizer app as follows,
- Users photo a restaurant menu for upload
- Go (single file) backend extracts menu content with GPT-4o XML structured output and generates dish images using Replicate (polling) API
- PostgreSQL DB (with gorm) stores menu text and all image blobs
- React (plain css) app displays the results

Create a complete SDD.md for this app?

If you want to explore the final result or reproduce this workflow yourself, all the resources are available:

• Final Application Code: The complete source code for the MenuGen rebuild is available from,

GitHub - isala404/menugen

• Replicate OpenAPI Specification: To make the Replicate API discoverable in the Choreo Marketplace, we created a custom OpenAPI spec. You can find the replicate-openapi.yaml file in the project repository.
• Connecting the Dots (Choreo MCP Server): This is the secret sauce that makes the agent platform-aware. This article walks you through how to configure it.

Karpathy’s experience was a perfect diagnosis of a widespread problem: the operational complexity that smothers creative momentum. By giving the AI agent a reliable, auditable bridge to the underlying infrastructure, the developer stays focused on the product. You get to remain in that exhilarating “vibe coding” moment, while the repetitive, painful plumbing that turns prototypes into production systems is handled for you.

How We Used an AI Agent to Skip the DevOps Grind was originally published in Choreo Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Three weeks later, you’re debugging redirect loops at midnight, questioning your life choices, and wondering why something that’s supposed to be “standard” feels so fragile.

After years of working on identity solutions and helping teams implement OAuth2 in production SPAs, I can tell you that this frustration is universal. Here’s what those tutorials don’t tell you about OAuth2 in SPAs.

Over the next few sections, we’ll walk through the specific ways OAuth2 implementations break down in production: the grant type confusion that leads teams astray, the impossible token storage decisions, the refresh token nightmares that cause random logouts, and why even popular SDKs don’t actually solve these fundamental problems. Then we’ll explore a different architectural approach that sidesteps these issues entirely.

The Grant Type Maze That No One Explains

Every OAuth2 tutorial starts the same way: “First, choose your grant type!” followed by a neat little comparison table. What they don’t mention is that the “right” answer has changed multiple times, and half the information on the internet is outdated.

In my experience working with identity systems, I’ve seen the same grant type mistakes over and over, often discovered only during security audits or production incidents.

The Implicit Grant Legacy Problem

Most tutorials from 2018 to 2020 recommended the Implicit grant for SPAs, but the OAuth2 Security Best Current Practices RFC now considers it harmful. As of OAuth 2.1, the Implicit grant has been completely removed from the specification due to security vulnerabilities with tokens exposed in URL fragments. Despite this, outdated blogs and Stack Overflow answers still point developers down this dangerous path.

The Client Credentials Confusion

Here’s a common developer thought process: “My API is protected and requires a token to invoke it. Let me just use Client Credentials to get a token.” This seems logical at first glance, especially when you’re frustrated with OAuth2 complexity and just want something that works.

This is fundamentally wrong for SPAs, and I see it more often than you’d expect. Client Credentials is designed for server-to-server communication where you can securely store secrets. In a SPA:

• The client secret is exposed in the JavaScript bundle
• Anyone can extract it and impersonate your application
• You can’t differentiate between users, everyone has the same permissions

I’ve encountered production applications doing exactly this during security reviews. The conversation usually starts with “why is your client secret visible in your JavaScript bundle?” and ends with emergency remediation work.

Authorization Code + PKCE: The “Current” Recommendation

So you switch to the Authorization Code flow with PKCE because that’s what the security best practices recommend. But here’s the thing: these recommendations keep evolving, and what’s “secure” today might be deprecated tomorrow.

The Moving Target Problem: OAuth 2.0 originally recommended Implicit flow for SPAs. Tutorials, Stack Overflow answers, and official documentation all said it was the right choice for browser-based applications. Then security researchers found issues, and suddenly everyone’s telling you to use PKCE instead. Now OAuth 2.1 mandates PKCE for all clients and removes the Implicit flow entirely.

What’s next? Will there be OAuth 2.2 with new mandatory requirements? Will PKCE be replaced by something else? You’re not just implementing authentication, you’re constantly chasing evolving security standards, refactoring working code, and explaining to stakeholders why the “best practice” from six months ago is now considered harmful.

From an identity provider’s perspective, I’ve watched teams spend more time maintaining auth code than building the features their users actually want.

Token Storage: The Problem No One Wants to Talk About

The tutorials make token storage look simple: “Just store the token!” But each browser storage option comes with security trade-offs and implementation headaches.

Cookies vs. HTML5 Storage: Pick Your Security Risk

Cookies are familiar but vulnerable to both XSS and CSRF attacks. Regular cookies can be accessed by any JavaScript on your page. HttpOnly cookies solve the XSS problem but create a new one: your SPA can’t access them via JavaScript, so you need to build and maintain your own backend proxy for every API call.

localStorage and sessionStorage seem like obvious choices for SPAs. They’re easy to use and don’t get sent with every request. But they can be accessed via JavaScript, making them vulnerable to XSS attacks. Any script running on your page can read localStorage.getItem(‘access_token’). This includes third-party analytics scripts, compromised CDN resources, malicious browser extensions, and any XSS vulnerability in your application.

Browser Memory vs. Web Workers: UX vs. Complexity

Storing tokens only in JavaScript memory sounds secure: no persistence, no cross-tab access, harder for attackers to reach. But the user experience is terrible. You lose your session every time you refresh the page, navigate to another site and come back, or experience any JavaScript error that reloads the page.

Web Workers run in a separate context, making tokens inaccessible to the main thread. But they create debugging complexity, multi-tab synchronization issues, and still don’t solve persistence problems.

Every storage option forces you to choose between exposing tokens to XSS attacks, losing user sessions on page refresh, or building complex authentication infrastructure that you’ll spend months debugging.

Token Refresh: Where Everything Falls Apart

Token refresh is where most OAuth2 implementations completely break down in production. The tutorials show the happy path, “just use the refresh token to get a new access token”, but production is full of edge cases.

The Multi-tab Synchronization Nightmare

Modern OAuth security best practices require refresh token rotation, issuing a new refresh token each time an access token is refreshed and immediately invalidating the previous one. Each browser tab runs independently, but they all start with the same refresh token. When one tab refreshes and gets new tokens, the other tabs are left holding invalidated refresh tokens. Users get randomly logged out of some tabs but not others.

The Concurrent Request Problem

Here’s a scenario I’ve debugged multiple times: A dashboard makes multiple API calls on page load. When tokens expire, all calls fail simultaneously and trigger concurrent refresh attempts. The first refresh succeeds and invalidates the refresh token. The remaining attempts fail, causing the auth server to revoke all tokens. Users get logged out unexpectedly.

The race condition questions multiply: Do you trigger multiple refresh attempts? Queue the failed requests? How do you prevent the second refresh request from using an already-invalidated token? Most teams don’t handle this correctly.

Token refresh turns from a simple “get new token” operation into a complex distributed systems problem with race conditions and state synchronization issues.

The SDK Trap: Hiding Complexity, Not Solving It

At this point, many developers turn to OAuth2 SDKs. “Don’t reinvent the wheel”, so you pick a popular library and dive into the configuration.

The problem with SDKs isn’t that they’re hard to use, it’s that they make you think you’ve solved OAuth2, when really you’ve just hidden the complexity behind an API.

The False Security of Abstraction

SDKs abstract away the implementation details, but all those OAuth2 problems we just discussed? They’re still there:

Token storage dilemma: The default storage method may vary, and you have to configure it correctly to balance between UX and security, you’re still vulnerable to XSS attacks, you just don’t realize it. The security risk doesn’t disappear because it’s hidden behind auth.getToken().

Token refresh nightmares: The SDK handles refresh automatically, until it doesn’t work across multiple tabs and users get randomly logged out. Now you’re debugging someone else’s refresh logic instead of your own.

Configuration complexity: Your auth config file still grows into a monster with dozens of options, and the documentation assumes you already know which ones you need. You still need to become an OAuth2 expert to configure the SDK correctly.

SDK updates break things regularly. I’ve seen teams stuck with libraries they picked six months ago that now require refactoring half their auth code to upgrade. You think you’ve solved OAuth2, but you’ve just moved the problems into a black box that breaks in production.

The Real-World Consequences

But the frustration goes beyond just technical complexity. These OAuth2 implementation challenges create real business impact:

Security incidents and compliance violations: Token leakage through browser storage, session hijacking, and privilege escalation require incident response and expensive remediation. Auditors flag tokens in browser storage as security risks, triggering months of compliance work.

Development velocity suffers: Teams spend 30–40% of their sprint capacity on auth-related fixes instead of building features. Product roadmaps slip because you’re constantly firefighting authentication issues instead of delivering value.

The cost isn’t just technical debt, it’s real business impact that compounds over time.

The Fundamental Problem: Tokens Don’t Belong in Browsers

Looking at all these problems together reveals a pattern: we kept trying to solve the wrong problem.

We keep trying to make browser-based token management secure, but browsers are fundamentally insecure environments. Think about it — you’re putting sensitive credentials in a runtime that executes arbitrary JavaScript from multiple sources, where any XSS vulnerability or malicious script can access your tokens. Every “solution” just shuffles the same security risks around while creating new problems.

Store tokens securely? Users lose sessions on page refresh. Make the UX smooth? You’re back to XSS vulnerabilities. Use complex SDKs? Now you’re debugging someone else’s authentication logic. Roll your own? You’ve just signed up to become an OAuth2 expert and maintain complex security code forever.

The core insight:

“Tokens don’t need to exist in browser JavaScript at all”

The industry is quietly moving toward Backend-for-Frontend (BFF) architectures for exactly these reasons. Your SPA makes API calls through a BFF layer that handles OAuth complexity and securely forwards requests to your APIs. The browser never sees an access token, refresh token, or any OAuth complexity.

This isn’t a new pattern — it’s how server-rendered applications have worked forever. We just got distracted by the “single page” part and forgot that SPAs can still have backends. Instead of browser-based token management, the BFF handles OAuth2 flows, manages tokens securely, and proxies API requests. The browser only needs to make normal HTTP requests.

A Different Approach: Platform-Managed Authentication

This is exactly why we built managed authentication into Choreo. After watching countless teams struggle with OAuth2 complexity, the pattern became clear — developers kept saying the same thing: “We just want authentication that works reliably without becoming OAuth2 experts.”

Rather than asking teams to become OAuth2 experts or build their own BFF layers, we implemented the BFF pattern at the platform level. This eliminates the burden of implementing OAuth/OIDC requirements and removes the need for applications to manage tokens issued by identity providers.

Enable Authentication with One Toggle

Teams enable Choreo’s managed authentication and follow clear guides on which grant types to enable, without having to manage tokens, handle refresh logic, or deal with the complex state synchronization issues we discussed earlier. Teams can easily connect with any OIDC-compliant identity provider directly from the Choreo console.

Add a Login Button That’s Surprisingly Simple

To allow Choreo to manage sign-in functionality, teams implement a simple authentication flow:

// Sign in
<button onClick={() => window.location.href = '/auth/login'}>
  Sign In
</button>

// Sign out
<button onClick={() => window.location.href = '/auth/logout'}>
  Sign Out
</button>

// Check if logged in
fetch('/auth/userinfo')
  .then(response => response.ok ? response.json() : null)
  .then(user => console.log(user?.name || 'Not logged in'));

API Calls Without Token Management

For APIs deployed in Choreo, invocations work relative to the app URL. APIs are invoked via <app-url>/choreo-apis/<api-path>. The authenticated session details are used during API invocation to enforce access control automatically.

No manual token attachment, no refresh logic, no concurrent request handling:

// Just call APIs normally
fetch('/choreo-apis/orders')
  .then(response => response.json())
  .then(data => setOrders(data));

Stop Fighting OAuth2, Start Shipping Features

Your users don’t care about OAuth2 implementation details. They want your application to work reliably. Every hour you spend debugging token refresh logic is time you’re not spending on features that matter to your business.

When you’re staring at browser network tabs at 2 AM wondering why tokens work in Chrome but fail in Safari, or explaining to stakeholders why the security best practice from six months ago is now considered harmful, remember: this problem is already solved.

Next time you’re planning authentication for a production SPA, ask yourself: Do you want to spend the next month becoming an OAuth2 expert, or do you want to ship features your users actually care about?

Ready to skip the OAuth2 complexity? Try Choreo’s managed authentication — it’s free to get started, and your future self will thank you when you’re shipping features instead of debugging token refresh race conditions.

OAuth2 doesn’t have to be the hardest part of your SPA development. Sometimes the best code is the code you don’t have to write.

Why OAuth2 for SPAs Is Harder Than the Tutorials Make It Look was originally published in Choreo Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

AI agents are quickly becoming indispensable in modern software development. From generating code and managing builds to handling deployments and debugging, these agents streamline day-to-day engineering tasks. However, for AI agents to truly operate effectively in real-world environments, they need access to context — structured, dynamic, and actionable information about your code, tools, and systems.This is where the Model Context Protocol (MCP) comes in.

What is MCP?

Model Context Protocol (MCP) is a standard for exposing structured context to language model agents. MCP acts as a bridge between your tools (like IDEs, CLIs, or APIs) and the AI agents interacting with them. It allows models to understand and act on the current state of your projects, environments, and components through a unified protocol.

Why MCP Matters for Choreo

Choreo, an internal developer platform for building, deploying, and managing cloud-native applications, now supports MCP via the Choreo CLI. This opens the door for powerful AI-assisted operations — right from your code editor (VS Code, Cursor, etc.).

With Choreo’s MCP integration, AI agents can:

• Understand the current structure of your Choreo projects
• Create or modify components
• Trigger builds and deployments
• Retrieve and analyze logs
• All through natural language interactions

Getting Started

1. Install Choreo CLI

For Linux and Mac OS

curl -o- https://cli.choreo.dev/install.sh | bash

For Windows(via PowerShell)

iwr https://cli.choreo.dev/install.ps1 -useb | iex

After installing you can verify your installation by running choreo -v

2. Follow the steps below to use Choreo MCP with your preferred code editor

• VS Code

"mcp": {
       "inputs": [],
       "servers": {
           "mcp-choreo": {
               "command": "choreo",
               "args": [
                   "start-mcp-server"
               ],
               "env": {}
           }
       }
   },

• Cursor

{
 "mcpServers": {
   "choreo": {
     "type": "stdio",
     "command": "choreo",
     "args": [
         "start-mcp-server"
     ]
 }
}

3. The Choreo MCP server does not implement authentication via chat interfaces. Therefore, users must login to choreo using the Choreo CLI using the following command.

choreo login

This authenticates the CLI and establishes the context for AI-driven operations via MCP.

Choreo MCP in Action: How I vibe deployed a service in Choreo

With the recent introduction of MCP support for the Choreo CLI, I decided to explore whether I could build and deploy a simple backend service entirely through prompt-driven development — no manual code writing, no clicking through UIs. Just prompts.

The Plan

I wanted to create a minimal Express.js service to manage a collection of books. The requirements were simple:

A REST API with the following endpoints:

• POST /books — Add a new book
• GET /books — Retrieve all books
• GET /books/{id} — Retrieve a book by ID

Each book should have:

• name
• author
• An auto-generated id

Prompt-Driven Development

So I kicked things off with a simple prompt:

Create an Express.js service which does the following: "Book manager"
 It has the following endpoints:
 POST /books – add a book to the bookstore
 GET /books – get all the books
 GET /books/{id} – get specific book by id
 The book object contains: name, author. An id will be automatically created when adding the book.

With the help of GitHub Copilot and some quick API testing via prompts like:

Start the server
Test the API

I was able to verify that the service worked as expected.

Next, I moved to deployment. I used the following prompt to initialize and push my code to GitHub:

Create a GitHub repository "choreo-mcp-demo" and push this service to the repo

Then, I created a Choreo component using the CLI with this prompt:

Create the book-manager component in Choreo
Endpoint type: REST

Create a build from latest commit

Check the build status

Get the deployment status of the component

I tested the API using following prompts

Invoke the /books endpoint

Generate a test key for the development environment

Create a curl command to invoke the api with the token

In just a few steps — all prompt-driven — the service was deployed and live on Choreo. No UI interaction needed, just pure prompts.

This experiment demonstrated how vibe coding — building entirely through AI-assisted prompts combined with Choreo’s new MCP capabilities can deliver a seamless developer experience from ideation to deployment. It’s fast, efficient, and surprisingly empowering.

Whether you’re a solo developer prototyping an idea or part of a team delivering production services, Choreo MCP makes the deployment pipeline effortless.

Ready to try it yourself? Try out into Choreo today, and explore how Choreo MCP can revolutionize your development and deployment workflow. Check out the official documentation to get started, and see just how far you can go with only a few well-crafted prompts.

Enhancing Developer Workflows with MCP Support in Choreo CLI was originally published in Choreo Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

In today’s world, AI is no longer a buzzword — it’s an integral part of how modern apps are built. From chatbots to personalized recommendations, AI is quietly transforming user experiences. As a developer, learning how to blend AI with app development is not just exciting — it’s essential.

In this blog, we’ll explore QuickEats, a simple AI-Integrated food ordering app. If you already have a full-stack app running in production, this will give you practical ideas on how to seamlessly integrate AI into your existing architecture. Whether you’re a student, a junior developer, or a curious learner, this app will spark ideas on integrating AI into real-world scenarios and building intelligent applications from end to end. By the end of this blog, you’ll have deployed it on Choreo, seeing how it simplifies taking your code to production in 5 minutes!

QuickEats

QuickEats is a food ordering app which lets users browse a curated menu, add items to their cart, and place orders through a simple checkout flow. To make the experience smarter and more interactive, we’ve integrated AI in key parts of the user journey. Here are the three powerful AI features built into this app:

1. Food Assistant Chatbot

This intelligent chatbot can answer questions about the menu, suggest dishes based on your cravings, or even help you customize your order with natural, conversational interactions.

2. Smart Order Summary Generator
To make your experience unique, once your order is placed, this feature leverages GPT to craft a fun, personalized message that intelligently summarizes your order details, providing a delightful and accurate confirmation.

3. Personalized Food Recommendations
Based on previous orders or food preferences, it gives tailored dish suggestions that you’re highly likely to love, ensuring you always find something satisfying.

Tech Stack

• Next.js — For the frontend, we opted for Next.js, a powerful React framework. Beyond speed, it provides an exceptional developer experience with features like API routes, routing, and image optimization built-in, making it a robust choice for scalable applications.
• OpenAI GPT — This is the core of QuickEats’ AI capabilities (Chatbot, smart order summary and recommendation). GPT models excel at understanding complex human language and generating coherent, contextually relevant, and even creative text.
• Choreo — An AI-native Internal Developer Platform (IDP) which simplifies a developer’s journey from code to production. You can connect your Git repo, build, deploy (across multiple environments), manage and monitor cloud-native applications in real time (with AI guiding you every step of the way).

AI Features in QuickEats

All 3 AI features in QuickEats — food assistant chatbot, smart order summary generator and personalized food recommendations — are implemented using simple prompt engineering. We use simple, effective prompts that include relevant context, tone, constraints (like word count), and user data.

Here’s how we implement Personalized Food Recommendations using simple prompting:

const previous = lastOrder?.map((item) => item.name).join(", ") || "no previous orders";
const menuList = menuItems
  .map((item) => `${item.name}: ${item.description}`)
  .join("\n");

const prompt = `
You are a food assistant for an app. Based on this past order: ${previous}, suggest 1–2 new items from the menu below.

Menu:
${menuList}

Respond with a short and friendly message under 30 words.
`;

This prompt is passed into the OpenAI API like this:

const chatResponse = await openai.chat.completions.create({
  messages: [
    {
      role: "system",
      content: "You are a helpful food recommendation assistant.",
    },
    {
      role: "user",
      content: prompt,
    },
  ],
  model: "gpt-4.1",
});

The model uses the provided context to generate short, friendly suggestions based on past user preferences. Similar approaches are used for generating natural-sounding order summaries and answering menu-related questions via the chatbot.

While this demo uses simple prompting, there’s a lot of room to improve. For production-grade applications, adopting self-hosted or fine-tuned language models offers greater control, improved performance, and compliance with internal data governance policies. Beyond basic prompting, mature AI implementations typically rely on techniques such as like few-shot prompting, retrieval-augmented generation (RAG), and structured output formatting to ensure responses are context-aware, accurate, and aligned with business logic. Frameworks like LangChain or Semantic Kernel can orchestrate complex, multi-step workflows, while vector databases (e.g., Pinecone, Weaviate, or Chroma) enhance personalization by enabling semantic search over user behavior or domain-specific knowledge. Together, these components form the foundation for building scalable, intelligent systems that go far beyond simple API calls — making AI a core, reliable part of production software.

Source Code

If you’re curious about how this demo app is built, feel free to dive into the code. https://github.com/LakshanSS/quickeats

Ship to Production in 5 Minutes with Choreo

You can take this app from code to live production in under 5 minutes using Choreo. No manual infrastructure setup, no complex CI/CD — just connect your GitHub repo, and Choreo takes care of the rest. It’s the fastest way to launch, scale, and show off QuickEats in production!

1. Login to Choreo.
2. Create a new Project and add a Component (Web Application).

3. Connect the GitHub repo and choose:

• Build Preset: NodeJS
• Language Version: 20.x.x
• Port: 8080

4. Add Environment Variable:

• Go to DevOps → Configs & Secrets
• Add: OPENAI_API_KEY -> your actual key (mark as secret)

5. Trigger a Build and ensure it completes successfully.

6. Deploy to Development Environment, test it using the Web App URL.

7. Promote to Production Environment. You can also optionally create a short URL for your Web App.

The Road Ahead: Scaling QuickEats for Real-World Use with Choreo

This is a very simple AI-Integrated app. To make it production-ready and scalable for a real-world scenario, we need to move beyond this simple demo. This involves enhancing the architecture, data management and AI strategy. The good news? Choreo is perfectly suited to facilitate this evolution.

1. Dedicated backend services

Break down the application into dedicated microservices — for order processing, chatbot interactions, recommendations, payments, and delivery tracking etc. Choreo makes it easy to develop, deploy, observe, and manage these service components independently. This modular design improves scalability, maintainability, and team collaboration.

2. Database Integration

Any real-world food ordering app needs a reliable data layer. Choreo simplifies database integration by offering native support for connecting to managed cloud databases or external data sources. Whether you’re storing orders, user profiles, or menu metadata, it’s easy to plug in a robust backend.

3. Automate Repetitive Backend Workflows

In a real-world food ordering app like QuickEats, there are several repetitive backend operations such as sending daily order summaries to restaurant partners, purging abandoned carts overnight to keep the system clean, syncing menu data from external vendors each morning, or updating popular item rankings based on order trends. Choreo’s Scheduled Tasks make it easy to automate routine backend operations by running code at defined intervals.

4. Advanced AI Strategies

For production-grade AI features, especially in enterprise environments, relying solely on external APIs may not always be ideal due to latency, cost, or data privacy concerns. Instead, you can deploy self-hosted or fine-tuned models. You can modularize AI features — such as recommendation engines, dynamic menu generators, or feedback summarizers — into standalone services in Choreo. Techniques like few-shot prompting, retrieval-augmented generation (RAG), function calling, and structured output formatting allow your app to generate more accurate, personalized, and context-aware responses. Tools like LangChain can be used to orchestrate multi-step AI workflows, while vector databases enable semantic search and contextual retrieval for more intelligent, data-driven interactions.

5. Authentication and Authorization
Secure access is non-negotiable for real-world apps. Choreo provides built-in support for OAuth2, JWT, and third-party identity providers, making it easy to protect APIs and restrict access based on user roles. You can enable user authentication and enforce fine-grained authorization policies with minimal configuration.

6. Event-Driven Architecture
In a food ordering app like QuickEats, features such as real-time order status updates, delivery notifications, and automated actions triggered by events like payment completion are essential for delivering a responsive user experience. Choreo makes it easy to build these event-driven workflows by supporting asynchronous communication between services using Webhooks, Kafka, or other event sources.

7. Auto Scaling and Resource Optimization
During peak hours like lunch or dinner time, your app may experience sudden traffic spikes. Choreo’s built-in auto-scaling ensures your services adjust dynamically based on real-time load. You don’t need to manually manage replicas or worry about over-provisioning; Choreo intelligently scales resources up or down to maintain high availability and optimize cost.

These are just a few examples of how you can scale and evolve QuickEats into a production-grade AI-powered app. With Choreo’s comprehensive platform, the possibilities are endless! — it supports you every step of the way!

What’s Next?

I encourage you to fork the GitHub Repo and experiment with QuickEats yourself. Deploy it in Choreo, try out the AI features, and then challenge yourself to implement some of the production-ready improvements we discussed.

To dive deeper, explore the comprehensive Choreo Documentation. If you encounter any issues or have questions, don’t hesitate to join the Choreo Discord channel. It’s a great place to get support, discuss ideas, and connect with the community as you continue your development journey!

Building a simple AI-Integrated Food Ordering App with Next.js, GPT and Choreo was originally published in Choreo Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

A few months ago, I built a simple backend service in WSO2 Choreo to scrape and return the daily USD to LKR exchange rate from a local bank’s website. The goal? Pure convenience. I was tired of clicking through layers of menus just to find the rate.

I deployed the service on Choreo and started using it myself. A few co-workers got wind of it and began using it too. Everything was chill — until one of them pinged me:

“Hey, this is awesome, but can we see the past few days’ trends as well? It’s hard to get context from just today’s rate, especially with all the fluctuations lately.”

That hit a nerve. Around the same time, the exchange rate was bouncing due to new tariff talks, and suddenly this weekend project felt… valuable.

I decided to evolve it — into a full-fledged, full-stack dashboard that not only fetches today’s exchange rate but also displays historical data in a sleek UI. I wanted this to be a complete product. And most importantly — I wanted it all live, in minutes, not hours.

The Stack: Keep It Simple, Keep It Clean

This is what I went with:

• Frontend: React + Chart.js (Hosted as a Web App in Choreo)
• Backend: Go service exposing an API to fetch data from MongoDB (Hosted as a Service in Choreo)
• Job Service: Go job scheduled to fetch rates from the bank’s site periodically (Runs as a Scheduled Job in Choreo)
• Database: MongoDB Atlas (external, not hosted within Choreo)

The frontend, backend, and job are all deployed as separate components in Choreo, communicating securely via environment-based connections.

Why Choreo?

I could’ve gone with Vercel for frontend and maybe Fly.io or AWS Lambda for backend and cron — but I didn’t want to:

• Glue services together manually.
• Set up CI/CD pipelines for each part.
• Think about API Gateways, Secrets, Scaling.

Instead, with Choreo:

• No platform concerns — I never touched Kubernetes or Docker.
• Secure connections — My services talk to each other without writing a line of auth/config code.
• Secrets management — Database credentials, API keys, all handled with a click.
• Dev → Prod — Environments supported out-of-the-box.
• Monitoring & Logs — Without any extra setup.

The killer feature? Connecting the frontend to the backend was just a matter of setting a connection from the UI. Done.

Architecture

The 21-Minute Breakdown

🧠 7 Minutes: Ideation

• Got the idea to visualize exchange rate trends.
• Decided to go full-stack.
• Sketched component boundaries: job → DB → API → frontend.

👨‍💻 7 Minutes: Coding (with a little help from ChatGPT)

• Wrote the cron job in Go to fetch exchange rate.
• Built backend API in Go to serve the stored data.
• Created a minimal React UI with a line chart.

🚀 7 Minutes: Deployment with Choreo

• Pushed code to GitHub.
• Created 3 components in Choreo (webapp, service, job).
• Set up secrets and environment variables.
• Defined a cron schedule for the job.
• Connected components via the UI.
• Hit Deploy. Everything just worked.

Code Sample: Fetching the Rate

req, _ := http.NewRequest("GET", "https://www.sampath.lk/api/exchange-rates", nil)
req.Header.Set("User-Agent", "My Custom Agent")
resp, _ := http.DefaultClient.Do(req)
// Parse USD rate, insert into MongoDB

Code Sample: Frontend Fetch

const res = await axios.get<ExchangeRate>(`${CHOREO_CONNECTION_URL}/latest-rate`);

Full Code

👉 GitHub Repo: github.com/chameerar/usd-lkr-exchange-rate-dashboard

Final Thoughts

This project reminded me how much productivity skyrockets when the platform doesn’t get in the way. With Choreo, I didn’t need to fight the cloud to bring an idea to life.

Whether you’re a solo dev hacking on a weekend project or a team evaluating microservice-friendly platforms — give Choreo a shot. It’s the platform that lets you forget about the platform.

✍️ Built with Choreo. Inspired by curiosity. Powered by caffeine.

References

• You Code, We Orchestrate
• Atlas Database

How I Created and Deployed an Exchange Rate Dashboard App in 21 Minutes with Choreo was originally published in Choreo Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Large Language Models have turned the technology world upside down in the past two years and continue to disrupt the technology landscape. What ChatGPT started has been supported by other technology giants with the creation of LLMs like Claude, Gemini, and Llama. Both tech-savvy individuals and those less familiar with technology have integrated LLMs into their daily lives.

One caveat of LLMs can be said as not have enough context about our own data or custom information. LLMs are trained on a specific data set and able to answer almost all the questions we ask using their language skills. But if we want to ask it about a specific custom piece of data, which is outside of its knowledge base, it would struggle and hallucinate. RAG or Retrieval-Augmented Generation, is a type of application that is designed to overcome this challenge. This combines a custom knowledge-base of our own and a LLMs language ability to refine the output of the LLM and make it answer queries about custom/specific scenarios more accurately.

A birds-eye view of a RAG system can be shown below.

In simple terms, when a user asks a query, a RAG application queries the connected knowledge base for relevant context and bundles that context with the user's query, and sends it to the LLM. LLM then processes the data and provides an appropriate answer.

Steps of Creating a RAG Application

We can divide the creation of a simple RAG application into 3 parts.

1. Data Collection & Preparation
2. Embedding and indexing data
3. Retrieving indexed data and quarrying the LLM.

Let's look at the 3 steps in detail below.

Data Collection and Preparation

In this step, all relevant data must be collected and pre-processed if necessary. These data can be PDFs, Markdown files, Text files and even Plain Text.

Embedding and indexing data

In this step, we send the collected data to, what we call an “Embedding Model” and it returns a vector representation, a numerical model, of the provided data. Those “vector embeddings” are then stored in a “Vector Database”. Vector database is a database that stores mathematical representations of structured and unstructured data.

Retrieving data embeddings and Quarrying the LLM

When a prompt is provided, the vector database is queried to find relevant data context to the prompt, and the found context is sent along with the prompt to a LLM, which in turn takes the prompt and the provided data into account and responds with a relevant answer.

As the title already says, in this article let's see how we can create a RAG application to ask questions about a PDF file we provide. In a sense, we could “chat” with our PDF file by utilizing this application.

To create the RAG application we use Langchain, which is a popular Python framework for creating RAG applications. As the underlying models, we are utilizing OpenAIs GPT models and embedding models. As for the knowledge base, we are using “Pinecorne” which is a popular vector database that you can try online for free. Even after all these, we would need to expose our application to the internet for it to be useful, for this, we use WSO2’s state-of-the-art integration platform as a service, Choreo. We would deploy our Python/Langchain application on Choreo and expose it as an API anyone can invoke.

Overall our tech-stack would look like the following.

1. Langchain — For creating the RAG application
2. OpenAI Embedding Model — Embedding data
3. Pinecorne — Storing vector embeddings
4. OpenAI GPT — LLM
5. WSO2 Choreo — Building, Deploying, and Exposing our application to the public.

Simple RAG Architecture

Let's get coding!

Before we start coding our application, you would need to create three accounts.

1. Choreo Account — able to build and deploy your application
2. OpenAI Platform Account — allows you to access OpenAI APIs
3. Pinecone Account — allows you to access the Pinecone serverless vector database.

Then you need to get two API keys for OpenAPI and Pinecone

1. OpenAI — Go to https://platform.openai.com/api-keys and create a new secret key. Copy and save it in a safe location.
2. Pinecone — Left Side Pane → Manage → API Keys → Create API Key. Copy and save it.

I am using Python 3.11 and PyCharm IDE to develop this project.

First, you need to create a new project in PyCharm ( or your preferred IDE ) and add the following requirements.txt .

langchain~=0.1.12
langchain-community
pinecone-client
langchain-openai
openai
langchain_pinecone
uvicorn~=0.30.5
fastapi~=0.112.0
pydantic~=1.10.17
pypdf

Then add your Pinecone API Key and OpenAI API key to a .env file. Make sure NEVER to push this file to git!

OPENAI_API_KEY=""
PINECONE_API_KEY=""

After that copy this PDF I’ve exacted from NASA [1], about dark matter, to your directory. After this, your directory would look like this

.
├── .env
├── dark-energy.pdf
└── requirements.txt

If you refer to our first step of creating a RAG application, the data collection (i.e. the pdf) and the setting up of the project is done. Next, we have to create the embeddings and store them in Pinecone. Let's create a file embed.py for this purpose.

import os
from langchain_community.document_loaders import PyPDFLoader
from langchain_openai import OpenAIEmbeddings
from langchain.text_splitter import CharacterTextSplitter
from langchain_pinecone import PineconeVectorStore
from pinecone import Pinecone, ServerlessSpec

OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
PINECONE_API_KEY = os.getenv("PINECONE_API_KEY")

def create_or_update_index(texts, index_name, embedding):
    pc = Pinecone(api_key=os.environ.get("PINECONE_API_KEY"))
    if index_name not in pc.list_indexes().names():
        pc.create_index(
            name=index_name,
            metric='cosine',
            dimension=1536,
            spec=ServerlessSpec(
                cloud='aws',
                region='us-east-1'
            )
        )
        print(f"Created new index: {index_name}")
        vectorstore = PineconeVectorStore.from_documents(
            texts,
            embedding=embedding,
            index_name=index_name
        )
    else:
        print(f"Index {index_name} already exists")
        vectorstore = PineconeVectorStore.from_existing_index(
            index_name=index_name,
            embedding=embedding
        )

    return vectorstore

pdf_path = "dark-energy.pdf"
loader = PyPDFLoader(pdf_path)
documents = loader.load()

text_splitter = CharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=200
)

texts = text_splitter.split_documents(documents)

embedding = OpenAIEmbeddings(model="text-embedding-ada-002")
index_name = "pdf-index"

vectorstore = create_or_update_index(texts, index_name, embedding)

Here we utilize the PyPDFLoader from langchain_community library to load our stored PDF. Then we need to break down our large PDF into small chunks of text, for this purpose we can use CharacterTextSplitter which specifies a chunk_size (maximum size of a chunk) and a chunk_overlap (amount of overlap between two chunks). The following web application “ChunkViz” can visualize the chunking and overlapping of chunks in a text neatly.

The dark green parts are overlapping portions of the chunks.

After defining the TextSplitter we can input the document to the text splitter and get chunked text.

texts = text_splitter.split_documents(documents)

After that, next step is the embedding process. We need to send the chunked texts into an embedding model and get vector embeddings for those in return. In this case, we are using the text-embedding-ada-002 model of OpenAI.

embedding = OpenAIEmbeddings(model="text-embedding-ada-002")

After getting the embeddings we can create a new index in Pinecone vector db and store our embeddings using create_or_update_index the method. This method uses pc.create_index the method of Pinecone Python library to send the embeddings to a new index.

After creating the embed.py file, install the dependencies by running,

pip install -r requirements.txt

Then you can run the embed.py .

python3 embed.py

This would take some time and if all goes well, you will see the following message.

Created new index: pdf-index

And now If you go to the Pinecone UI, you’ll see a new index has been created!

Now, we have to create an API that would allow a user to ask questions about the contents of the PDF. For the creation of the API, we will use FastAPI which is a powerful web framework for Python.

Create the following api.py file.

import os
from fastapi import FastAPI
from pydantic import BaseModel
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_pinecone import PineconeVectorStore
from langchain.chains import RetrievalQA
from langchain.prompts import PromptTemplate
from pinecone import Pinecone

app = FastAPI()

OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
PINECONE_API_KEY = os.getenv("PINECONE_API_KEY")

# Initialize Pinecone
pc = Pinecone(api_key=PINECONE_API_KEY)

embedding = OpenAIEmbeddings(model="text-embedding-ada-002")
index_name = "pdf-index"

vectorstore = PineconeVectorStore.from_existing_index(
    index_name=index_name,
    embedding=embedding
)

# Initialize ChatOpenAI
llm = ChatOpenAI(
    openai_api_key=OPENAI_API_KEY,
    model_name='gpt-4o',
    temperature=0.0
)

# Creating Prompt
prompt_template = """Use the following pieces of context to answer the question at the end. 
If you don't know the answer, just say that you don't know, don't try to make up an answer.

{context}

Question: {question}
Provide a concise answer in 1-4 sentences:"""

PROMPT = PromptTemplate(
    template=prompt_template, input_variables=["context", "question"]
)

# Creating a Langchain QA chain
qa = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=vectorstore.as_retriever(),
    chain_type_kwargs={"prompt": PROMPT}
)

class Query(BaseModel):
    question: str
   
# Create API Endpoint
@app.post("/ask")
async def ask_question(query: Query):
    response = qa.invoke(query.question)
    return {"answer": response['result']}

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8080)

First, we will do the necessary initialization like setting up API Keys, Pinecone connection, and setting up a LLM connection using OpenAI’s gpt-4o model.

After that, we create a prompt-template which would be sent to the LLM. Here you can see two replaceable parts, {context} and {question} , context is the data we receive from Pinecone regarding the users question. This template would be populated by the original question and the relevant context received from Pinecone and then it would be sent to the LLM.

Then we need to create a QA (Question Answering) chain using Langchain and provide the created variables. This QA chain will answer questions based on the provided context, using the LLM.

Now, we can invoke this qa object inside a HTTP resource, /ask to finalize our application.

Deploying the RAG Application

Now that we have finished developing our application. We need to deploy it. For that, we will use Choreo, a Kubernetes-based internal developer platform as a service by WSO2. We can deploy this application as a Python service without having to create any Dockerfiles, images, or Kubernetes configurations, all underlying tinkering would be managed by Choreo.

First you need to create a Procfile which will instruct how to start the application to a Buildpack.

web: uvicorn api:app --host=0.0.0.0 --port=${PORT:-8080}

Then create a component-config.yaml file in a .choreo directory. This is a Choreo-specific config that will configure the endpoint which is exposed to the internet.

apiVersion: core.choreo.dev/v1beta1
kind: ComponentConfig
spec:
  inbound:
    - name: RAG API
      port: 8080
      type: REST
      networkVisibility: Public

This would be the final directory structure.

.
├── .choreo
│   └── component-config.yaml
├── .env
├── .gitignore
├── Procfile
├── api.py
├── dark-energy.pdf
├── embed.py
└── requirements.txt

Now push the code to a GitHub repository. You can find the full application at https://github.com/rashm1n/RAG101

Now you are all set to deploy your application. Create a Python service in Choreo following the below steps.

1. Create a choreo.dev account if you don't have one already.
2. Create a project → Create a new Service and enter the following details.

• Component Display Name — RAGApp
• Public Repository URL — <Your Git repository URL>
• Buildpack — Python
• Python Project Directory — /
• Language Version — 3.11.x

And build the service component.

Now navigate to the deploy menu and click Configure & Deploy. Create two Environment Variables (Mark as Secret) and add your API Keys. Choreo will securely store these as Kubernetes Secrets.

Then create next and Deploy the component. The deployed application would look like the following.

Let's navigate to the Choreo Test Console and acquire a Test API Key to invoke our application. Since Choreo applications are protected by OAuth by default, you will need this API Key to test out the API. Copy the invoke-URL as well.

Then use the API-Key and Invoke URL as below in a curl and invoke your component.

curl --location 'https://5d1fa89f-bb68-4cee-a1c9-5ca459e02768-dev.e1-us-cdp-2.choreoapis.dev/springproject/ragapp/v1.0/ask' \
--header 'accept: */*' \
--header 'API-Key: <API-Key>'
--header 'Content-Type: application/json' \
--data '{"question": "What is dark energy?"}'

And you will get a response according to your provided data! Within a few minutes, you are able to go from code to cloud and invoke your application over the internet.

In this article, we discussed about creating and deploying a RAG application which would give the function of asking questions about a PDF document of your choice. This is a very simle example of a RAG application and LLMs but you can make much more complicated applications utilizing these, and use Choreo to deploy them securely.

Please comment below if you have any questions. See you in another article.

Resources

[1] — https://science.nasa.gov/universe/the-universe-is-expanding-faster-these-days-and-dark-energy-is-responsible-so-what-is-dark-energy/

[2] — https://choreo.dev

[3] — https://python.langchain.com/v0.2/docs/introduction/

[4] — https://platform.openai.com/docs/concepts

Chat With Your PDFs — Deploying a Simple RAG Application with Langchain, OpenAI and Choreo was originally published in Choreo Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this article, we will walk you through how we can effectively use Choreo to deploy a service written using Flask. We will be using this sample designed to showcase a restaurant reservation system. This API offers users the ability to effectively manage restaurant reservations. It supports various operations like creating, retrieving, updating, and deleting reservation entries.

Pre requisites

1. A Choreo user account
2. A Bitbucket account
3. Import this repository to create a new repository in your Bitbucket Workspace.

Let’s get started with connecting your source code in Bitbucket to Choreo.

Step 1 — Create a common credential in Choreo

1. Login to your Bitbucket account and navigate to Personal Bitbucket Settings under the User Settings section. From the left menu , select App passwords in the Access Management section. Click on Create App password and provide the following information.

Label — Choreo console
Scopes — 
Account (Read),
Pull requests (Read/Write),
Repositories (Read/Write),
Webhooks (Read/Write)

Copy the generated app password and save it securely as you won’t be able to access it again.

2. Login to Choreo.

From the left menu of the Organization home page navigate to Settings and then in the ‘Credentials’ tab, click on + Add Git Credentials.

Provide the following information to create a new credential and click on save.

Credential Name — BitbucketCredentials
Service Provider — Bitbucket
Username — Username of your Bitbucket Account
Access Token — Copy paste the app password generated earlier

The credential you created should be listed under the common credential tab as shown below.

Step 2 — Create a component

From the Organization home page click the + Create Project button to create a new project with the name ‘Reservation Management’.
Once the project is successfully created, navigate to the ‘Reservation Management’ project.

Create a Service by providing the following information :
Component Name — ReservationService
Source Provider — Navigate to the Bitbucket tab
Authorize with Bitbucket — Select the credential you created earlier

User group — Select the workspace you created the repository under
Repository — Select ‘choreo-samples’ or the repository name you provided
Branch — Main
Buildpack — Docker
Docker Context — /python-flask-app
Dockerfile — /python-flask-app/Dockerfile

Step 3 — Build an image

Once the component is created, you will be directed to the Build page, where the initial build for the latest commit in your repository will be in progress.

Once the build is successful, click on the ‘Deploy’ link to navigate to the Deploy page.

Step 4 — Deploy to production

Click on ‘Configure & Deploy’ in the Set Up card.

The service we just created is now ready to be deployed to the development environment.

To promote this service to the production environment, click ‘Promote’ and select the ‘Use Development configuration’ option from the right drawer.

Step 5 — Test the service

Once the service is successfully promoted to production, navigate to the Test -> Console page from the left menu.

Click on the POST request and enter the following schema to test the service we created.

{

“contact”: “011–123–1111”,

“reservationCreator”: “John Doe”,

“reservationId”: “1111”

}

Click on execute to add this new entry to the reservation management system and check if the response is successful.

Then, click on GET to send a request to retrieve the list of reservations and verify if the new reservation we just added is available.

You have successfully created a service and deployed it to production!

Summary

In this article, we demonstrated how to deploy a Python-based service for managing a list of reservations using Choreo. The API allows users to create, retrieve, update, and delete reservation entries. The process began with creating a new project in a Bitbucket workspace and connecting it to Choreo by creating a service component. We then built an image from the latest commit , deployed it to the development environment and promoted it to production to make it live. Finally, we tested the service to ensure everything was working as expected. Sign up to Choreo for free to explore these features and unlock more possibilities!

Deploying Code from Bitbucket Cloud to Production with Choreo was originally published in Choreo Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

In recent years, Python has seen a surge in popularity for web application development, becoming a go-to language for developers worldwide. This rise can be attributed to Python’s simplicity, readability, and the rich ecosystem of frameworks it supports such as FastAPI, Flask, Django etc. Among these frameworks Django stands out as a comprehensive, “batteries-included” framework as it offers a wide range of features and tools that are ready to use out of the box. This enables developers to focus more time on implementing the actual business logic rather than having to worry about things like authentication and authorization, user management, database integration etc. So with Django, you can rapidly build your applications in no time without much hassle. Once your application is built and ready to launch, you may encounter a new set of challenges such as where to run, how to scale it with increasing traffic, monitoring and troubleshooting etc. This is where Choreo comes in.

Choreo is an internal developer platform as a service, which facilitates building, deploying, and running your cloud-native applications in state-of-the-art Kubernetes based environments and provides you with a rich set of tools to monitor and observe your application. Choreo provides a seamless platform to deploy and manage your Django applications, eliminating the complexities of software delivery and infrastructure management.

Django and Choreo together create a powerful combination, enabling developers to effortlessly build, test, deploy, scale, monitor, and troubleshoot applications reducing the go to market time.

In this article, we will explore how to develop a simple e-commerce web application using Django and how to deploy, scale and monitor it using Choreo.

Setting up the local development environment

I will be using python 3.11 to develop the sample e-commerce web application with Django. First let’s create a virtual environment in python using venv. This helps to isolate the project dependencies from the global python environment.

python3 -m venv ./django

Next we will activate the virtual environment created and install the required dependencies.

source ./django/bin/activate

Use the following command to install django in this virtual environment.

pip install django

Bootstrapping a django project using django-admin

Execute the following command to bootstrap a django project using django-admin cli.

django-admin startproject django_ecommerce

Next we need to initialise an app for the ecommerce store. In django, apps are self contained modules that serve a specific purpose within a project. In a typical django project there maybe several apps responsible for different functional aspects of the project. However as this is a sample project, let’s implement all the functionalities in the same app.

python manage.py startapp store

Store application implementation

Let’s now implement the e-commerce store related functionality in this store app. This will be a simple e-commerce store named Textile Treasures with 4 products namely Silk Brocade, Merino Wool Yarn, Embroidery Hoop Set and Cotton Batik. Customers can view the products, add them to the cart and do a simple checkout without any payment information. The customers also have the option to maintain user accounts and they can view the order history. All these functionality will be implemented in the store app itself.

While this article is not intended to deeply explain Django itself, it aims to provide a high-level overview of how you can use Django and Choreo together to fast-track your development process. So I will not explain all the implementation in detail. The source code is publicly available and you can explore the implementation by going through the source code. To provide a high level idea about the implementation, I will explain the folder structure of the store app and what goes into each file. The code is available here.

store/
├── __init__.py
├── admin.py
├── apps.py
├── forms.py
├── models.py
├── templatetags/
│   ├── __init__.py
│   └── custom_filters.py
├── migrations/
│   ├── __init__.py
│   └── 0001_initial.py
├── fixtures/
├── static/
│   └── store/
│       ├── css/
│       │   └── styles.css
│       ├── js/
│       │   └── scripts.js
│       └── images/
│           └── logo.png
├── templates/
│   └── store/
│       ├── base.html
│       ├── home.html
│       ├── product_detail.html
│       ├── cart.html
│       ├── checkout.html
│       ├── order_success.html
│       ├── signup.html
│       └── order_history.html
├── tests.py
├── urls.py
└── views.py

• admin.py: Contains admin site configurations
• apps.py: Contains app configuration
• forms.py: Contains form classes
• models.py: Contains model definitions
• migrations/: Directory for database migrations
• fixtures/: Directory for fixture files.
• static/: Directory for static files.
• templates/: Directory for HTML templates.
• urls.py: Contains URL configurations.
• views.py: Contains view functions.

Data models, migrations and seeding

Also when we consider the data model, we are going to have 3 tables in the database namely store_product, store_order and store_orderitem. There are 3 models in the models.py to represent each table in the database. Once the models are implemented, you can run the migrations to create the tables.

python manage.py makemigrations
python manage.py migrate
python manage.py createsuperuser

The important thing to note here is that you don’t have to worry about things like which ORM to use and which database connector library to use as Django has a built in ORM and has support for a wide range of databases out of the box. For the local development and testing, we will go ahead with the default database which is SQLite.

Before starting the application locally for testing, we need to seed the database with some sample data. For this purpose you don’t need to write any shell scripts or any python scripts. Django supports database seedings through fixtures. Execute the following command to seed the local SQLite database with the intial_data.json file in the fixtures directory.

python manage.py loaddata initial_data.json

Now we are all set to test our Textile Treasures e-commerce store locally. Execute the following command to run the application.

python manage.py runserver

If everything is working as expected, you will see the following output in the terminal and you can click on the development server url to access the e-commerce store.

September 02, 2024 - 14:09:31
Django version 5.1, using settings 'django_ecommerce.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

Deploying the application in Choreo

Now we have tested and verified the functionality of the e-commerce store locally and now it’s time to deploy our application in Choreo for production.

1. Sign up to Choreo and create an account.
2. Provide a name and create an organization.
3. Go to Dependencies > Databases and create a database server. For this application we will use a PostgreSQL database.

4. Execute the database migrations and seeds against the created PostgreSQL database.

5. Create a project by providing a name and description.

6. Navigate to Components and Click + Create Component.

7. From the Create a New Component menu, select Web Application and provide the following details.

Name - Django Ecommerce App
Description - Simple e-commerce application implemented using Django framework
Repository URL - https://github.com/NomadXD/django-ecommerce
Branch - main
Buildpack - Python
Language version - 3.11.x
Python Project Directory - /
Port - 8000

8. After creating the component, navigate to the Build section from the left sidepane and click Build Latest Commit, this would start building a container image using the code provided with a Python Buildpack.

9. After the build is successful, navigate to the Deploy section from the left pane, and click Configure & Build. Copy the database credentials of the previously created database and provide the DATABASE_URL as a field in the config.json. DATABASE_URL should be in the following format.

postgres://<DB_USER>:<DB_PASSWORD>@<DB_HOST>:<DB_PORT>/<DB_NAME>

10. You can access the application using the Web App URL in the environment card and start using the application.

If you followed the steps correctly, you will have a fully functional application up and running in the Choreo Cloud Dataplane managed Kubernetes cluster with a Choreo managed PostgreSQL database. And just like that, in a few minutes you have a production grade Python application running on state of the art Kubernetes infrastructure! It’s that simple. And the best part is no YAMLs and no Dockerfiles. So how does Choreo containerize your application without a Dockerfile ? Choreo uses cloud native buildpacks to identify and containerize your applications intelligently into kubernetes workloads. If you have gone through the project repository, you would have noticed a Procfile with a single command. This defines the command to execute when the containerized application is started.

web: python3 manage.py runserver 0.0.0.0:8000

Monitoring, Scaling and beyond

By now you have fully functioning e-commerce application up and running in Choreo. Now you would want to monitor your application for potential issues and troubleshoot them. Choreo offers a unified log analyse view for monitoring and troubleshooting issues. Go to Observability > Runtime Logs for accessing the logs.

When the actual production traffic comes and more and more users start using your application, you would want to scale your application with the increased demand. Choreo offers autoscaling capabilities and allows you to scale your applications based on CPU, memory and HTTP traffic. Go to DevOps > Scaling to configure autoscaling for your applications running in Choreo.

Deploying applications in the cloud can be a daunting task given today’s complex technology landscape. However, Choreo abstracts away these complexities, providing a developer-friendly platform that makes it effortless to build, deploy, and manage your workloads. We’ve just experienced how easy and efficient this process can be by deploying a Django eCommerce app with Choreo.

Source code for the sample project available at: [1] https://github.com/NomadXD/django-ecommerce

Effortless Development of Cloud-Native Applications in Django with Choreo was originally published in Choreo Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.