OpenAI vs Azure OpenAI vs Anthropic vs Gemini for .NET: Provider Comparison Guide
The Problem
Your team just shipped a working prototype — a customer support bot that calls GPT-4o directly from your ASP.NET Core controller. It works. The product demo was a success. Now the enterprise team asks: “Can we deploy this in our Azure tenant so our data doesn’t leave the region?” The security team asks: “Can we swap to Claude for the next feature because it handles long documents better?” The finance team asks: “Why is every question costing ten times what we estimated?”
Each of those questions exposes the same structural problem: the code is hardwired to one SDK, one provider, one way of calling a model. Changing the provider means rewriting every service that touches AI. Comparing costs between models requires creating throwaway branches. Testing business logic means provisioning a live model endpoint or writing elaborate mocks that are tightly coupled to one client library.
This is where provider abstraction earns its value. The right architecture decouples your business logic from the model provider entirely — you swap providers through configuration, not code. That decision is only possible if you understand what the providers actually are, where they differ, and which one fits your requirements before you commit.
What This Blog Covers
This guide gives .NET developers a complete, honest map of the four providers, then shows how to build an abstraction layer that makes them interchangeable at runtime. Here are the concepts covered:
- Provider overview — What each provider is and the deployment model it uses
- SDK maturity — Which .NET packages are official, which are community-maintained, and what that means for production use
- Pricing mechanics — Token-based cost structures, tiers, and the levers you can pull to reduce spend
- Enterprise fit — Compliance certifications, data residency, private deployments, and SLA coverage
- Structured outputs — How each provider enforces schema compliance and where the gaps are
- Microsoft.Extensions.AI and IChatClient — The abstraction layer that makes providers interchangeable in .NET
- Hands-on PoC — A working provider abstraction with interchangeable adapters and a common prompt contract
Concept 1: Provider Overview
What it is?
Each “provider” in this guide refers to an API endpoint that accepts a prompt and returns a completion. The four providers covered here are: OpenAI (direct API), Azure OpenAI (OpenAI models hosted inside Microsoft’s cloud), Anthropic (Claude models), and Google Gemini (via the Gemini Developer API or Vertex AI).
Why it exists?
Enterprise software teams rarely commit to a single model indefinitely. Models are deprecated, better models ship, pricing changes, and different tasks suit different models. A provider layer exists so your application can evolve without rewriting its core logic.
OpenAI and Azure OpenAI both serve GPT-family models. Azure OpenAI is not a different model — it is the same model deployed inside Azure, with Microsoft’s compliance and network controls layered on top. This distinction matters enormously for enterprise buyers.
When to use which?
Use OpenAI direct when you are prototyping, building an internal tool, or operating in a startup context where simplicity and access to the latest models matter more than data residency.
Use Azure OpenAI / Azure AI Foundry when your organization runs on Azure, requires GDPR data zone isolation, SOC 2, HIPAA, or ISO 27001 alignment, or needs a private endpoint that never routes through public internet.
Use Anthropic Claude when long-context document processing, nuanced instruction following, or safety-first enterprise positioning are primary requirements. Anthropic holds 32% of enterprise LLM market share as of 2025.
Use Gemini when you need a context window beyond 200,000 tokens, are already in Google Cloud, or need multimodal (text + image + audio) processing at scale.
When not to use a multi-provider abstraction?
Do not build an abstraction layer for a prototype or a single-feature application. The abstraction adds indirection that slows down debugging and adds DI complexity that is only justified when you have multiple providers to swap. For a toy project or a one-shot internal script, call the SDK directly.
Concept 2: SDK Maturity
What it is?
SDK maturity refers to how complete, stable, and officially maintained the .NET client library for each provider actually is not how polished the marketing page looks.
Why it matters?
An unmaintained SDK means you are writing workarounds the day a model update changes the response shape. A beta SDK means breaking API changes between minor versions. This is not theoretical risk in 2025, the official Anthropic C# SDK shipped as version 10+ (official release), replacing the community-built Anthropic.SDK package that previously held the Anthropic NuGet name.
SDK status per provider
OpenAI .NET SDK (dotnet add package OpenAI) is the official library, generated from OpenAI's OpenAPI specification in collaboration with Microsoft. It targets .NET Standard 2.0 and is generally available. It supports both synchronous and asynchronous APIs, structured outputs via ChatResponseFormat.CreateJsonSchemaFormat<T>, streaming completions, and function calling.
Azure.AI.OpenAI is Microsoft’s official Azure-specific wrapper on top of the OpenAI SDK. It requires specifying an AzureOpenAIClientOptions.ServiceVersion to access newer features like structured outputs. As of 2026, Azure AI Foundry (the evolved form of Azure OpenAI Service) uses the v2 SDK.
Anthropic C# SDK (dotnet add package Anthropic) is the official SDK as of version 10+, but remains in beta. The SDK implements IChatClient from Microsoft.Extensions.AI.Abstractions, which is the critical integration point for .NET DI. It also ships Anthropic.Bedrock and Anthropic.Vertex packages for deploying Claude on AWS and Google Cloud respectively.
Google Gen AI .NET SDK (Google.Cloud.VertexAI.Extensions) was announced in January 2026 as the Vertex AI implementation of Microsoft.Extensions.AI. Before this, developers relied on either Google.Cloud.AIPlatform.V1 (the Vertex AI gRPC SDK) or the unofficial Google_GenerativeAI community package. The official recommendation is now: if you only use Google, use the Google Gen AI .NET SDK; if you plan to switch between providers, use Google.Cloud.VertexAI.Extensions with IChatClient.
Code sample: SDK maturity check — installing all four
// Official OpenAI SDK
dotnet add package OpenAI
// Azure AI Foundry (Azure OpenAI)
dotnet add package Azure.AI.OpenAI
// Anthropic Claude (official, beta as of 2026)
dotnet add package Anthropic
// Gemini via Google Cloud Vertex AI (IChatClient-compatible, January 2026)
dotnet add package Google.Cloud.VertexAI.Extensions --prerelease
// Abstraction layer (ships with all provider adapters)
dotnet add package Microsoft.Extensions.AIConcept 3: Pricing Mechanics
What it is?
Every provider bills per token roughly per word fragment for both input (your prompt) and output (the model’s response). The specific rates, tier names, and features included at each price point differ significantly.
Why it matters?
A 20x price difference exists between the cheapest and most expensive tier within a single provider’s model family. Getting this wrong on a production system with real traffic is how teams end up with surprise invoices before they understand what drove the cost.
How pricing is structured?
- OpenAI direct uses consumption-based pricing. GPT-4o-mini at $0.15 per million input tokens delivers 16.7x lower cost than GPT-4o while maintaining the same feature set. There is no reserved capacity option.
- Azure AI Foundry (Azure OpenAI) adds a “Provisioned Throughput Units” (PTU) model for steady workloads reserve capacity in exchange for a committed spend, which reduces per-token variance and guarantees throughput. The GPT-5.4 family on Azure in 2026 spans from ~$0.10/M tokens (GPT-5.4-nano) to ~$2.00/M tokens (GPT-5.4) for input.
- Anthropic pricing is competitive with OpenAI at the mid tier. Anthropic has a prompt caching feature that can reduce input costs by up to 90% when the same context is reused across calls a significant advantage for applications that reuse large system prompts or reference documents.
- Google Gemini offers the lowest base rates at several tiers. Gemini 2.5 Pro costs $1.25/M input tokens under 200K context. The generous free tier during development and Gemini Flash models for speed-sensitive workloads make Google the default choice for budget-conscious teams.
Cost control in C#
// Partial example — model routing based on task complexity
public class ModelRouter
{
// Route cheap, simple tasks to the nano model
// Route complex multi-step reasoning to the full model
public string SelectModel(TaskComplexity complexity) => complexity switch
{
TaskComplexity.Classification => "gpt-5.4-nano", // ~20x cheaper than full
TaskComplexity.Conversational => "gpt-5.4-mini", // ~5x cheaper
TaskComplexity.Reasoning => "gpt-5.4", // full capability
_ => "gpt-5.4-mini"
};
}Concept 4: Enterprise Fit
What it is?
Enterprise fit is shorthand for: can a regulated organization healthcare, financial services, government use this provider without violating data privacy law, contractual obligations, or internal security policy?
Why it matters?
A .NET developer building a customer support chatbot for a bank does not get to decide “we’ll use the cheapest API”. The security, legal, and compliance teams all have veto power. Understanding where each provider actually stands on compliance saves weeks of due diligence cycles.
How each provider positions for enterprise?
Azure OpenAI / Azure AI Foundry has the strongest enterprise story by a significant margin. It aligns with ISO 27001, ISO 27701, SOC 2, SOC 3, HIPAA (via Business Associate Agreement), FedRAMP High, PCI DSS, and GDPR. The “Data Zone Standard” deployment type guarantees data processing stays within a specific geographic zone (e.g., EU). Customer data is never used to train foundation models without explicit opt-in. Private endpoints and VNet integration keep traffic off the public internet.
OpenAI direct has completed a SOC 2 audit, supports data encryption (AES-256 at rest, TLS 1.2+ in transit), and can execute a Data Processing Addendum (DPA) for GDPR compliance. However, it lacks the regional data zone isolation that Azure provides, and has no native private endpoint option.
Anthropic operates a business-to-business model where 80% of revenue comes from enterprises. It runs multi-cloud infrastructure (AWS, Google, and Azure) and emphasizes a safety-first approach. Compliance documentation is available but not as extensively pre-certified as Azure.
Google Gemini via Vertex AI inherits Google Cloud’s compliance portfolio (SOC 2, HIPAA, ISO 27001, FedRAMP). For organizations in the Google Cloud ecosystem, Vertex AI provides enterprise controls comparable to Azure.
Concept 5: Structured Outputs
What it is?
Structured outputs is a provider feature where you define a JSON schema and the API guarantees not just encourages that the model’s response will match that schema exactly. This is different from “JSON mode”, which asks the model to return valid JSON but does not enforce field names, types, or required properties.
Why it exists?
Without structured outputs, every AI-integrated service needs defensive parsing code: try/catch around JsonSerializer.Deserialize, retry logic for malformed JSON, regex fallbacks for "almost JSON" responses. Structured outputs eliminate an entire category of runtime failures by enforcing the contract at the API level.
How each provider handles it?
Azure OpenAI / OpenAI has the most mature implementation. Using ChatResponseFormat.CreateJsonSchemaFormat<T>(strict: true), the SDK generates your C# type's JSON schema and sends it with the request. The model cannot return a response that violates the schema. If it cannot produce a valid response, the API returns a 400 error. Supported types are limited to JSON Schema primitives — DateTime, Uri, and TimeSpan must be represented as string with [Description] attributes.
Anthropic Claude launched structured outputs as a public beta in November 2025, requiring the structured-outputs-2025-11-13 beta header. It supports JSON schema via output_config.format with type: "json_schema". Important limitations: recursive schemas and numerical constraints (minimum, maximum) are not supported. Citations and structured outputs cannot be used in the same request.
Get Bhargava Koya - Fullstack .NET Developer’s stories in your inbox
Join Medium for free to get updates from this writer.
Google Gemini supports response schemas via GenerationConfig.ResponseSchema. The official Google Gen AI .NET SDK supports configuring a response schema and generation settings. Schema enforcement is described as "constrained decoding" on the model side.
Code sample: Azure OpenAI structured output in C# (SDK direct approach)
// Full example — Azure OpenAI structured output with strict schema enforcement
using Azure.AI.OpenAI;
using OpenAI.Chat;
using System.ComponentModel;
using System.Text.Json;
// Define the schema type — only JSON Schema primitives are supported
public class SupportTicketAnalysis
{
public string Category { get; set; } = string.Empty; // e.g. "billing", "technical"
public string Priority { get; set; } = string.Empty; // "low", "medium", "high", "critical"
[Description("One to two sentence summary of the issue")]
public string Summary { get; set; } = string.Empty;
public string[] RequiredTeams { get; set; } = []; // which teams must be looped in
[Description("Estimated resolution time as a human-readable string, e.g. '2 business days'")]
public string EstimatedResolution { get; set; } = string.Empty;
}
// Build the client — structured outputs require API version 2024-08-01-preview or later
var azureClient = new AzureOpenAIClient(
new Uri(Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT")!),
new AzureKeyCredential(Environment.GetEnvironmentVariable("AZURE_OPENAI_KEY")!),
new AzureOpenAIClientOptions(AzureOpenAIClientOptions.ServiceVersion.V2024_08_01_Preview));
ChatClient chatClient = azureClient.GetChatClient("gpt-5.4-mini-deployment");
// Configure the request to enforce the schema strictly
var options = new ChatCompletionOptions
{
ResponseFormat = ChatResponseFormat.CreateJsonSchemaFormat<SupportTicketAnalysis>(
"support_ticket_analysis", // snake_case — spaces and hyphens cause 400 errors
strict: true) // without this, schema compliance is best-effort only
};
List<ChatMessage> messages =
[
new SystemChatMessage("You are a support ticket classifier. Analyze the ticket and return a structured assessment."),
new UserChatMessage("User reports: 'My invoice shows $999 but the contract says $299. Need this fixed immediately.'")
];
// If structured outputs fail with a 400, the schema has unsupported types — check for DateTime, Uri, TimeSpan
ClientResult<ChatCompletion> result = await chatClient.CompleteChatAsync(messages, options);
// JsonSerializer.Deserialize will always succeed if the API call succeeded
SupportTicketAnalysis analysis = JsonSerializer.Deserialize<SupportTicketAnalysis>(
result.Value.Content.Text)!;
Console.WriteLine($"Category: {analysis.Category}");
Console.WriteLine($"Priority: {analysis.Priority}");
Console.WriteLine($"Summary: {analysis.Summary}");Concept 6: Microsoft.Extensions.AI and IChatClient
What it is?
Microsoft.Extensions.AI is the official .NET abstraction layer for AI providers. Its central interface, IChatClient, defines a single way to send chat messages and receive responses — regardless of whether the underlying provider is OpenAI, Azure, Anthropic, or Gemini.
Why it exists?
Without this abstraction, every provider has its own method names, request/response types, and configuration patterns. Switching providers means finding and rewriting every call site. IChatClient plays the same role for AI that ILogger plays for logging and HttpClientFactory plays for HTTP — you code to the interface, push provider specifics into configuration.
How it works?
The Microsoft.Extensions.AI.Abstractions package defines the IChatClient interface. Provider packages implement it:[^28]
Azure.AI.OpenAIships anIChatClientimplementation for Azure OpenAIAnthropic(official SDK, v10+) implementsIChatClientGoogle.Cloud.VertexAI.ExtensionsimplementsIChatClientfor Gemini on Vertex AI
The abstraction supports composable middleware (decorators) for logging, retries, caching, and OpenTelemetry — added by wrapping IChatClient implementations using the ChatClientBuilder pipeline.
Code sample: Registering providers behind IChatClient in DI
// Partial example — Program.cs provider registration with IChatClient
using Microsoft.Extensions.AI;
using Azure.AI.OpenAI;
using OpenAI;
using Anthropic;
using Google.Cloud.VertexAI.Extensions;
var builder = WebApplication.CreateBuilder(args);
var config = builder.Configuration;
// Read which provider to use from configuration — no code change to switch
string provider = config["AI:Provider"]!; // "azure", "openai", "anthropic", "gemini"
builder.Services.AddSingleton<IChatClient>(sp =>
{
// Return the appropriate IChatClient implementation based on config
return provider switch
{
"azure" => new AzureOpenAIClient(
new Uri(config["AI:Azure:Endpoint"]!),
new Azure.AzureKeyCredential(config["AI:Azure:Key"]!))
.GetChatClient(config["AI:Azure:Deployment"]!)
.AsIChatClient(), // extension method from Azure.AI.OpenAI
"openai" => new OpenAIClient(config["AI:OpenAI:Key"]!)
.GetChatClient(config["AI:OpenAI:Model"]!)
.AsIChatClient(),
"anthropic" => new AnthropicClient(config["AI:Anthropic:Key"]!)
.AsIChatClient(), // Anthropic SDK v10+ implements IChatClient
"gemini" => new VertexAIChatClient(
projectId: config["AI:Gemini:ProjectId"]!,
location: config["AI:Gemini:Location"]!,
modelId: config["AI:Gemini:Model"]!),
_ => throw new InvalidOperationException($"Unknown AI provider: {provider}")
};
});
// Add OpenTelemetry instrumentation via pipeline decorator — applies to all providers
builder.Services.AddSingleton(sp =>
{
var inner = sp.GetRequiredService<IChatClient>();
return new ChatClientBuilder(inner)
.UseOpenTelemetry(loggerFactory: sp.GetRequiredService<ILoggerFactory>(),
sourceName: "YourApp.AI")
.Build();
});Hands-On: Provider Abstraction PoC
Problem statement
Build a working ASP.NET Core Minimal API that accepts a support ticket as plain text and returns a structured analysis. The provider OpenAI, Azure, Anthropic, or Gemini is selected via appsettings.json without touching any service code. This is the minimal implementation of a common prompt contract with interchangeable adapters.
The goal is not feature completeness. It is demonstrating the seam: the part of the architecture that lets you swap providers by changing one line of config.
Step 1: Create the project and install packages
This comes first because the package references determine which IChatClient adapters are available at compile time.
dotnet new webapi -n AiProviderPoc --no-openapi
cd AiProviderPoc
dotnet add package Microsoft.Extensions.AI
dotnet add package OpenAI
dotnet add package Azure.AI.OpenAI
dotnet add package Anthropic
dotnet add package Google.Cloud.VertexAI.Extensions --prereleaseStep 2: Define the prompt contract
The prompt contract is the interface between your application and the model. It defines what you ask and what structure you expect back. Defining it before wiring up providers means the contract is never specific to any one SDK.
// PromptContracts.cs — shared across all providers
using System.ComponentModel;
namespace AiProviderPoc;
// The input to every analysis request
public record AnalysisRequest(string TicketText, string? Context = null);
// The structured output — must use only JSON Schema primitives (string, int, bool, arrays)
public class TicketAnalysis
{
[Description("Category of issue: 'billing', 'technical', 'account', 'feature-request'")]
public string Category { get; set; } = string.Empty;
[Description("Severity: 'low', 'medium', 'high', 'critical'")]
public string Severity { get; set; } = string.Empty;
[Description("One to two sentence summary of the customer's issue")]
public string Summary { get; set; } = string.Empty;
[Description("Suggested action for the support agent, one sentence")]
public string SuggestedAction { get; set; } = string.Empty;
public bool RequiresEscalation { get; set; }
[Description("Estimated time to resolve as human-readable string, e.g. '30 minutes' or '2 business days'")]
public string EstimatedResolution { get; set; } = string.Empty;
}Step 3: Define the prompt service
The prompt service depends only on IChatClient. It never imports an SDK package. This is the seam the point where provider differences disappear.
// TicketAnalysisService.cs
using Microsoft.Extensions.AI;
using System.Text.Json;
namespace AiProviderPoc;
public class TicketAnalysisService(IChatClient chatClient)
{
// The system prompt is the stable contract — versioned and centralized here
private const string SystemPrompt =
"""
You are a support ticket classifier for an enterprise software product.
Analyze the provided support ticket and return a structured assessment.
Be concise and accurate. Do not invent information not present in the ticket.
""";
public async Task<TicketAnalysis> AnalyzeAsync(AnalysisRequest request, CancellationToken ct = default)
{
// Build the message list — system prompt is fixed, user content is the ticket
var messages = new List<ChatMessage>
{
new(ChatRole.System, SystemPrompt),
new(ChatRole.User, request.TicketText)
};
// Use JSON mode via ChatOptions — provider-agnostic structured output
// For strict schema enforcement with Azure/OpenAI, use the SDK-direct approach instead
var options = new ChatOptions
{
ResponseFormat = ChatResponseFormat.Json
};
ChatResponse response = await chatClient.GetResponseAsync(messages, options, ct);
// Deserialize — IChatClient normalizes the response shape across all providers
return JsonSerializer.Deserialize<TicketAnalysis>(response.Text,
new JsonSerializerOptions { PropertyNameCaseInsensitive = true })
?? throw new InvalidOperationException("Model returned unparseable JSON");
}
}Step 4: Wire providers in Program.cs
The provider selection logic lives entirely in Program.cs. No service class knows this exists.
using AiProviderPoc.Api;
using Anthropic;
using Azure.AI.OpenAI;
using Google.GenAI;
using Microsoft.Extensions.AI;
using OpenAI;
using System.ClientModel; // ApiKeyCredential
var builder = WebApplication.CreateBuilder(args);
var config = builder.Configuration;
// Read provider selection from config — change "AI:Provider" to switch providers
string provider = config["AI:Provider"]
?? throw new InvalidOperationException("AI:Provider must be set in appsettings");
// Register IChatClient — each branch returns a different SDK's implementation
builder.Services.AddSingleton<IChatClient>(_ =>
{
return provider.ToLowerInvariant() switch
{
"azure" =>
// Azure AI Foundry: needs endpoint + key + deployment name
new AzureOpenAIClient(
new Uri(config["AI:Azure:Endpoint"]!),
new ApiKeyCredential(config["AI:Azure:Key"]!))
.GetChatClient(config["AI:Azure:Deployment"]!)
.AsIChatClient(),
"openai" =>
// OpenAI direct: needs API key + model name
new OpenAIClient(config["AI:OpenAI:Key"]!)
.GetChatClient(config["AI:OpenAI:Model"]!)
.AsIChatClient(),
"anthropic" =>
// AnthropicClient expects a ClientOptions instance (not a string API key).
// Create a ClientOptions and pass it to the constructor.
new AnthropicClient(
new Anthropic.Core.ClientOptions
{
ApiKey = config["AI:Anthropic:Key"]!
})
.AsIChatClient(),
"gemini" =>
// Google.GenAI: named params on Client constructor, no ClientConfig class needed.
// Uses GOOGLE_APPLICATION_CREDENTIALS / ADC for auth; no API key required for Vertex AI.
new Client(
project: config["AI:Gemini:ProjectId"]!,
location: config["AI:Gemini:Location"]!,
vertexAI: true)
.AsIChatClient(config["AI:Gemini:Model"]!),
_ => throw new InvalidOperationException($"Unknown provider: '{provider}'. Valid: azure, openai, anthropic, gemini")
};
});
// Register the service — depends only on IChatClient
builder.Services.AddScoped<TicketAnalysisService>();
var app = builder.Build();
// Single endpoint — provider is invisible here
app.MapPost("/analyze", async (AnalysisRequest request, TicketAnalysisService service, CancellationToken ct) =>
{
var analysis = await service.AnalyzeAsync(request, ct);
return Results.Ok(analysis);
});
app.Run();Step 5: Configure per provider in appsettings
{
"AI": {
"Provider": "azure",
"Azure": {
"Endpoint": "https://your-resource.openai.azure.com/",
"Key": "your-azure-key",
"Deployment": "gpt-5.4-mini-deployment"
},
"OpenAI": {
"Key": "sk-...",
"Model": "gpt-4o-mini"
},
"Anthropic": {
"Key": "sk-ant-..."
},
"Gemini": {
"ProjectId": "your-gcp-project",
"Location": "us-central1",
"Model": "gemini-2.0-flash"
}
}
}To switch providers: change "AI:Provider" to "openai", "anthropic", or "gemini". Restart. No code changes.
Final verification
Run the application and send a test request:
dotnet run
curl -X POST http://localhost:5000/analyze \
-H "Content-Type: application/json" \
-d '{"ticketText": "My subscription renews tomorrow but I was charged twice this month. Account #A-1234."}'Expected output:
{
"category": "billing",
"severity": "high",
"summary": "Customer was charged twice for their subscription and is seeking a refund before their next renewal date.",
"suggestedAction": "Review billing history for account A-1234 and initiate a refund for the duplicate charge.",
"requiresEscalation": false,
"estimatedResolution": "1 business day"
}Changing AI:Provider from "azure" to "openai" and rerunning returns the same structure from a different model — the endpoint, service, and contract are unchanged.
Best Practices
- Centralize the system prompt. Treat it as a versioned configuration asset, not an inline string. When the system prompt changes, you want one place to update and one test to verify the contract still holds.
- Use
[Description]attributes to constrain string properties. For fields that represent enum-like values ("low" | "medium" | "high"), the description attribute is the model's instruction. Without it, the model will invent its own vocabulary. - Validate your schema before deploying. Use
JsonSchemaExporter.GetJsonSchemaAsNode()(.NET 9+) to inspect the generated schema for any$ref,format, or non-primitive types that will cause 400 errors on Azure/OpenAI structured outputs. - Never hardcode model names in service classes. Model names belong in configuration. When GPT-4o is deprecated or Claude Sonnet 4.5 is replaced, you want a config change and a redeployment, not a code review.
- Add OpenTelemetry from day one. The
ChatClientBuilder.UseOpenTelemetry()middleware adds spans with model name, token counts, and latency to every call across all providers. This is the only way to answer "which provider is slower?" with data rather than anecdote. - Test against a fake
IChatClientin unit tests. The abstraction exists for testability as much as portability. AFakeChatClientthat returns predictable JSON lets you test classification logic, error paths, and output mapping without ever calling a live API.
Common Mistakes
- Mixing provider SDK types into service classes. If your
SupportService.csimportsAzure.AI.OpenAI, the abstraction is broken before it starts. Every service should depend onIChatClientfromMicrosoft.Extensions.AI, not on any provider namespace. - Using JSON mode and trusting it completely. JSON mode (
ResponseFormat = ChatResponseFormat.Json) asks the model politely. It is not a guarantee. Structured outputs withstrict: trueon Azure/OpenAI is the guarantee. If you are parsing AI output and storing it in a database, the difference is between a reliable pipeline and an intermittent production incident. - Ignoring API version requirements for Azure. Azure OpenAI structured outputs require
ServiceVersion.V2024_08_01_Previewor later. Using the default service version silently falls back to JSON mode without structured output guarantees. The error only surfaces at runtime when schema enforcement is expected but not happening. - Not accounting for Anthropic’s beta headers. Claude structured outputs (as of November 2025) require the
structured-outputs-2025-11-13beta header. Omitting it means the feature silently does not apply. This kind of silent degradation is hard to catch without integration tests that verify the response shape.
Bottlenecks at Scale
- Token budget overruns. The most common cost spike in production is an unbounded context window. A single request can include conversation history, a long document, a large system prompt, and the model’s output. Add up four of those in a RAG pipeline and you are looking at 50,000+ tokens per request. Enforce per-request token budgets in your
ChatOptionsand add alerting on token usage before it surprises the finance team. - Rate limits hitting before you expect them. Every provider has per-minute and per-day rate limits that apply at the API key level. Under real load — especially during business hours when multiple services share a key — you will hit 429 responses. Implement exponential backoff with jitter as a
ChatClientBuilderdecorator, not inside each service. - Structured output schema caching. Azure OpenAI processes and validates your JSON schema on the first call and caches it per deployment. Frequently changing schema types during development or A/B testing adds per-call latency. Stabilize your schema types before load testing.
- Provider latency differences. Gemini’s large context window comes at a latency cost for long inputs. Anthropic’s prompt caching helps when you reuse large system prompts, but requires the cache to warm up on first use. At scale, provider latency variance becomes a P99 problem even when P50 looks acceptable.
Alternatives
- Semantic Kernel — Microsoft’s AI orchestration framework built on top of
Microsoft.Extensions.AIabstractions. Adds agent planning, plugin architecture, and memory. Appropriate when you need multi-step reasoning pipelines, not just single completions. Trade-off: higher abstraction means more framework knowledge required and more difficulty debugging low-level API behavior. - Ollama / local models — Run open-weight models (Llama, Mistral) on-premise. The
IChatClientabstraction supports Ollama viaMicrosoft.Extensions.AI.Ollama. Trade-off: you own the infrastructure, which eliminates provider cost but adds DevOps burden and eliminates frontier model capabilities. - AWS Bedrock — Amazon’s multi-model gateway gives access to Claude, Llama, and other models through a single AWS API. The Anthropic SDK ships
Anthropic.Bedrockfor .NET. Trade-off: best option for AWS-native teams, but adds a second provider abstraction on top of the model API.
Closing Thought
Every team starts with one provider because one is enough to ship. The abstraction only seems necessary the day the enterprise security team blocks the direct API, or the day pricing for the model you depend on doubles. By the time you need it, adding the abstraction is painful.
The question worth sitting with: in your current codebase, how many files would you need to change to switch from OpenAI to Azure OpenAI today? If the answer is more than one, the abstraction has already earned its cost.
I hope you enjoyed reading this blog!
I’d love to hear your thoughts. please share your feedback or questions in the comments below and let me know if you’d like any clarifications on the topics covered. If you enjoyed this blog, don’t forget to like it and subscribe for more technology insights.
Stay tuned! In upcoming posts, I’ll be diving into advanced .NET topics such latest updates on .NET, UI frameworks like Angular and React, Blazor, AI, Machine learning, Python. Additionally, much more topics on DSA and System Design.
Thank you for joining me on this learning journey!
Stay curious, keep coding and never stop learning.
Connect with me:
