That question catches almost every quantum developer by surprise. Understanding it is essential if you plan to build larger, more reliable quantum applications.

This tutorial assumes you're already comfortable creating and executing basic quantum circuits in Qiskit.

The first time you execute a circuit on real hardware, you'd expect the output to match the simulator. After all, the code, algorithm, and compiler remain the same. Yet the results often do.

Sometimes the difference is barely noticeable. Other times, a circuit that looked perfect in simulation suddenly produces outputs that are difficult to explain. As your circuits become deeper, involve more qubits, or include more gates, those differences become increasingly significant.

When I first encountered this behavior, my instinct was the same as many beginners: I must have made a mistake somewhere.

I reviewed my code, checked my gates, and compared the circuit diagrams. I reran the simulator. Everything looked correct. The problem wasn't the algorithm. It was the hardware.

Unlike the ideal environment simulated by Qiskit Aer, real quantum processors operate in a world filled with imperfections. Qubits gradually lose their quantum information. Gates are never perfectly accurate. Measurements introduce uncertainty. Even qubits waiting for their turn in a computation continue interacting with their environment, accumulating errors before they perform another operation.

These challenges are collectively known as quantum noise, and they are one of the biggest obstacles preventing today's quantum computers from performing long, complex calculations reliably.

Fortunately, quantum researchers haven't been standing still. Over the years, they've developed a growing collection of techniques to reduce the impact of noise and improve the quality of quantum computations. Broadly speaking, these techniques fall into two categories:

• Error mitigation, which estimates and compensates for errors after a circuit has executed.

• Error suppression, which attempts to prevent many of those errors from occurring in the first place while the circuit is running.

More recently, these advanced techniques have started becoming accessible through developer-friendly tools instead of requiring researchers to manually tune every circuit.

One of the newest examples is Orbit, an automated quantum error suppression solution available through the Qiskit Functions Catalog. Rather than requiring developers to become specialists in techniques like dynamical decoupling, Orbit is designed to integrate advanced error suppression into existing Qiskit workflows with minimal additional effort.

But before we can appreciate why tools like Orbit matter, we first need to understand the problem they're solving.

That's exactly what we'll do in this tutorial. Instead of jumping straight into a new tool, we'll investigate one of the most common and most important questions in quantum computing:

Why do quantum circuits behave differently on real hardware than they do in a simulator?

Along the way, you'll learn where quantum errors come from, how to reproduce many of them locally using Qiskit Aer, why larger circuits become increasingly difficult to execute reliably, and how modern error suppression techniques help developers get more useful results from today's quantum computers.

By the end of this guide, you'll understand not only what causes quantum circuits to fail on real hardware, but also what developers can do about it.

Table of Contents

• The Experiment: Running the Same Circuit in a Simulator and on Real Hardware

  • Starting with a Familiar Circuit

  • Step 1: Running the Circuit on the Simulator

  • Step 2: Running the Same Circuit on a Real Quantum Computer

• What Happens Inside a Real Quantum Computer?

  • From Python Code to Physical Qubits

  • Every Quantum Operation Is a Physical Process

  • What Is Quantum Noise?

  • Four Common Sources of Quantum Noise

  • Why Simulators Don't Show These Problems

• Simulating Quantum Noise with Qiskit Aer

  • Creating a Simple Noise Model

  • Running the Bell State with Noise

  • Comparing the Results

  • Making the Noise Worse

  • Why Not Just Remove the Noise?

• Error Mitigation vs. Error Suppression: What's the Difference?

  • What Is Error Mitigation?

  • What Is Error Suppression?

  • Comparing the Two Approaches

  • Why Error Suppression Is Becoming More Important

  • Introducing Dynamical Decoupling

  • Where Orbit Fits

• How Automated Error Suppression Fits into a Modern Quantum Workflow

  • Moving from Manual Optimization to Automated Workflows

  • What Orbit Publicly Says It Does

  • A Real Hardware Example

  • Should You Use Orbit?

The Experiment: Running the Same Circuit in a Simulator and on Real Hardware

One of the biggest advantages of learning quantum computing with Qiskit is that you don't need immediate access to a quantum computer. You can write, test, and debug your circuits locally using Qiskit Aer before running them on real IBM Quantum hardware.

Let's begin with one of the first circuits you may likely build as a quantum developer: the Bell State.

Starting with a Familiar Circuit

The Bell State is often the first example developers encounter when learning quantum programming because it demonstrates one of quantum computing's most fascinating properties: entanglement.

Create bell_state.py file:

from qiskit import QuantumCircuit

# Create a quantum circuit with two qubits and two classical bits 
qc = QuantumCircuit(2, 2)

# Place the first qubit into superposition 
qc.h(0)

# Entangle the second qubit with the first 
qc.cx(0, 1) 

# Measure both qubits 
qc.measure([0, 1], [0, 1]) 

print(qc)

In this code, the Hadamard gate places the first qubit into a superposition, while the CNOT gate entangles the second qubit with it. Once measured, both qubits should always produce matching values.

In an ideal quantum computer, you should expect only two measurement outcomes:

• 00

• 11

Each outcome should appear with roughly the same probability.

States like 01 and 10 shouldn't appear at all because they violate the expected Bell State correlations.

Step 1: Running the Circuit on the Simulator

You will begin by executing the circuit using the Qiskit Aer simulator:

from qiskit_aer import AerSimulator

simulator = AerSimulator()

result = simulator.run(
    qc,
    shots=4096
).result()

counts = result.get_counts()

print(counts)

Adding your simulator to bell_state.py, you now have:

from qiskit import QuantumCircuit
from qiskit_aer import AerSimulator

qc = QuantumCircuit(2, 2)

qc.h(0)

qc.cx(0, 1)

qc.measure([0, 1], [0, 1])

simulator = AerSimulator()

result = simulator.run(
    qc,
    shots=4096
).result()

counts = result.get_counts()

print(counts)

Make sure your virtual environment is activated (source .venv/bin/activate), and you installed Qiskit and Qiskit Aer (pip install qiskit qiskit-aer).

Run: python bell_state.py, a typical output looks like this:

{'00': 2039, '11': 2057}

Your numbers will likely be slightly different because quantum measurements are probabilistic. However, the overall pattern should remain the same.

Only 00 and 11 appear. There are no unexpected measurement outcomes, and everything behaves exactly as quantum theory predicts.

At this point, it's easy to feel confident that your circuit is correct. And it is. But there's an important detail hiding behind these perfect results.

Note: The simulator assumes an ideal quantum computer.

It doesn't have to worry about hardware limitations because it's simply calculating the mathematical evolution of your quantum state.

Among other things, the simulator assumes that:

• Every quantum gate is executed perfectly.

• Qubits never lose their quantum state.

• Measurements are always accurate.

• The environment never interferes with the computation.

• No additional noise is introduced while the circuit runs.

Those assumptions make simulators incredibly valuable for learning, debugging, and verifying quantum algorithms.

Unfortunately, real quantum processors don't operate under ideal conditions.

Step 2: Running the Same Circuit on a Real Quantum Computer

Now imagine taking this exact same circuit and executing it on a real quantum processor.

Notice that nothing changes. Not the code, algorithm, or the Bell State itself. The only thing we're changing is where the circuit runs.

If you submit this circuit to a real quantum computer, you might expect results that closely match the simulator. After all, if the algorithm is correct, shouldn't the output be the same?

In reality, you'll often observe something more like this:

{
    '00': 1912,
    '11': 1834,
    '01': 161,
    '10': 189
}

The first thing that stands out is the appearance of two unexpected outcomes: 01 and 10.

Those states weren't present in the simulator. So where did they come from? The answer isn't that your code suddenly became incorrect.

The Bell State circuit hasn't changed. The simulator wasn't misleading you.

Instead, the quantum hardware is introducing small imperfections while your circuit executes.

A gate may be applied with slightly less than perfect accuracy. A qubit may begin losing its quantum information before the computation finishes. A measurement may occasionally report the wrong value.

Individually, these errors are usually very small. Collectively, they begin to change the final measurement statistics. For a simple Bell State, the differences are relatively minor.

But quantum algorithms rarely stop at two qubits and two gates.

As circuits become deeper and more complex, these small imperfections accumulate. Eventually, they can overwhelm the quantum information your algorithm is trying to preserve, making the final results less reliable.

This is one of the biggest challenges facing today's quantum computers.

A simulator shows us how a quantum algorithm is expected to behave under ideal conditions.

Real hardware shows us how that same algorithm behaves in the presence of noise. Closing that gap is one of the central goals of modern quantum computing research.

Before you explore techniques like quantum error suppression or see how tools like Orbit help automate parts of that process, you first need to understand where these errors come from.

What Happens Inside a Real Quantum Computer?

At this point, we've established something that surprises almost every new quantum developer:

The same quantum circuit can produce different results depending on where it runs.

But that naturally leads to another question:

What exactly is happening inside a real quantum computer that doesn't happen inside a simulator?

To answer that, you need to look beyond your Python code and understand what happens after you click Run.

From Python Code to Physical Qubits

When you execute a circuit with Qiskit Aer, the simulator performs mathematical calculations to determine how the quantum state evolves. It works with complex numbers and linear algebra, faithfully applying each gate exactly as quantum mechanics predicts.

Nothing interferes with the computation unless you explicitly introduce a noise model.

Real quantum computers work very differently. Instead of manipulating mathematical objects, they manipulate physical qubits.

Depending on the hardware architecture, these qubits might be:

• superconducting circuits cooled to temperatures colder than outer space

• trapped ions suspended by electromagnetic fields

• neutral atoms held in optical tweezers

• another emerging quantum technology.

Although these platforms use different hardware, they all share one important characteristic:

Qubits are extremely fragile.

Unlike classical bits, which remain either 0 or 1 until they're changed, qubits must preserve delicate quantum properties such as superposition and entanglement throughout an entire computation.

Maintaining those properties is far more difficult than it sounds.

Every Quantum Operation Is a Physical Process

When you write code like this:

qc.h(0)
qc.cx(0, 1)

It looks almost effortless. Two lines of Python, less than a second to execute.

Behind the scenes, however, the quantum processor performs a carefully orchestrated series of physical operations.

Control electronics generate microwave pulses or laser pulses. Those signals travel through specialized hardware.

The pulses interact with individual qubits for incredibly short periods of time. The timing must be extraordinarily precise.

If any part of this process deviates even slightly from what was intended, the resulting quantum state can change.

Now imagine repeating this process dozens, hundreds, or even thousands of times within a single algorithm. Tiny imperfections begin to accumulate.

Eventually, those small errors become noticeable in the final measurement results. This is what we broadly refer to as quantum noise.

What Is Quantum Noise?

This is a general term for anything that causes a quantum computer to drift away from the ideal behavior predicted by quantum mechanics.

It doesn't usually mean something dramatic has happened.

Most of the time, the errors are incredibly small.

A gate may rotate a qubit by an angle that's only slightly different from the intended value.

A qubit may lose a little of its quantum information while waiting for another operation. A measurement might occasionally report the wrong state.

Each error seems insignificant on its own. The challenge is that quantum algorithms often involve many operations.

Even tiny inaccuracies begin to add up. Imagine trying to copy a handwritten page. One typo probably doesn't matter.

Copy the same page hundreds of times, introducing one small typo during each copy, and eventually the final document barely resembles the original.

Quantum circuits behave in much the same way. The longer the computation continues, the more opportunities there are for errors to accumulate.

Four Common Sources of Quantum Noise

Although researchers study many different types of quantum errors, most developers encounter four major categories.

Understanding these will help you make sense of why quantum hardware behaves differently from an ideal simulator.

1. Decoherence

One of the biggest challenges in quantum computing is decoherence. A qubit can maintain its quantum state only for a limited amount of time. Eventually, interactions with its surrounding environment cause it to lose the information stored in its superposition.

Think of spinning a coin on a table. When you first spin it, the coin exists in a rapidly changing state that's neither clearly heads nor tails. As time passes, friction slows it down until it finally settles.

Qubits experience a similar loss of information. Except instead of friction, they're affected by tiny interactions with the surrounding environment.

If your circuit takes too long to execute, some qubits may begin losing their quantum information before the computation finishes.

2. Gate Errors

Every quantum gate is a physical operation. Ideally, a Hadamard gate always performs exactly the same transformation. In reality, no hardware is perfect.

The pulse implementing the gate may be slightly stronger, weaker, or slightly delayed than intended. These tiny inaccuracies create gate errors.

One imperfect gate isn't usually a problem, hundreds of imperfect gates quickly become one

This is one reason deeper quantum circuits tend to perform worse than shallow ones.

3. Measurement Errors

Even if your computation completes successfully, there's still one final challenge:

Reading the result.

Measuring a qubit is itself a physical process. Sometimes the hardware incorrectly identifies a qubit as 1 when it should be 0, or vice versa.

Imagine stepping on a bathroom scale that occasionally reports your weight two kilograms heavier than it actually is.

The measurement instrument — not you — is introducing the error.

Quantum computers face a similar problem when reading qubit states.

4. Idle Errors

One of the least intuitive sources of quantum noise occurs when a qubit isn't doing anything at all.

Suppose one qubit is waiting while another qubit is being measured or participating in a multi-qubit operation.

Although it appears idle, it doesn't freeze in time. The qubit continues interacting with its environment. During that waiting period, it can gradually lose coherence.

As quantum circuits become larger, these idle periods become more common.

Reducing the impact of these waiting times is one of the motivations behind advanced error suppression techniques such as dynamical decoupling — a technique we'll explore later when we discuss Orbit.

Why Simulators Don't Show These Problems

If you've only worked with Qiskit Aer so far, you may wonder why you've never encountered any of these issues.

The answer is simple.

By default, the simulator isn't trying to model an imperfect quantum computer. It's trying to model an ideal one.

That makes it an excellent learning environment because you can verify whether your algorithm is logically correct without worrying about hardware limitations.

But it also means a simulator can't fully prepare you for what happens on real quantum devices.

To understand that difference, you need to recreate it yourself.

Fortunately, Qiskit gives us a way to do exactly that.

Instead of waiting until you have access to a real quantum computer, you can intentionally introduce realistic noise into your local simulator and observe how your Bell State begins to change.

Simulating Quantum Noise with Qiskit Aer

So far, you've compared two different worlds.

In the first world, our Bell State circuit runs inside an ideal simulator, where every quantum operation is mathematically perfect.

In the second world, that same circuit runs on a real quantum processor, where qubits are constantly affected by noise from their surrounding environment.

The obvious challenge is this:

What if you don't have access to a quantum computer?

Can you still learn how noise affects your algorithms? Fortunately, you can.

One of Qiskit's most useful features is its ability to simulate realistic hardware imperfections locally using Qiskit Aer. Instead of waiting until your circuit reaches a real quantum processor, you can inject different kinds of noise into your simulator and observe how those imperfections influence the final results.

This allows you to experiment, debug, and better understand the behavior of quantum algorithms — all from your own computer.

Let's see how it works.

Creating a Simple Noise Model

Qiskit Aer includes a collection of tools for building custom noise models. These models let you simulate many of the errors you've just learned about, including gate errors, measurement errors, and qubit decoherence.

For your first experiment, keep things simple by introducing a small amount of random error after every single-qubit and two-qubit gate:

from qiskit_aer.noise import NoiseModel, depolarizing_error

# Create an empty noise model
noise_model = NoiseModel()

# Define gate errors
single_qubit_error = depolarizing_error(0.01, 1)
two_qubit_error = depolarizing_error(0.03, 2)

# Apply errors to common quantum gates
noise_model.add_all_qubit_quantum_error(
    single_qubit_error,
    ["h", "x", "y", "z"]
)

noise_model.add_all_qubit_quantum_error(
    two_qubit_error,
    ["cx"]
)

In this code you created an empty NoiseModel and defined two depolarizing errors.

A depolarizing error is one of the most common ways to simulate hardware noise. Instead of applying a gate perfectly every time, the simulator introduces a small probability that the qubit's state becomes partially randomized.

Think of it like taking a slightly blurry photograph.

The picture still resembles the original, but every small imperfection makes it a little harder to recover the exact details.

That's essentially what depolarizing noise does to a quantum state.

Notice that we're using two different error probabilities:

• 1% for single-qubit gates

• 3% for two-qubit gates

This reflects an important reality of today's quantum hardware.

Two-qubit operations are generally more difficult to perform accurately than single-qubit operations, which is why they often have lower fidelities on real quantum processors.

Running the Bell State with Noise

Rename the bell_state.py we used earlier to bell_state_noise.py to specify adding a NoiseModel.

Reconfigure the simulator with our noise model:

from qiskit_aer import AerSimulator

noisy_simulator = AerSimulator(
    noise_model=noise_model
)

compiled = transpile(qc, noisy_simulator)

job = noisy_simulator.run(
    compiled,
    shots=4096
)

result = job.result()

counts = result.get_counts()

print(counts)

At this point your bell_state_noise.py should look like this:

from qiskit import QuantumCircuit
from qiskit_aer import AerSimulator
from qiskit_aer.noise import NoiseModel, depolarizing_error

# Step 1: Build the Bell State circuit
qc = QuantumCircuit(2, 2)

# Put qubit 0 into superposition
qc.h(0)

# Entangle qubit 1 with qubit 0
qc.cx(0, 1)

# Measure both qubits
qc.measure([0, 1], [0, 1])

print("Bell State Circuit")
print(qc)


# Step 2: Run on the ideal simulator

ideal_simulator = AerSimulator()

ideal_result = ideal_simulator.run(
    qc,
    shots=4096
).result()

ideal_counts = ideal_result.get_counts()

print("\nIdeal Simulator Results")
print(ideal_counts)


# Step 3: Create a noise model
noise_model = NoiseModel()

single_qubit_error = depolarizing_error(0.01, 1)
two_qubit_error = depolarizing_error(0.03, 2)

noise_model.add_all_qubit_quantum_error(
    single_qubit_error,
    ["h", "x", "y", "z"]
)

noise_model.add_all_qubit_quantum_error(
    two_qubit_error,
    ["cx"]
)

# Step 4: Run with simulated noise
noisy_simulator = AerSimulator(
    noise_model=noise_model
)

noisy_result = noisy_simulator.run(
    qc,
    shots=4096
).result()

noisy_counts = noisy_result.get_counts()

print("\nNoisy Simulator Results")
print(noisy_counts)

For windows, to run:

Activate your virtual environment source .venv/Scripts/activate then run python bell_state_noise.py

You may see output similar to this:

Your exact numbers will be different, but one thing should immediately stand out.

Unlike the ideal simulator, two unexpected states have appeared:

• 01

• 10

These outcomes shouldn't exist in a perfect Bell State.

Yet they now appear because we intentionally introduced hardware imperfections into the simulation.

Without changing a single line of our quantum algorithm, the results became noticeably less reliable.

Comparing the Results

Let's compare all three scenarios we've discussed so far.

The noisy simulator isn't trying to perfectly reproduce a specific IBM Quantum processor. Instead, it helps you understand how quantum noise changes the behavior of an algorithm.

That's an important distinction. You're no longer asking whether your Bell State circuit is correct. You already know it is.

Instead, you're asking a new question:

How resilient is my circuit when the hardware isn't perfect?

That's the kind of question quantum developers ask every day.

Making the Noise Worse

To see how quickly errors accumulate, try increasing the depolarizing probabilities.

For example, change the code to:

single_qubit_error = depolarizing_error(0.05, 1)
two_qubit_error = depolarizing_error(0.10, 2)

Run the circuit again.

You'll likely notice that the incorrect outcomes become much more common.

The Bell State begins to lose its characteristic correlation, and the measurement distribution drifts farther away from the ideal 50/50 split.

This simple experiment illustrates an important principle of quantum computing.

Small increases in hardware noise can have a surprisingly large impact on the quality of your results.

Now imagine running a circuit containing hundreds of gates instead of just two.

Each additional operation introduces another opportunity for error.

By the time the computation finishes, the accumulated noise may overwhelm the useful quantum information your algorithm was trying to preserve.

This is why reducing noise has become one of the biggest priorities in quantum computing.

Why Not Just Remove the Noise?

At this point, you might wonder:

If noise causes so many problems, why can't you simply eliminate it?

Researchers have been working toward that goal for decades.

The challenge is that quantum systems are extraordinarily sensitive.

Completely isolating qubits from their environment while simultaneously controlling and measuring them is one of the hardest engineering problems in modern science.

Instead of waiting for perfect hardware, researchers have developed techniques that help quantum computers produce more reliable results even when noise is unavoidable. These techniques fall into two categories as mentioned: Error mitigation* and *Error suppression

Although both approaches aim to improve the quality of quantum computations, they solve the problem in fundamentally different ways.

Understanding that distinction is essential before we explore how Orbit brings automated error suppression into modern Qiskit workflows.

Error Mitigation vs. Error Suppression: What's the Difference?

After seeing how even a small amount of noise can change the outcome of a simple Bell State circuit, it's natural to ask an important question:

If quantum hardware is so noisy, how do researchers still run useful quantum algorithms?

The answer is that they rarely rely on raw hardware results alone. Instead, they use error mitigation and error suppression to improve the quality of quantum computations.

Although these terms are sometimes used interchangeably, they solve two different problems.

Understanding the difference is essential because Orbit belongs to one of these categories — not the other.

Let's look at each approach.

What Is Error Mitigation?

Imagine taking a slightly blurry photograph. Once the picture has been taken, you open an editing application to sharpen the image, adjust the colors, and reduce the blur.

You didn't prevent the camera from capturing a blurry image. Instead, you improved the image after it was captured.

That's essentially what error mitigation does.

Error mitigation doesn't stop errors from occurring while the quantum circuit runs. Instead, it uses mathematical and statistical techniques to estimate how much noise affected the computation and then attempts to compensate for it after execution.

The goal isn't to create a perfect quantum computer. The goal is to extract a better approximation of the correct answer from imperfect hardware.

A simplified workflow looks like this:

Write Circuit
       ↓
Run on Noisy Hardware
       ↓
Collect Results
       ↓
Estimate Hardware Errors
       ↓
Correct the Final Output

This approach has become an important part of today's quantum computing landscape because it doesn't require fault-tolerant quantum hardware.

Instead, it works with the devices we have today.

Some common error mitigation techniques include:

• Measurement error mitigation

• Zero-noise extrapolation (ZNE)

• Probabilistic error cancellation (PEC)

• Clifford data regression (CDR)

You don't need to understand these techniques in detail right now.

The important takeaway is that error mitigation tries to improve the final answer after the computation has already finished.

What Is Error Suppression?

Error suppression takes a very different approach.

Instead of correcting errors after the circuit finishes, it tries to prevent many of those errors from happening in the first place.

Imagine you're hiking through a muddy trail. Error mitigation is like cleaning your boots after the hike. Error suppression is like wearing waterproof boots before you start walking.

Both approaches improve the final outcome. One acts after the problem occurs. The other acts during the journey to reduce the problem altogether.

A simplified workflow looks like this:

Write Circuit
      ↓
Reduce Noise During Execution
      ↓
Execute Circuit
      ↓
Measure Results

Instead of estimating corrections afterward, error suppression focuses on protecting fragile quantum information while the computation is taking place.

This often involves techniques that reduce the impact of environmental noise, improve gate execution, or protect qubits during idle periods.

One of the best-known examples is dynamical decoupling, a technique you'll explore shortly

Comparing the Two Approaches

Although both methods improve quantum computations, they operate at different stages of the workflow.

Neither approach completely eliminates quantum noise.

Instead, they complement each other.

In fact, you'll often get better results by combining both techniques

Why Error Suppression Is Becoming More Important

As quantum algorithms become larger, the number of opportunities for noise to accumulate also increases.

Imagine a circuit containing only two gates, a tiny error may have almost no noticeable effect.

Now imagine a circuit containing hundreds or thousands of gates. Those same tiny errors can accumulate until the final result becomes unreliable.

This is especially challenging for algorithms that require qubits to remain coherent over longer periods or spend time waiting while other operations complete.

In these situations, reducing noise during execution becomes increasingly valuable.

Rather than trying to recover lost information afterward, researchers look for ways to preserve that information before it disappears.

That's where error suppression techniques have attracted significant attention.

Introducing Dynamical Decoupling

This is one of the most widely studied error suppression techniques. The name sounds intimidating, but the underlying idea is surprisingly intuitive.

Imagine balancing a broomstick upright on your hand. If you leave your hand perfectly still, the broomstick quickly falls over. But if you make small, carefully timed adjustments, you can keep it balanced much longer.

You're not changing the broomstick. You're continually making tiny corrections that prevent small disturbances from growing into larger problems.

Dynamical decoupling works in a similar way.

While a qubit is temporarily idle, carefully chosen pulse sequences are applied to help reduce the effects of environmental noise and preserve its quantum state for longer.

The underlying theory has been studied for decades and has become one of the foundational techniques in quantum error suppression research.

However, applying these techniques hasn't always been straightforward.

Developers often needed specialized knowledge to determine when and where these pulse sequences should be inserted into a circuit.

For many software developers, that level of hardware expertise sits well outside their day-to-day workflow.

Where Orbit Fits

This brings us to the motivation behind Orbit.

Rather than expecting every developer to become an expert in dynamical decoupling and other advanced error suppression techniques, Orbit is designed to make those capabilities more accessible through a familiar Qiskit workflow.

Conceptually, the workflow changes from this:

Write Circuit
     ↓
Manually Analyze Idle Periods
     ↓
Design Error Suppression Strategy
     ↓
Modify Circuit
     ↓
Execute on Hardware

to something much simpler:

Write Circuit
     ↓
Orbit Applies Error Suppression
     ↓
Execute on Hardware

Notice what hasn't changed. You still design your quantum algorithm. You still write your Qiskit circuit. You still execute it on quantum hardware.

The difference is that the error suppression strategy can become part of the workflow instead of another manual optimization task.

In other words, Orbit isn't trying to replace Qiskit.

It's designed to help developers get more reliable results from the quantum circuits they already know how to build.

How Automated Error Suppression Fits into a Modern Quantum Workflow

By this point, we've established two important ideas.

First, today's quantum computers are inherently noisy. As circuits become larger and more complex, even small hardware imperfections accumulate and reduce the quality of the final results.

Second, developers have two broad ways to deal with that noise: error mitigation, which improves results after execution, and error suppression, which attempts to reduce errors while the circuit is running.

The obvious question now is:

How do developers actually apply error suppression in practice?

Historically, the answer hasn't been particularly simple.

Many error suppression techniques require a deep understanding of quantum hardware. Developers often need to analyze their circuits, identify where qubits remain idle, experiment with different optimization strategies, and repeatedly execute the circuit to determine which approach produces the best results.

That process can be both time-consuming and highly specialized.

Even worse, a strategy that improves one circuit may provide little benefit for another.

As Quantum Elements explains in its recent technical blog, developers often end up repeating a cycle of testing, tuning, and rerunning experiments because there isn't a one-size-fits-all solution to quantum noise.

Moving from Manual Optimization to Automated Workflows

Modern software development has steadily moved toward automation.

We use formatters instead of manually adjusting indentation. We use linters instead of searching for style issues ourselves. We use CI/CD pipelines instead of deploying applications by hand.

Quantum software is beginning to follow the same pattern.

Instead of asking every developer to become an expert in hardware-aware optimization techniques, newer tools aim to automate parts of that workflow while allowing developers to continue writing standard Qiskit circuits.

One example is Orbit, which Quantum Elements recently made available as a Qiskit Function for IBM Quantum Network members.

Conceptually, the workflow changes from something like this:

Write Quantum Circuit
        ↓
Study Hardware Characteristics
        ↓
Experiment with Error Suppression
        ↓
Modify Circuit
        ↓
      Execute

To a simpler workflow:

Write Quantum Circuit
        ↓
Apply Automated Error Suppression
        ↓
      Execute

The important thing to notice is that your algorithm doesn't change.

You still design the circuit and write Qiskit code. The goal is to make advanced optimization techniques easier to integrate into an existing development workflow.

What Orbit Publicly Says It Does

Quantum Elements has shared a high-level overview of how Orbit works without disclosing its proprietary implementation.

Orbit accepts an existing Qiskit circuit through the Qiskit Functions interface and prepares it for execution by applying a combination of techniques that may include:

• circuit-level optimization during transpilation,

• measurement error mitigation, and

• advanced dynamical decoupling sequences inserted during idle periods where qubits would otherwise accumulate additional noise.

Notice that none of these techniques require developers to redesign their algorithms from scratch.

Instead, the emphasis is on improving how an existing circuit executes on today's quantum hardware.

Exactly how those optimizations are chosen internally is part of Orbit's implementation, but from a developer's perspective the workflow remains familiar:

1. Build your quantum circuit.

2. Submit it through the supported workflow.

3. Execute the optimized circuit on compatible IBM Quantum hardware.

A Real Hardware Example

So far, you've seen how noise affects a simple Bell-state circuit. But the real challenge appears when circuits become larger and qubits spend more time waiting for other operations to finish.

That's exactly the kind of situation Quantum Elements used in a recent public benchmark for Orbit.

In the experiment, the circuit was executed on IBM's ibm_aachen quantum processor. The goal wasn't to show a completely different quantum algorithm. It was to test what happens when a circuit contains more operations, more waiting periods, and more opportunities for noise to accumulate.

As circuits grow, some qubits often remain idle while other qubits are being measured or processed. Earlier in this article, you learned that idle qubits don't freeze in time. They continue interacting with their environment, and that interaction can gradually destroy the quantum information you're trying to preserve.

According to Quantum Elements' published benchmark, Orbit applies error-suppression techniques during these idle periods and combines them with other circuit-level optimizations.

The company compared three versions of the same workload:

• a standard implementation,

• a dynamic implementation without additional protection, and

• the dynamic implementation with Orbit enabled.

The reported results showed that the protected version maintained stronger performance across multiple runs on ibm_aachen.

Quantum Elements also reported an increase in the effective qubit lifetime for this particular experiment, which allowed larger versions of the circuit to remain usable for longer.

The important takeaway isn't that every quantum circuit will improve by the same amount.

The more useful lesson is the one you've been building throughout this tutorial:

As quantum circuits become larger and qubits spend more time idle, reducing the accumulation of noise becomes just as important as designing the algorithm itself.

That's why automated error suppression is becoming an increasingly interesting part of modern quantum software workflows. Instead of manually analyzing every idle period and tuning every optimization yourself, tools such as Orbit aim to make those hardware-aware improvements easier to apply to circuits you've already written in Qiskit.

Should You Use Orbit?

If you're just beginning your quantum-computing journey, probably not yet.

Your time is better spent learning how quantum circuits work, becoming comfortable with Qiskit, and understanding concepts such as superposition, entanglement, quantum noise, and circuit depth.

However, once you start running larger circuits on IBM Quantum hardware, you'll likely encounter situations where noise becomes a practical limitation rather than just a theoretical concept.

That's the kind of workflow automated error-suppression tools are designed to support.

At the time of writing, Quantum Elements is offering developers three months of complimentary access to Orbit for eligible users through a request process. If you're already experimenting with IBM Quantum hardware and would like to evaluate how automated error suppression fits into your workflow, you can request access from Quantum Elements.

Whether you eventually use Orbit or another solution, the bigger lesson remains the same:

Writing a correct quantum algorithm is only part of the challenge. Learning how that algorithm behaves on real quantum hardware — and learning how to reduce the impact of noise — is becoming an increasingly important skill for every quantum developer.

So I built and open-sourced a complete SaaS landing page template called ChatDeck. It runs on Next.js 16, React 19, shadcn/ui with the new base-nova style, Tailwind CSS v4, and TypeScript. The full source is on GitHub under the MIT license. I built and open-sourced this template, and everything here comes from decisions made during that process.

Building it forced me to make real decisions on a stack that moved significantly in the past 12 months. This article is about those decisions: what worked, what didn't, and what I'd do differently if I started today.

Prerequisites: This article assumes you're comfortable with React and TypeScript. Some familiarity with the Next.js App Router is helpful but not required. Each lesson is explained from first principles.

Table of Contents

• The Stack Choices and Why They Matter

• Getting Started

• Project Structure

• Lesson 1: shadcn/ui's Newbase-novaStyle Changes What "Accessible" Means

• Lesson 2: Tailwind CSS v4 Requires a Mental Model Shift

• Lesson 3: OKLCH Colors Make Dark Mode Predictable

• Lesson 4: Page Architecture — Flat Beats Clever

• Lesson 5: Staggered Animations Without Managing Individual Delays

• Lesson 6: CSS-Only Infinite Scroll — No Library Needed

• Lesson 7: CSS Subgrid Solves Pricing Card Alignment Natively

• Lesson 8: Inline SVGs Beat Image Libraries for Simple Logos

• What I'd Do Differently

• Summary

The Stack Choices and Why They Matter

Before getting into the code, here's what the template runs on. Each choice was deliberate — none of these are defaults you get from create-next-app.

The most interesting decisions in this list are the ones that reflect how the ecosystem changed in the past year: Base UI replacing Radix inside shadcn/ui, and Tailwind v4's shift to CSS-first configuration. The lessons below walk through each of these in detail, starting with the choices that had the biggest impact on how the code is actually written.

Getting Started

Before diving into the lessons, here's how to get the project running locally. Having it open alongside this article makes the code examples easier to follow.

git clone https://github.com/ShadcnDeck/chatdeck-shadcn-saas-landing-page-template.git
cd chatdeck-shadcn-saas-landing-page-template
pnpm install
pnpm dev

Open http://localhost:3000 and you'll see the full landing page running locally.

All section content lives as plain TypeScript arrays inside each Block component. To change the features, edit the features array in FeatureSection.tsx. To change pricing tiers, edit the plans array in PricingSection.tsx. No CMS, no config files — just TypeScript objects.

To customize colors, update the OKLCH values in app/globals.css under the :root block. Change --primary and every button, link, and accent color updates across the entire template.

Deploy to Vercel with a single vercel command or by pushing to GitHub and connecting the repo. Next.js is detected automatically.

Project Structure

Here's the full directory layout before we go through each part of it:

chatdeck/
├── app/
│   ├── globals.css         # Theme tokens + custom animations (Tailwind v4 @theme)
│   ├── layout.tsx          # Root layout — Navbar, Footer, fonts
│   └── page.tsx            # Section imports — 16 lines
├── components/
│   ├── Blocks/             # Page sections (Hero, Features, Pricing, etc.)
│   ├── ui/                 # shadcn/ui components — base-nova style
│   └── navbar.tsx          # Scroll-aware sticky navbar
└── lib/
    └── utils.ts            # cn() helper (clsx + tailwind-merge)

The key separation is Blocks/ vs ui/. The ui/ folder holds primitive components — Button, Badge, Accordion — that come from shadcn/ui and rarely change. The Blocks/ folder holds page-level sections that are specific to this template and change often. When you're customising, you mostly work in Blocks/. When you upgrade shadcn/ui components, you touch ui/.

The lessons below go through specific files in this structure piece by piece: components.json and ui/accordion.tsx in Lesson 1, app/globals.css in Lessons 2 and 3, app/page.tsx in Lesson 4, and the individual Block components in Lessons 5 through 8.

Lesson 1: shadcn/ui's New base-nova Style Changes What "Accessible" Means

If you've used shadcn/ui before, you know the default setup uses Radix UI primitives, headless components that handle focus management, keyboard navigation, and ARIA attributes. Radix has been the default for years.

But shadcn/ui introduced a new style in 2025 called base-nova, which replaces Radix with Base UI, the headless primitive library from MUI.

Based on shadcn's public direction and the components released through 2025, base-nova appears to be the intended default going forward (though shadcn hasn't yet deprecated the Radix style).

In the project's components.json:

{
  "$schema": "https://ui.shadcn.com/schema.json",
  "style": "base-nova",
  "rsc": true,
  "tsx": true,
  "tailwind": {
    "css": "app/globals.css",
    "baseColor": "neutral",
    "cssVariables": true
  },
  "iconLibrary": "lucide"
}

The "style": "base-nova" line means every component the shadcn/ui CLI installs wraps Base UI primitives instead of Radix. To understand what this changes in practice, here's what the same Accordion trigger component looks like in the older Radix-based default style:

// Radix-based default style (the old way)
import * as AccordionPrimitive from "@radix-ui/react-accordion"

const AccordionTrigger = React.forwardRef<
  React.ElementRef<typeof AccordionPrimitive.Trigger>,
  React.ComponentPropsWithoutRef<typeof AccordionPrimitive.Trigger>
>(({ className, children, ...props }, ref) => {
  const [isOpen, setIsOpen] = React.useState(false)

  return (
    <AccordionPrimitive.Header className="flex">
      <AccordionPrimitive.Trigger
        ref={ref}
        className={cn("flex flex-1 items-center justify-between ...", className)}
        onClick={() => setIsOpen(!isOpen)}
        {...props}
      >
        {children}
        <ChevronDownIcon
          className={cn(
            "h-4 w-4 shrink-0 transition-transform duration-200",
            isOpen ? "hidden" : "block"
          )}
        />
        <ChevronUpIcon
          className={cn(
            "h-4 w-4 shrink-0 transition-transform duration-200",
            isOpen ? "block" : "hidden"
          )}
        />
      </AccordionPrimitive.Trigger>
    </AccordionPrimitive.Header>
  )
})

Notice the useState(false) tracking whether the accordion is open, and the onClick handler that toggles it. This means the component has to manually keep its own isOpen state in sync with what Radix internally knows about the open/closed state.

Now here's the same component using the base-nova style with Base UI:

// components/ui/accordion.tsx — base-nova style (the new way)
import { Accordion as AccordionPrimitive } from "@base-ui/react/accordion"

function AccordionTrigger({ className, children, ...props }: AccordionPrimitive.Trigger.Props) {
  return (
    <AccordionPrimitive.Header className="flex">
      <AccordionPrimitive.Trigger
        data-slot="accordion-trigger"
        className={cn(
          "group/accordion-trigger relative flex flex-1 items-start ...",
          className
        )}
        {...props}
      >
        {children}
        <ChevronDownIcon
          className="pointer-events-none shrink-0 group-aria-expanded/accordion-trigger:hidden"
        />
        <ChevronUpIcon
          className="pointer-events-none hidden shrink-0 group-aria-expanded/accordion-trigger:inline"
        />
      </AccordionPrimitive.Trigger>
    </AccordionPrimitive.Header>
  )
}

No useState. No onClick. No isOpen variable. The chevron visibility is controlled entirely by group-aria-expanded/accordion-trigger:hidden — a Tailwind class that reads the aria-expanded attribute Base UI sets automatically on the trigger element.

The lesson here: in the Radix version, you have two parallel systems: the component's own isOpen state, and the ARIA attributes that the library manages separately for screen readers. These can drift out of sync — for example, if the accordion closes via keyboard navigation, the ARIA state updates correctly but your isOpen state doesn't unless you wire up the right callbacks. In the Base UI version, there is only one system. ARIA state IS the state. Tailwind reads it directly. There's nothing to keep in sync and nothing that can drift.

Lesson: use the primitive library's ARIA attributes as your source of truth for visual state. If your headless component library already sets aria-expanded, aria-selected, or aria-checked, Tailwind can respond to those directly with aria-* variant classes — no parallel JavaScript state needed.

So when you install shadcn/ui today, choose base-nova over the default Radix style. You get tighter Base UI integration, a smaller peer dependency footprint, and components that are more aligned with where the ecosystem is moving.

Lesson 2: Tailwind CSS v4 Requires a Mental Model Shift

Tailwind CSS v4 moves primary theme configuration out of the JavaScript config file and into CSS. This sounds small. In practice, it changes how you think about the entire theming system.

In Tailwind v3, you'd extend the theme in tailwind.config.js:

// OLD — tailwind.config.js (v3)
module.exports = {
  theme: {
    extend: {
      animation: {
        marquee: "marquee 40s linear infinite",
      },
      keyframes: {
        marquee: {
          from: { transform: "translateX(0)" },
          to: { transform: "translateX(calc(-100% - var(--gap)))" },
        },
      },
    },
  },
}

In Tailwind v4, that same configuration lives in your CSS file instead:

/* app/globals.css — Tailwind v4 */
@import "tailwindcss";

@theme inline {
  --animate-marquee: marquee var(--duration) infinite linear;
  --animate-marquee-vertical: marquee-vertical var(--duration) linear infinite;

  @keyframes marquee {
    from { transform: translateX(0); }
    to   { transform: translateX(calc(-100% - var(--gap))); }
  }

  --radius-2xl: calc(var(--radius) * 1.8);
  --radius-3xl: calc(var(--radius) * 2.2);
  --radius-4xl: calc(var(--radius) * 2.6);
}

The @theme inline block extends Tailwind's design token system. Define --animate-marquee here and you can use className="animate-marquee" anywhere in your components. Tailwind generates the utility class automatically from the CSS variable.

Custom animations, radius scales, and color tokens all live in CSS now. The benefit is that CSS is where styles belong. The config file was always an indirection layer between "what I want my design system to look like" and "where that actually lives." Tailwind v4 removes the indirection.

The friction: if you start a Tailwind v4 project with a v3 mental model, you'll spend time looking for theme config in the wrong place. Read the v4 migration guide before you start, not after you're confused.

Lesson: move your mental model of "theme config" from JavaScript to CSS. In Tailwind v4, if you want a custom animation, a new radius scale, or a color token, define it in @theme inline inside globals.css. That's where it belongs, and that's where every developer on your team will find it.

Lesson 3: OKLCH Colors Make Dark Mode Predictable

The template uses OKLCH color values throughout, not hex or HSL:

:root {
  --background: oklch(1 0 0);        /* white */
  --foreground: oklch(0.145 0 0);    /* near-black */
  --primary: oklch(0.205 0 0);
  --border: oklch(0.922 0 0);
}

.dark {
  --background: oklch(0.145 0 0);    /* near-black */
  --foreground: oklch(0.985 0 0);    /* near-white */
  --primary: oklch(0.922 0 0);
  --border: oklch(1 0 0 / 10%);      /* white at 10% opacity */
}

OKLCH is a perceptually uniform color space. When you increase the lightness value in OKLCH, the color actually looks lighter to human eyes, consistently. Hex and HSL don't guarantee this. You can increase the L in HSL and get a color that looks the same or even darker depending on the hue.

For dark mode specifically, this matters because you're inverting a whole color system. With HSL, you'll often end up manually tweaking individual color values until contrast ratios look right. With OKLCH, increasing or decreasing the lightness value gives you predictable results across all your tokens.

The dark mode switch itself is zero JavaScript. Adding class="dark" to the <html> element swaps every CSS variable. Tailwind reads the updated variables and re-renders every component. There's no context provider and no useTheme hook needed for the CSS layer — just a class toggle on the root element.

Lesson: swap your color tokens to OKLCH. When defining dark mode values, adjust the first OKLCH parameter (lightness) and the result will look predictably lighter or darker. With hex or HSL you're often guessing; with OKLCH you're reasoning.

Lesson 4: Page Architecture — Flat Beats Clever

The main page file is 16 lines:

// app/page.tsx
import Hero from "@/components/Blocks/Hero";
import { LogoCarousel } from "@/components/Blocks/LogoCarousel";
import { FeatureSection } from "@/components/Blocks/FeatureSection";
import { TeamSection } from "@/components/Blocks/TeamSection";
import { TestimonialSection } from "@/components/Blocks/TestimonialSection";
import { PricingSection } from "@/components/Blocks/PricingSection";
import { FaqSection } from "@/components/Blocks/FaqSection";

export default function Home() {
  return (
    <main className="min-h-screen bg-white dark:bg-black">
      <div className="mx-auto max-w-7xl px-6 pt-40">
        <Hero />
        <LogoCarousel />
        <FeatureSection />
        <TeamSection />
        <TestimonialSection />
        <PricingSection />
        <FaqSection />
      </div>
    </main>
  );
}

No dynamic imports, no lazy-loading config, no context providers wrapping everything. Each section is a completely self-contained component in components/Blocks/. None of them import from each other.

This decision came from watching how developers actually use shadcn templates. The first thing anyone does after cloning is delete the sections they don't need and reorder the ones they keep. With flat imports, removing the Team section is one deleted line. Reordering sections is moving one line. Adding a new section is creating a file and adding one import.

The alternative (a sections array, a renderer loop, a config file that controls order) sounds sophisticated. In practice, it adds indirection that makes the template harder to understand and slower to customize. Templates should be obvious, not impressive.

Lesson: in a template context, the simplest architecture is the correct architecture. The developer cloning your template isn't impressed by abstraction. They want to understand the code fast and change it faster.

Lesson 5: Staggered Animations Without Managing Individual Delays

The Hero section uses entrance animations where each element fades up sequentially: badge first, then heading, then subheading, then CTA. The naïve approach sets a different delay prop on each element manually. The correct approach uses staggerChildren:

// components/Blocks/Hero.tsx
"use client"
import { motion, type Variants } from "motion/react"

const containerVariants: Variants = {
  hidden: { opacity: 0 },
  visible: {
    opacity: 1,
    transition: {
      staggerChildren: 0.15,  // each child animates 150ms after the previous
      delayChildren: 0.1,
    },
  },
}

const fadeUpVariants: Variants = {
  hidden: { opacity: 0, y: 20 },
  visible: {
    opacity: 1,
    y: 0,
    transition: { duration: 0.5, ease: "easeOut" },
  },
}

const Hero = () => (
  <motion.div variants={containerVariants} initial="hidden" animate="visible">
    <motion.div variants={fadeUpVariants}>
      {/* Badge */}
    </motion.div>
    <motion.h1 variants={fadeUpVariants}>
      AI Chatbot for Customer Support.
    </motion.h1>
    <motion.p variants={fadeUpVariants}>
      {/* Subheading */}
    </motion.p>
    <motion.div variants={fadeUpVariants}>
      {/* CTA */}
    </motion.div>
  </motion.div>
)

The parent defines staggerChildren: 0.15. Every child with variants={fadeUpVariants} automatically inherits a 150ms offset from the previous child. Want to add a new element? Give it variants={fadeUpVariants} and the stagger chain extends automatically. No manually updated delay values.

The Features section uses scroll-triggered animations with a different easing:

// components/Blocks/FeatureSection.tsx
<motion.div
  initial={{ opacity: 0, y: 40 }}
  whileInView={{ opacity: 1, y: 0 }}
  viewport={{ once: true, amount: 0.3 }}
  transition={{
    duration: 0.5,
    delay: index * 0.15,
    ease: [0.22, 1, 0.36, 1],
  }}
>

viewport={{ once: true }} fires the animation once when the element enters the viewport, not on every scroll pass. amount: 0.3 starts the animation when 30% of the element is visible, not when the full element is on screen. The cubic bezier [0.22, 1, 0.36, 1] is a fast-out-slow-in curve that feels physical rather than mechanical.

Quick note on the import: most of the core API is compatible, but motion/react isn't a straight drop-in rename of framer-motion. If you're upgrading an existing project, check the official migration guide before swapping the import. Layout animations, AnimatePresence behaviour, and some hooks changed.

Lesson: define animation variants at the parent level and use staggerChildren to orchestrate the sequence. Never set delay manually on individual elements — that creates a brittle list of numbers you have to update every time you add or remove an element. Let the parent handle timing; let children just declare what they animate to.

Lesson 6: CSS-Only Infinite Scroll — No Library Needed

The testimonials use a dual-row auto-scrolling marquee. The second row scrolls in reverse. There's no third-party marquee package. It's a small component built entirely on CSS animations defined in Tailwind v4's @theme block.

// components/ui/marquee.tsx
export function Marquee({
  reverse = false,
  pauseOnHover = false,
  vertical = false,
  children,
  repeat = 4,
  ...props
}) {
  return (
    <div className="group flex gap-(--gap) overflow-hidden [--duration:40s] [--gap:2rem]">
      {Array(repeat).fill(0).map((_, i) => (
        <div
          key={i}
          className={cn("flex shrink-0 justify-around gap-(--gap)", {
            "animate-marquee flex-row": !vertical,
            "group-hover:paused": pauseOnHover,
            "[animation-direction:reverse]": reverse,
          })}
        >
          {children}
        </div>
      ))}
    </div>
  )
}

The repeat={4} prop renders the children 4 times side by side. As the CSS animation scrolls the container left, the repetitions create a seamless loop. By the time the first set has scrolled off screen, the second set is already in position.

group-hover:paused is Tailwind applying animation-play-state: paused when the parent has group class and is hovered. No onMouseEnter/onMouseLeave handlers or state, just pure CSS.

To customize the scroll speed without touching the component source, you override the CSS variable inline:

<Marquee pauseOnHover className="[--duration:20s]">
  {items.map(item => <Card key={item.id} {...item} />)}
</Marquee>

[--duration:20s] is a Tailwind arbitrary property. It sets --duration directly on the element, which the animation reads via var(--duration). Speed customization without a prop, without touching the component.

Lesson: before reaching for a third-party animation package, check whether a CSS keyframe animation and a couple of Tailwind utilities can do the same job. A marquee, a fade loop, a pulsing skeleton — all of these are achievable with native CSS. Fewer dependencies means fewer breaking changes when the ecosystem moves.

Lesson 7: CSS Subgrid Solves Pricing Card Alignment Natively

The pricing section has three cards: Free, Pro, and Business. Each card has four rows: plan name, price, CTA button, and features list. The features list height varies between plans. Without CSS subgrid, the rows don't align across cards.

The common workaround is min-height on each row, or JavaScript that measures each card and sets explicit heights. Both approaches are fragile. Subgrid solves it in CSS:

// components/Blocks/PricingSection.tsx
<div className="grid lg:grid-cols-3">
  {plans.map((plan) => (
    <div className="p-8 grid grid-rows-subgrid row-span-4 gap-6">
      <div>{/* Plan name + description */}</div>
      <div>{/* Price */}</div>
      <div>{/* CTA button */}</div>
      <div>{/* Features list */}</div>
    </div>
  ))}
</div>

grid-rows-subgrid tells each card to participate in the parent grid's row tracks rather than creating its own. Each card spans 4 rows (row-span-4). The plan name row, price row, CTA row, and features row align across all three cards (regardless of content height) because they're all on the same row tracks.

Each card's row-span-4 reserves four rows in the parent's implicit grid. Because every card spans the same four shared row tracks, their internal rows align automatically even though the parent never declares explicit row heights.

CSS subgrid has been in all modern browsers since late 2023. There's no reason to reach for a JavaScript layout solution when the platform handles it.

Lesson: when you have a grid of cards where each card has multiple internal rows that need to align across columns, reach for grid-rows-subgrid before reaching for min-height or JavaScript. Define the number of rows each card spans with row-span-N, and the browser handles the rest.

Lesson 8: Inline SVGs Beat Image Libraries for Simple Logos

The logo carousel renders 12 brand logos: Shopify, Stripe, GitHub, Google, and others. The first instinct is to use a package like react-icons or simple-icons. I went a different direction: inline SVG paths stored as a plain TypeScript object.

// components/Blocks/LogoCarousel.tsx
const iconMap = {
  stripe: "M13.976 9.15c-2.172-.806...",
  github: "M12 .297c-6.63 0-12...",
  google: "M12.48 10.92v3.28h7.84...",
  // ...
} as const

const SimpleIcon = ({ iconSlug, size = 24 }: { iconSlug: string; size?: number }) => {
  const iconPath = iconMap[iconSlug as keyof typeof iconMap]
  return (
    <svg role="img" viewBox="0 0 24 24" className="fill-black dark:fill-white">
      <path d={iconPath} />
    </svg>
  )
}

The fill-black dark:fill-white class means every logo automatically inverts in dark mode: no separate dark mode logo assets, and no conditional rendering based on theme.

The carousel itself duplicates the logo array to create a seamless loop:

{/* First pass */}
{techCompanies.map((company, i) => <LogoCard key={`first-${i}`} {...company} />)}
{/* Second pass — identical, creates the seamless loop */}
{techCompanies.map((company, i) => <LogoCard key={`second-${i}`} {...company} />)}

The CSS animation (animate-logo-scroll) scrolls the container left. When the first pass disappears off the left edge, the second pass is already in position. The loop is seamless.

The trade-off: maintaining SVG paths manually is fine for a fixed set of logos. If you need a large dynamic icon set, reach for simple-icons or a proper icon library. For 12 brand logos that rarely change, this approach ships zero extra dependencies.

Lesson: match your tooling to your actual requirements. A logo carousel with a fixed set of brand logos doesn't need an icon library — it needs a TypeScript object and two Tailwind classes. Installing a package to solve a problem you could solve with 10 lines of code adds maintenance surface for no gain.

What I'd Do Differently

These are the three decisions I'd change if starting the template today.

1. Extract Animation Variants to a Shared File

containerVariants and fadeUpVariants are currently defined locally in both Hero.tsx and FeatureSection.tsx. If you want to change the global animation timing (say, reduce duration from 0.5s to 0.3s) you update two files. A shared lib/animations.ts exporting the standard variants would make global timing changes a one-line edit.

// lib/animations.ts
export const fadeUpVariants: Variants = {
  hidden: { opacity: 0, y: 20 },
  visible: { opacity: 1, y: 0, transition: { duration: 0.5, ease: "easeOut" } },
}

export const containerVariants: Variants = {
  hidden: { opacity: 0 },
  visible: { opacity: 1, transition: { staggerChildren: 0.15, delayChildren: 0.1 } },
}

2. Use Subgrid in the Features Section Too

The Features grid uses a border-based visual separation pattern — borders between cells create the grid appearance. It works, but the hover states have an inconsistency: the gradient hover overlay height varies slightly between cells in the same row because content heights differ. Subgrid would lock those row heights across cards the same way it does in the Pricing section.

3. Use next/font More Consistently

The layout loads both Geist and Inter font families. Inter is used via --font-sans. Geist is loaded but the geistSans.variable and geistMono.variable are applied to <body> as className strings while Inter drives the actual font rendering through the CSS variable. The result is that Geist is loaded but not actually displayed. Cleaning this up could shave tens of kilobytes from the font payload — worth verifying in Lighthouse or the Network tab before deploying.

Summary

These are the five things from this build worth taking into your next project:

1. shadcn/ui's base-nova style runs on Base UI primitives. ARIA state drives visual state — no parallel JavaScript state needed.

2. Tailwind v4 moves theme config to CSS. All theme tokens, custom animations, and radius scales live in CSS via @theme inline. This is the right place for them.

3. OKLCH gives predictable dark mode contrast. Adjusting lightness in OKLCH actually changes perceived brightness. Hex and HSL don't guarantee this.

4. staggerChildren in motion/react eliminates manually managed animation delays. The parent orchestrates while the children just declare their animation variant.

5. CSS subgrid (grid-rows-subgrid) aligns card rows across columns natively. No JavaScript measurement, no fixed heights.

The full template is MIT-licensed and available at github.com/ShadcnDeck/chatdeck-shadcn-saas-landing-page-template. If it's useful, a star helps others find it.

Between those two moments runs a long chain of machinery: builds, artefacts, migrations, health checks, traffic shifts. Every production engineer depends on that chain, and many teams still build and operate it themselves.

Deployment infrastructure has quietly become operational overhead. It started as a technical necessity, something every team had to assemble because nothing else existed.

Today it is a second system your engineers maintain alongside the product, consuming on-call rotations, sprint capacity, and 2 a.m. attention that could go somewhere better.

In this article, we'll walk through each stage of a real production deployment: the build, the artefact it produces, database migrations, health checks, rolling updates, and rollbacks. Along the way, we'll look at why platform-as-a-service (PaaS) tools handle most of these steps for you, and what it costs a team to keep handling them itself.

Table of Contents

• The Build: Turning Code into Something That Can Run

• The Artefact: One Version, Frozen in Time

• Database Migrations: The Riskiest Step

• Health Checks: Proving the New Version Is Alive

• Rolling Updates: Replacing the Plane's Engine Mid-Flight

• Rollbacks: The Escape Hatch

• When You Don't Need a PaaS

• Should You Still Be Running This Yourself?

The Build: Turning Code into Something That Can Run

A deployment does not ship your source code as-is. It ships the result of a build. The build stage takes your code and turns it into something a server can run.

What this looks like depends on your stack. A Java or Go project gets compiled into a binary. A JavaScript front end gets bundled and minified. A Python app gets its dependencies resolved and pinned. In most modern setups, all of this gets packed into a container image, which is a frozen snapshot of your app plus everything it needs to run.

The build stage also runs your tests. Unit tests, linting, and security scans all happen here. If any of them fail, the deployment stops before it can touch production. This is the cheapest place to catch a bug. A failed build costs you a few minutes. A failed deployment can cost you customers.

Teams that run their own pipelines spend real effort here. They maintain build servers, cache dependencies, and debug flaky test runners.

None of that work ships a feature. It is pure upkeep, and it never ends. A PaaS bakes this whole stage into the platform. You push code, and the platform detects your language, builds it the same way every time, and fails fast when something is wrong. The build still happens. Your engineers just stop paying for it in hours.

The Artefact: One Version, Frozen in Time

The output of a build is called an artefact. It might be a container image, a compiled binary, or a zipped bundle. Whatever the format, the artefact has one job: to be exact. It represents one precise version of your app, frozen at one point in time.

This matters more than it sounds. The artefact that passed your tests must be the exact same one that reaches production. If you rebuild between testing and shipping, you risk shipping something slightly different. A dependency may have updated. A build flag may have changed. "It worked in staging" often means "we built it twice and got two different results."

Good pipelines build once and promote the same artefact through every stage. Artefacts get versioned and stored in a registry, so any version can be pulled and run again later. That stored history is also what makes rollbacks possible, which we will get to soon.

On a PaaS, artefact handling is standard practice by default. Every deploy produces a numbered release. The platform stores it, tracks it, and can restore it. You do not have to design a registry strategy, write promotion scripts, or assign an engineer to own them. The discipline is built in.

Database Migrations: The Riskiest Step

Before new code goes live, the database often has to change with it. Maybe the new version needs a new column or a new table. These changes are called migrations, and they are the most dangerous part of most deployments.

Why? Code is easy to replace. Data is not. If you deploy a bad code version, you can swap it out. If a migration corrupts or drops data, there may be no clean way back. Migrations also create a tricky window of time. For a few minutes, old code and new code may run against the same database at once. Both versions have to work with the schema during that window.

The safe pattern is to make migrations backwards-compatible. Add the new column first, deploy code that can handle both shapes, then clean up the old column in a later release. It takes more steps, but each step is safe on its own.

A PaaS cannot write your migrations for you. No tool can know what your data means. But a good platform gives migrations a defined place in the release process, runs them in order, and logs exactly what ran and when. That structure prevents the classic failure where someone runs a migration by hand and forgets to tell the team.

Health Checks: Proving the New Version Is Alive

Once the new version starts, the platform does not just trust it. It checks. A health check is a small endpoint in your app, often just a route that returns "OK." The platform calls it over and over. If the app answers, it is considered healthy. If it does not, the platform assumes something is wrong.

There are usually two kinds of checks. A readiness check asks, "Are you ready to receive traffic?" A liveness check asks, "Are you still working, or should I restart you?" The difference matters. An app can be alive but not ready, such as when it is still warming up a cache.

Health checks are the gatekeepers of a deployment. No traffic reaches a new version until it proves it can handle requests. Without them, you would be routing real users to an app that might still be crashing on startup.

Every serious PaaS runs health checks automatically. You define the endpoint, and the platform handles the polling, the timeouts, and the decisions. Teams that build this themselves tune all of those settings by hand, and they usually learn the right values through painful trial and error. That tuition is paid in engineering time, on a problem the industry solved years ago.

Rolling Updates: Replacing the Plane's Engine Mid-Flight

Here is the hard part. Your old version is serving live traffic right now. You need to replace it without dropping a single request. The most common answer is a rolling update.

It works like this. Say you have four copies of your app running. The platform starts one copy of the new version and waits for its health checks to pass. Then it shifts a slice of traffic to it and shuts down one old copy. It repeats this, one copy at a time, until only the new version remains. Users never notice, because at every moment there are enough healthy copies to serve everyone.

Some teams use variations of this idea. A blue-green deployment runs the full new version beside the old one, then flips all traffic at once. A canary release sends a tiny share of users to the new version first, watching for errors before going wider.

Doing this by hand means writing orchestration logic, managing load balancer rules, and handling every edge case where a step fails halfway. That is months of engineering effort to build and a permanent tax to maintain, all for behavior a PaaS ships as the default. On a platform, you get zero-downtime releases out of the box, not as a project your team has to staff.

Rollbacks: The Escape Hatch

Sometimes the new version passes every check and still breaks something real. An error rate climbs. A page loads blank. Now speed matters more than anything, and the fastest fix is rarely a new patch. It is a rollback: redeploying the previous artefact that you already know works.

This is why frozen, versioned artefacts are so important. A rollback is only fast if the old version is stored, tested, and ready to run. Teams that rebuild from an old commit under pressure are gambling at the worst possible time.

On most PaaS platforms, a rollback is one command or one click. The platform keeps your release history and can restore any previous version in seconds. That single feature has saved more on-call engineers' nights than perhaps any other.

When You Don't Need a PaaS

The case for handing deployment to a platform is strong, but it isn't universal. There are teams for whom owning the pipeline is not overhead; it is a deliberate and justified engineering decision.

When Compliance Demands It

Regulated industries like finance, healthcare, government, often operate under requirements that a standard PaaS cannot satisfy out of the box. Data residency rules may dictate exactly which physical infrastructure your builds touch. Audit requirements may demand a level of provenance and access logging that a managed platform doesn't expose.

Security controls may need to extend into the build environment itself, not just the runtime. In these contexts, the cost of owning the pipeline is real, but it is the cost of operating in that industry.

When Deployment is Your Product

If your company sells deployment infrastructure, a CI/CD platform, a release orchestration tool, an internal developer platform, then your pipeline is not overhead at all. It is the product.

The engineers maintaining it are doing product work, not distraction work. The same applies to platform engineering teams at large organizations whose explicit charter is to build and own the deployment layer for dozens of other internal teams. In both cases, the question of "why are we running this ourselves" has an obvious answer: because this is what we do.

When Your Infrastructure is Genuinely Unusual

Most PaaS platforms are optimized for stateless web services and standard container workloads. If your system falls outside that envelope, GPU clusters, real-time systems with strict latency requirements, hybrid on-premise and cloud deployments, hardware-in-the-loop testing, a general-purpose platform may simply not fit.

Shoehorning an unusual workload into a PaaS often produces more friction than building narrow, purpose-built deployment tooling around the specific constraints you actually have.

The common thread across all three cases is specificity. The teams that are right to own their pipelines can usually state clearly why a platform doesn't fit. If the answer is "we've always done it this way" or "we like having control," that's worth questioning. If the answer is "our compliance requirements mandate X" or "we sell this," that's a reason.

Should You Still Be Running This Yourself?

A PaaS does not make any of these steps disappear. The build still runs. Artefacts still get stored. Migrations still execute, health checks still poll, and traffic still shifts one copy at a time. Abstracting these mechanics does not eliminate them. It standardizes them, and pushes their maintenance onto a team whose entire product is deployment.

That is the question every product team should now ask plainly: why are we still building and operating this machinery ourselves? A decade ago, a custom pipeline was unavoidable. Today it is a choice, and for most teams it is the wrong one. Every hour spent debugging a flaky build agent, tuning a health check timeout, or patching orchestration scripts is an hour taken from the product your customers actually pay for. The pipeline does not differentiate you. It cannot. Your competitors' deploys work the same way yours do.

Know how the chain works, because on-call at 2 a.m. demands it. But knowing how it works is not a reason to own it. "We built our own deployment system" is not a badge of honor anymore. It is an admission that your team maintains a second product with no customers. Unless deployment infrastructure is your business, hand the machinery to a platform, and put your engineers back on the work only they can do.

Hope you enjoyed this article. You can connect with me on LinkedIn.

But something big is changing. AI is no longer just responding. It is planning, deciding, and acting on its own. This new kind of AI is called agentic AI, and it is quickly becoming one of the most important shifts in technology today.

In this article, we'll break down what agentic AI is, how it works, where it is being used, and what risks it brings along.

What We'll Cover

• What Does "Agentic" Even Mean?

• How a Chatbot Works vs. How an Agent Works

• A Real Example: Booking a Business Trip

• The Building Blocks of an AI Agent

• Why Is This Happening Now?

• Where Agentic AI Is Being Used Today

• What Are the Risks?

• What This Means for You

What Does "Agentic" Even Mean?

The word comes from "agency": the ability to act independently toward a goal.

A regular chatbot waits for you to ask something. An agentic AI system is given a goal and then figures out the steps needed to reach it. It can use tools, browse the web, write and run code, send emails, and loop back to fix its own mistakes, without you guiding every move.

Think of the difference this way: A chatbot is like a very knowledgeable colleague who only speaks when spoken to. An AI agent is like giving that colleague a task and saying, "Handle this for me," then walking away.

How a Chatbot Works vs. How an Agent Works

To understand agentic AI, it helps to see the difference in action.

A chatbot follows a simple loop:

User types message → AI reads it → AI generates a reply → Done

An AI agent follows a much more complex loop:

User gives a goal
  → Agent breaks it into steps
  → Agent picks a tool (web search, code runner, email, etc.)
  → Agent takes action
  → Agent checks the result
  → If result is wrong or incomplete, agent adjusts and tries again
  → Agent moves to the next step
  → Repeats until the goal is achieved

That ability to plan, act, check, and retry is what makes agentic AI fundamentally different. It is not just predicting the next word in a sentence. It is running a small project.

A Real Example: Booking a Business Trip

Here is a concrete example to make this tangible.

You tell an AI agent: "Book me the cheapest flight to Mumbai next Monday, find a hotel near the conference centre, and add both to my calendar."

A chatbot would give you links or suggestions and leave the rest to you.

An AI agent would:

Step 1: Search for flights to Mumbai on Monday
Step 2: Compare prices and pick the cheapest option
Step 3: Fill in your passenger details and complete the booking
Step 4: Search for hotels near the conference centre
Step 5: Cross-check availability and price
Step 6: Complete the hotel booking
Step 7: Pull the confirmation details from both bookings
Step 8: Add flight and hotel to your Google Calendar
Step 9: Send you a summary email

Each of those steps involves calling a different tool or service. The agent handles all of it. You just gave it the goal.

The Building Blocks of an AI Agent

Every AI agent, no matter how complex, is built on a few core components.

A brain (the language model). This is the reasoning engine: usually a large language model like GPT-4 or Claude. It reads the goal, thinks through the plan, and decides what to do next.

Memory. Agents need to remember what they have already done. Short-term memory keeps track of the current task. Long-term memory lets the agent store information across sessions: so it remembers your preferences from last time.

Tools. An agent without tools is just a chatbot. Tools are what give agents power. Common tools include web search, code execution, file reading, API calls, email, and calendar access. The agent decides which tool to use and when.

A feedback loop. After taking an action, the agent checks whether it worked. If a step failed or returned a wrong result, it tries a different approach. This self-correction is what makes agents reliable for multi-step tasks.

Why Is This Happening Now?

Agentic AI is not a brand new idea. Researchers have explored autonomous agents for decades. So why is it suddenly everywhere in 2026?

Three things came together at the right time.

First, language models got dramatically better at reasoning. Earlier models were good at writing text but poor at logical planning. Newer models can break down complex tasks, spot errors in their own output, and change strategy mid-task.

Second, tool integration became much easier. Frameworks like LangChain, AutoGen, and OpenAI's function calling made it straightforward for developers to connect a language model to real-world tools. What once took months of custom engineering now takes days.

Third, businesses started demanding it. Copy-pasting AI suggestions into forms and emails gets old quickly. Companies want AI that completes workflows, not just assists with them.

Where Agentic AI Is Being Used Today

Agentic AI is already showing up across many industries, not just in tech companies.

In software development, AI agents write code, run tests, find bugs, and open pull requests: all from a single instruction like "fix the login error on the checkout page."

In customer support, agents handle entire conversations. They look up order history, process refunds, escalate complex cases to a human, and follow up via email: without a support agent touching the ticket.

In research, agents search dozens of sources, extract key data, cross-reference findings, and produce a summarized report. A task that used to take hours gets done in minutes.

In marketing, agents draft campaign content, schedule social posts, monitor performance metrics, and suggest adjustments based on what is working.

What Are the Risks?

Agentic AI is powerful, but it comes with real concerns that are worth knowing about.

The biggest one is unintended actions. An agent that misunderstands a goal can take a chain of wrong steps before anyone notices. Unlike a chatbot that gives a wrong answer you can simply ignore, an agent that makes a wrong booking or sends the wrong email has already caused a real-world consequence.

There is also the issue of security. Agents that can read emails, access files, and browse the web are attractive targets. A technique called "prompt injection" can trick an agent into following malicious instructions hidden inside a webpage or document it reads during a task.

Finally, there is accountability. When an AI agent makes a mistake across a ten-step workflow, it can be genuinely hard to trace exactly where things went wrong, and who or what is responsible.

This is why most well-designed agentic systems today include a "human in the loop": a checkpoint where a person reviews and approves key decisions before the agent acts on them.

What This Means for You

You do not need to be a developer to feel the impact of agentic AI. These systems are already being built into the tools people use every day: email clients, project management apps, CRM systems, and more.

The shift worth understanding is this: AI is moving from a tool you interact with to a system that works alongside you. The chatbot answered your questions. The agent handles your tasks.

That is a meaningful change: not just in how AI works, but in how we work with it. The more you understand what agents can and cannot do, the better placed you are to use them well, delegate wisely, and catch mistakes before they snowball.

Agentic AI is not science fiction. It is already in your workplace, and it is only going to become more capable from here.

Understanding the technology is the first step. The next is deciding how to put it to work.

Hope you enjoyed this article. You can connect with me on LinkedIn.

A hero section, feature grid, testimonials, pricing, FAQ, and footer are common building blocks, yet developers often spend hours recreating them for every new project.

In this guide, you'll learn how to build a modern marketing landing page using Next.js, shadcn/ui, Tailwind CSS, and reusable Shadcn blocks. Instead of building every section from scratch, you'll assemble a production-ready page, customize it to match your brand, and finish with a foundation that's ready for real-world projects.

Table of Contents

• Prerequisites

• What Are We Building?

• Project Setup with Base UI Using the Shadcn Preset

• Two Ways to Build Your Marketing Landing Page

• Option 1: Build Using the CLI

• Option 2: Build Using the MCP Server

• How to Optimize Your Landing Page

• How to Expand Your Marketing Website

• Live Preview:

• Conclusion

Prerequisites

Before you start, make sure you have:

• Node.js 18 or higher installed

• shadcn/ui initialized in your project (npx shadcn@latest init)

• Basic knowledge of React and TypeScript

If you haven't initialized shadcn/ui yet, run npx shadcn@latest init in your project root and follow the prompts before continuing.

What Are We Building?

In this tutorial, we'll build a modern marketing landing page using production-ready blocks from Shadcn Space. Instead of designing and developing every section from scratch, we'll assemble a complete landing page using reusable shadcn/ui blocks and customize them to fit our brand and product.

You can use the same approach to create landing pages for SaaS products, AI tools, startups, agencies, developer tools, portfolios, and many other types of websites. Since every block is built with React, Tailwind CSS, and shadcn/ui, you have full control over the code and can easily modify the content, layout, and styling.

Why Build with Shadcn Space?

Creating a professional marketing website typically involves designing multiple sections that work together to tell your product's story and guide visitors to take action.

But instead of building every section from scratch, you can start with production-ready blocks that are easy to customize.

This lets you build marketing websites faster with reusable blocks (and you can also mix and match blocks to create unique page layouts). It also gives you full ownership of your clean React and Tailwind CSS code. And overall, you save development time without sacrificing flexibility.

Sections We'll Build

We'll build our marketing landing page using the following sections:

• Hero section with a compelling headline, call-to-action, and trusted-by logos.

• Features section to highlight your product's key capabilities.

• Product Showcase & Benefits section to demonstrate your product and communicate its value.

• Testimonials section to build credibility with customer feedback.

• Pricing section to present your plans clearly.

• FAQ section answers common questions and reduces friction.

• Call-to-Action section to encourage visitors to get started.

• Footer with navigation and important links.

Final Page Structure

Our landing page will have this structure:

<main>
  {/* 1. Hero section + Trusted by / Logo cloud */}
  <AgencyHeroSection />

  {/* 2. Features section */}
  <Feature01 />

  {/* 3. Product showcase & Benefits */}
  <AboutAndStats01 />

  {/* 4. Testimonials */}
  <Testimonials />

  {/* 5. Pricing section */}
  <Pricing />

  {/* 6. FAQ section */}
  <Faq />

  {/* 7. Call-to-action section */}
  <CTA />

  {/* Footer */}
  <Footer />
</main>

Each section will be installed from the Shadcn Space registry and customized directly inside our project. By the end of this tutorial, you'll have a fully responsive marketing landing page built with Next.js, Tailwind CSS, and shadcn/ui that you can adapt for your own product or business.

Project Setup with Base UI Using the Shadcn Preset

Since Shadcn Space blocks are built using Base UI primitives, we'll create our project using the Base UI preset instead of the default Radix setup.

This ensures that our landing page uses the same foundation as the blocks we're going to install.

1. Create the Project with Base UI

Run the following command:

pnpm dlx shadcn@latest init --preset b0 --template next

This command does a few important things:

• Creates a Next.js project

• Configures Tailwind CSS

• Sets up Base UI as the component foundation

• Uses the Nova style preset

• Configures Lucide icons

• Uses Inter font

• Applies neutral theme tokens

You now have a Base UI-powered Next.js project ready for building your landing page.

2. Add the Shadcn Space Registry

Open your components.json and add the following registry configuration:

{
  "registries": {
    "@shadcn-space": {
      "url": "https://shadcnspace.com/r/{name}.json",
    }
  }
}

This tells the CLI where to fetch components and blocks from the registry.

For more information about how to use it in your project, check out the docs.

Two Ways to Build Your Marketing Landing Page

Now that your project is set up with shadcn/ui, it's time to start building the marketing landing page.

You can install production-ready blocks directly into your project in two different ways:

• Using the CLI

• Using the MCP Server inside your AI-powered editor

Both approaches install the actual React and Tailwind CSS source code into your project, giving you complete control over customization. The only difference is how you discover and install the blocks.

Option 1: Build Using the CLI

The CLI is the fastest way to browse the registry and install individual blocks into your project. It gives you full control over which sections you want to use and how you customize them.

Step 1: Browse Marketing Blocks

Visit the block registry and explore the available marketing blocks.

Let's review the sections we'll use for this tutorial:

• Hero section with a call-to-action and trusted-by logos

• Features section

• Product showcase & benefits section

• Testimonials section

• Pricing section

• FAQ section

• Call-to-action section

• Footer

Choose the blocks that best match the design and style of your website. Since every block is fully customizable, you can easily update the content, colors, spacing, and layout to match your brand.

In the following sections, we'll install each block and customize them.

Step 2: Install Selected Blocks

Once you find a block you like, install it using the CLI:

npx shadcn@latest add @shadcn-space/{block-name}

Each command downloads the block, places it inside components/shadcn-space/blocks, and installs the required dependencies.

Now your folder might look like this:

components/
  shadcn-space/
    blocks/
      about-us-section-01/
      hero-01/
      features-01/
      pricing-01/
      testimonial-01/
      faq-01/
      cta-01/
      footer-01/

Note: I've used the first block from each section in this tutorial. You can choose any other shadcn block that suits best according to your needs.

Step 3: Add a Hero Section

Every great marketing landing page starts with a strong hero section. It's the first thing visitors see, so it should clearly communicate what your product does, who it's for, and encourage users to take action.

Instead of building the section from scratch, we'll install a production-ready Hero block and customize it to match our landing page.

1. Install the Hero Block

Run the following CLI command to add the Hero block to your project:

npx shadcn@latest add @shadcn-space/hero-01

The CLI will automatically download the Hero block source code, add the component to your project, and install any required dependencies.

After the installation completes, you should see the following directory structure:

components/
  shadcn-space/
    blocks/
      hero-01/
        index.tsx

You can now open the component and customize the heading, description, call-to-action buttons, images, and other content to match your product and branding.

2. Understand the Hero Block Structure

Once the installation is complete, open the Hero block located at:

components/shadcn-space/blocks/hero-01/index.tsx

You'll see a component similar to the following:

import HeroSection from "@/components/shadcn-space/blocks/hero-01/hero";
import type { NavigationSection } from "@/components/shadcn-space/blocks/hero-01/header";
import Header from "@/components/shadcn-space/blocks/hero-01/header";
import BrandSlider, { BrandList } from "@/components/shadcn-space/blocks/hero-01/brand-slider";
import type { AvatarList } from "@/components/shadcn-space/blocks/hero-01/hero";

export default function AgencyHeroSection() {
  const avatarList: AvatarList[] = [...];
  const navigationData: NavigationSection[] = [...];
  const brandList: BrandList[] = [...];

  return (
    <div className="relative">
      <Header navigationData={navigationData} />
      <main>
        <HeroSection avatarList={avatarList} />
        <BrandSlider brandList={brandList} />
      </main>
    </div>
  );
}

What Should You Notice?

Before making any changes, take a moment to understand how the block is organized. Rather than being a single large component, it's composed of smaller, reusable components that work together.

In this example:

• The Header component renders the navigation and receives its menu items through the navigationData array.

• The HeroSection component contains the main headline, description, call-to-action buttons, and social proof, while the avatarList provides the data displayed in the hero.

• The BrandSlider component displays the company logos using the brandList array.

This separation keeps the code modular and makes each part of the landing page easier to customize or replace independently.

Why This Matters?

Because the block is copied directly into your project, you're working with standard React components instead of a compiled package. Every file is fully editable, allowing you to understand how the section is built and modify it to suit your own requirements.

For example, you can update the navigation links, replace the placeholder content with your own branding, add additional sections, integrate custom functionality, or adjust the styling using Tailwind CSS classes. Since everything lives inside your codebase, you're free to restructure the component however you like without being locked into predefined APIs or abstractions.

3. Render the Hero Section

Now that the Hero block has been installed, it's time to display it on the page.

Import the component into your app/page.tsx file:

import AgencyHeroSection from "@/components/shadcn-space/blocks/hero-01";

export default function Page() {
  return (
    <AgencyHeroSection />
  );
}

Save the file and start your development server if it isn't already running. When you open the application in your browser, you'll see the Hero section rendered as the first part of your marketing landing page.

With the Hero section in place, we've completed the first building block of our landing page. Next, we'll continue by adding the remaining sections to create a complete marketing website.

Install the Remaining Blocks

Now that you've added and rendered the Hero section, let's install the remaining blocks required for our marketing landing page.

Run the following command to install all the remaining sections at once:

npx shadcn@latest add @shadcn-space/feature-01 @shadcn-space/about-us-section-01 @shadcn-space/testimonial-01 @shadcn-space/pricing-01 @shadcn-space/faq-01 @shadcn-space/cta-01  @shadcn-space/footer-01

Note: If you're using Windows Command Prompt or PowerShell, run the command on a single line instead of using \ for line continuation.

After the installation is complete, update your app/page.tsx by importing the newly added blocks and rendering them in the following order:

import AgencyHeroSection from "@/components/shadcn-space/blocks/hero-01";
import AboutAndStats01 from "@/components/shadcn-space/blocks/about-us-01";
import Feature01 from "@/components/shadcn-space/blocks/feature-01";
import Pricing from "@/components/shadcn-space/blocks/pricing-01/pricing";
import Testimonials from "@/components/shadcn-space/blocks/testimonial-01/testimonial";
import Faq from "@/components/shadcn-space/blocks/faq-01/faq";
import CTA from "@/components/shadcn-space/blocks/cta-01/cta";
import Footer from "@/components/shadcn-space/blocks/footer-01/footer";


export const metadata = {
  title: "Acme Agency – Innovative Digital Solutions",
  description:
    "We craft immersive digital experiences for bold brands. Explore our services, pricing, and success stories.",
};


export default function Page() {
  return (
    <main>
      {/* 1. Hero section + Trusted by / logo cloud */}
      <AgencyHeroSection />


      {/* 2. Features section */}
      <Feature01 />


      {/* 3. Product showcase & Benefits section (Using About/Stats as a placeholder for these) */}
      <AboutAndStats01 />


      {/* 4. Testimonials */}
      <Testimonials />


      {/* 5. Pricing section */}
      <Pricing />


      {/* 6. FAQ section */}
      <Faq />


      {/* 7. Call-to-action section */}
      <CTA />


      {/* Footer */}
      <Footer />
    </main>
  );
}

That's it! Your complete marketing landing page is now assembled. You can start customizing the content, images, colors, and layout of each section to match your product and brand.

Customize the Marketing Landing Page

At this point, the overall structure of your marketing landing page is complete. The next step is to personalize each section so it reflects your product, brand, and messaging instead of the default placeholder content.

One of the biggest advantages of working with reusable React components is that every section can be customized independently. You can update the content, replace images, adjust layouts, and refine the styling without rebuilding the page from scratch.

The following examples show how the default blocks can be transformed into a polished marketing website.

Customize the Hero Section

The Hero section is the first thing visitors see, so it's the most important place to communicate your product's value. Replace the placeholder headline, supporting text, call-to-action buttons, and trusted brand logos with content that represents your own business.

Before:

After:

Notice how the customized version immediately establishes the product's identity through updated messaging, branding, imagery, and call-to-action buttons.

Customize the Features Section

The Features section should explain what your product offers and why it stands out. Replace the sample feature cards with capabilities that highlight your product's most valuable functionality.

Before:

After:

Updating the feature titles, descriptions, and icons makes the section more relevant to your audience while reinforcing your product's key selling points.

Customize the Pricing Section

Your pricing section should clearly communicate the plans you offer and help visitors choose the option that best fits their needs. Replace the default plans, pricing, feature lists, and button labels with information that matches your business model.

Before:

After:

A customized pricing section builds trust by presenting accurate information while making it easier for potential customers to compare plans.

Customize the FAQ Section

The FAQ section is a great opportunity to answer common questions before visitors contact your team. Replace the placeholder questions with answers related to your product, pricing, integrations, support, or onboarding process.

Before:

After:

Tailoring the FAQ to your product helps reduce uncertainty, improves the user experience, and can answer many questions before a customer reaches out.

The Result

With just a few content updates, the default blocks evolve into a professional marketing landing page tailored to your brand. Since every section is built with reusable React components, you can continue refining the design, adjusting layouts, and adding new content as your product grows without changing the overall page structure.

Option 2: Build Using the MCP Server

If you prefer a faster workflow, you can use the MCP Server to generate your landing page directly inside your editor. Instead of manually browsing and installing individual blocks, you simply describe the page you want to build, and the MCP Server assembles an initial version for you.

The MCP Server works with supported editors like Antigravity, VS Code, Cursor, Windsurf, and other MCP-compatible editors.

Step 1: Install the MCP Server

Quick Installation:

The fastest way to get started. Choose your package manager and run the command corresponding to your client:

For Claude Code:

claude mcp add shadcnspace-mcp -- npx -y shadcnspace-mcp@latest

For Others:

npx shadcnspace-cli install <client>

Replace <client> with cursor, antigravity, vscode, or windsurf.

Manual Installation For VS Code:

{
  "servers": {
    "shadcnspace-mcp": {
      "command": "npx",
      "args": ["-y", "shadcnspace-mcp@latest"]
    }
  }
}

• Open .vscode/mcp.json.

• Click Start next to the Shadcn Space MCP server

For a detailed guide, follow the MCP Server documentation to install it for your preferred editor.

Once the installation is complete, restart your editor to enable the MCP connection.

Step 2: Open the AI Chat

Open the AI chat panel inside your editor and describe the landing page you'd like to create.

For example:

Create a modern marketing landing page for an AI SaaS product.

Use Shadcn Space blocks and include:

- Hero section
- Features section
- Product showcase & benefits
- Testimonials
- Pricing
- FAQ
- Call-to-action
- Footer

Use a clean, modern, and responsive design.

Feel free to replace the product description with your own and customize as you like.

Step 3: Generate the Landing Page

After receiving your prompt, the MCP Server analyzes your requirements and selects the most suitable blocks for your landing page. It automatically assembles the page using a combination of reusable sections, giving you a working layout in just a few moments.

The generated sections are added directly to your project as standard React components. Since everything is real source code, you can customize every part of the page: update the content, replace images, modify the layout, adjust spacing, or restyle components using Tailwind CSS.

The MCP Server simply speeds up the initial setup, while giving you complete control over the final implementation.

How to Optimize Your Landing Page

Building a visually appealing landing page is only the first step. Before publishing your website, it's worth spending some time optimizing it for performance, search engines, and user experience.

A fast, responsive landing page not only feels more polished but also helps improve engagement and conversion rates. Fortunately, Next.js provides several built-in features that make these optimizations straightforward.

Optimize Images with Next.js

Images are often the largest assets on a marketing website, so optimizing them can significantly improve loading performance. If you're using Next.js, prefer the built-in next/image component instead of the standard HTML image tag. It automatically serves appropriately sized images, supports modern image formats, and reduces layout shifts as the page loads.

Before adding screenshots or illustrations to your project, make sure they're compressed and sized appropriately. Well-optimized images create a smoother browsing experience across desktop and mobile devices while also contributing to better Core Web Vitals.

Improve SEO for Your Landing Page

A well-designed landing page is only effective if people can discover it. Search engine optimization starts with creating meaningful content that clearly communicates what your product offers. Choose a descriptive page title, write a concise meta description, and organize your content using logical headings.

Your primary keyword should appear naturally throughout the page without forcing it into every paragraph. Focus on writing for your audience first, then structure the content in a way that search engines can easily understand. Combining valuable content with a clear page hierarchy gives your landing page the best chance of ranking for relevant searches.

Add Metadata and Open Graph Images

When someone shares your landing page on social media or in a messaging application, the preview is generated from your page's metadata. Configuring Open Graph and Twitter metadata allows you to control the title, description, and preview image that appear when your website is shared.

A custom preview image that reflects your branding makes your links look more professional and can encourage more people to click through. Taking a few minutes to configure these settings helps create a more polished experience whenever your content is shared online.

Optimize Performance

Performance plays a major role in how visitors perceive your website. A page that loads quickly feels more responsive and encourages users to continue exploring your content. As you customize your landing page, keep unnecessary JavaScript to a minimum, optimize static assets, and avoid loading resources that aren't immediately needed.

Even small improvements, such as reducing image sizes or simplifying animations, can noticeably improve loading speed. Before deploying your project, test the page under different network conditions to ensure it performs well for all visitors.

Improve Core Web Vitals

Core Web Vitals are Google's metrics for measuring real-world user experience. They evaluate how quickly your main content appears, how responsive the page feels during interactions, and whether elements remain stable as the page loads. Monitoring these metrics throughout development helps identify potential issues before they affect users.

Tools such as Lighthouse and PageSpeed Insights provide detailed reports that can help you improve loading performance and responsiveness. A landing page with strong Core Web Vitals not only creates a better experience for visitors but can also contribute to improved search rankings.

How to Expand Your Marketing Website

A landing page is often just the beginning of a complete marketing website. As your product grows, you'll likely need additional pages that provide more information, improve navigation, and create a better experience for your visitors. Common additions include Blog, Blog Details, Pricing, FAQ, Changelog, Contact, Integration, Error, and About Us pages.

Building these pages with a consistent design system helps maintain a unified look and feel across your entire website while reducing development time. Reusing layouts and components also makes your project easier to maintain as it evolves.

If you're looking to expand your website beyond a single landing page, explore the collection of production-ready Shadcn website pages, built with React, Next.js, Tailwind CSS, and shadcn/ui.

Live Preview:

Conclusion

In this tutorial, we built a complete marketing landing page using Next.js, shadcn/ui, Tailwind CSS, and reusable Shadcn Space blocks. Starting from a fresh project, we assembled a production-ready page by combining sections such as the Hero, Features, Product Showcase, Testimonials, Pricing, FAQ, Call-to-Action, and Footer.

Because every block is added as standard React source code, you're free to customize the content, layout, styling, and functionality to match your own product and branding. Whether you're building a SaaS application, a startup website, an AI product, an agency site, or a developer tool, the same approach can be adapted to your requirements.

A marketing landing page is just the foundation of your online presence. As your product grows, you can continue expanding your website with additional marketing pages while maintaining a consistent design system and development workflow.

I hope this guide has helped you understand how to quickly build a modern, responsive marketing landing page using reusable components. Feel free to experiment with different block combinations, personalize the design, and create a website that best represents your product.

Appreciation: I wrote this article with the help of Mihir Koshti (Sr. Full Stack Developer) – Connect on LinkedIn.

To help you master this new paradigm, we have just published a massive, comprehensive course on the freeCodeCamp.org YouTube channel that will teach you all about agentic AI.

This course is designed to take you from the foundational concepts of agentic AI all the way through to building production-ready, end-to-end agent workflows using LangChain and LangGraph.

Here are a few key takeaways you can expect from the course:

• The architectural differences between standard LLMs and Agentic AI.

• How to build both single and multi-agent systems using LangChain.

• The core components of LangGraph and why it is essential for stateful, cyclical agent workflows.

• Advanced techniques including Human-in-the-Loop (HITL), Retrieval-Augmented Generation (RAG), and streaming responses.

• Best practices for deploying agentic systems to production environments like AWS and Render using Docker and GitHub Actions.

This is a hands-on, project-driven course. By the end of the 24 hours, you will have built fully functional, deployable AI agents from scratch.

Head over to the freeCodeCamp.org YouTube channel to watch the full course (24-hour watch).

An operator extends the same pattern to resources Kubernetes doesn't know about natively, letting you manage custom, often external, systems the same declarative way you manage everything else in the cluster.

This guide is divided into four parts: what an operator actually is, the anatomy of one, building one from scratch, and preparing it for production.

Table of Contents

• Part 1: Introduction

  • What is an Operator?

  • Operator vs Controller vs CRD

  • Why Not Just a Helm Chart, a CronJob, or a Script?

• Part 2: Anatomy of an Operator

  • Custom Resource

  • Watching for Change

  • Manager

  • Reconciliation Loop

• Part 3: Building the Operator

  • Setup

  • Mock Provider

  • Defining the VirtualMachine CRD

  • Reconciler

  • Failure Handling & Retries

  • Finalizer

  • Predicate

  • Owned Resources

  • Cross-Resource Reconciliation

  • RBAC

• Part 4: Production & Deployment

  • Packaging & Deployment

  • Performance & Resilience

  • Security

  • Observability

• Next steps

Part 1: Introduction

What is an Operator?

Kubernetes works by comparing the state we describe against actual state. A controller acts to close the gap, whether that's the Deployment controller replacing a pod we killed or scaling down the ones we no longer want.

This loop of observe, compare, and act is called reconciliation. It means looking up a resource's desired and actual state, deciding what to do next, and recomputing that decision fresh on every run regardless of what changed.

That's what makes the loop resilient: it never has to trust that it saw every event, only that it gets called again.

An operator applies that exact same loop to a resource Kubernetes doesn't understand natively. We define a Custom Resource, describe our domain's desired state in it, and write a controller that knows how to reconcile that domain.

That's the whole concept. Everything else in this guide (informers, workqueues, finalizers, status conditions) exists to make that one loop reliable for a resource type Kubernetes only knows about because we defined it.

Operator vs Controller vs CRD

These three terms are often used interchangeably, but they describe three different layers of the same system.

• CRD (CustomResourceDefinition): a schema we register with the Kubernetes API server to teach it about a new resource type. On its own, a CRD does nothing. It only gives the API server a shape to store, validate, and serve.

• Controller: any piece of software running a reconciliation loop against a resource type, from the built-in Deployment controller to a custom controller reconciling a PostgresCluster.

• Operator: a controller, or a small set of controllers, that targets a custom resource and encodes enough domain-specific knowledge to manage its full lifecycle without a human: provisioning, upgrades, failure recovery, and so on.

Every operator is a controller, but not every controller is an operator. A CRD without a controller behind it is just a schema that nothing acts on.

Why Not Just a Helm Chart, a CronJob, or a Script?

A Helm chart renders a set of values into YAML and applies it once. It has no way to keep watching afterward. if a resource it created is deleted or drifts, Helm has no idea until we run helm upgrade again by hand.

A CronJob gives us a loop back, at the cost of granularity, staleness up to one interval, no state carried between runs, and no way for one CronJob to react to a status change another one made.

And a one-off script only acts when triggered, manually or by a CI pipeline, and does nothing about drift in between runs. It's also rarely written with retries and idempotency as first-class concerns.

Why an operator wins here:

An operator is event-driven and continuous. The API server notifies it the instant a custom resource is created, updated, or deleted, and it keeps reconciling for that resource's entire lifetime, not just at apply time. That matters most for state that takes time to converge, can fail partway through, and can drift after it's first created.

This comes at a cost, though. An operator is a long-running process with its own RBAC (Role-Based Access Control), failure modes, and observability surface. This is more to build and operate than a chart or a script.

If the problem really is rendering some YAML once, a Helm chart is the right tool. An operator earns its keep when the problem is keeping something continuously correct, which is what the rest of this guide builds toward.

Part 2: Anatomy of an Operator

Next, we'll look at the pieces that make that loop actually work: the Custom Resource itself, the machinery that notices when something changed, and the manager that runs it all, before going into the reconciliation loop in detail.

Custom Resource

Before a controller can reconcile anything, the API server needs to know the shape of what it's storing. Registering a CRD teaches it that shape.

Every Custom Resource carries the same identity fields every Kubernetes object already has (kind, name, namespace, labels, and so on), plus two fields that are entirely ours to define: a spec and a status. That split isn't a style choice. It maps directly to the reconciliation loop.

• Spec is desired state. Whoever creates or edits the resource writes it, and the controller only ever reads it.

• Status is observed state. It's written only by the controller, to record what it found and what it did.

A client that writes to status directly is working around the controller instead of through it, which is why status is usually served as its own subresource with separate permissions.

The last piece is registration. The API server and any client talking to it need a shared, agreed-upon way to encode and decode our type, so we register it once against a scheme before anything can use it. Without that, our type is just a definition nobody can serve. With it, the API server can store and serve it exactly the way it serves Pods or Deployments.

We'll see exactly what that registration looks like when we build one for real in Part 3.

Watching for Change

A reconciler doesn't poll the API server in a loop asking "did anything change yet?" Three pieces work together to avoid that.

An informer opens a long-lived watch against the API server and keeps a local, in-memory cache of every object of a given type, updating it as add, update, and delete events arrive.

A lister reads from that cache instead of the API server, so a reconciler checking "does this Resource already exist?" costs a local map lookup, not a network call.

A workqueue sits between the informer and the reconciler. When the informer sees a change, it doesn't call the reconciler directly. Instead, it enqueues a key, namespace, and name, not the object itself. Workers pull keys off the queue and reconcile them, and the queue deduplicates and rate-limits on our behalf, so ten rapid updates to the same object collapse into one pending item instead of ten redundant reconciles.

This is also why a reconciler receives a key and not an object. By the time a worker picks the key off the queue, the object may have changed again, so the reconciler always looks up the current state itself rather than trusting whatever triggered it.

Manager

The manager is the process that owns all of this: the shared cache the informers populate, the client the reconciler uses to read and write objects, and the health, readiness, and metrics endpoints the rest of the cluster uses to know the controller is alive. Every reconciler we register runs inside one manager.

If we run more than one replica of the same controller for availability, we don't want both replicas reconciling the same object at once and racing each other.

The manager coordinates this through leader election. Replicas compete for a lease, exactly one holds it and actively reconciles, and the rest sit idle until the leader stops renewing it.

We'll come back to this in practice in Part 4. For now it's enough to know the manager is what makes it possible.

Reconciliation Loop

This is the part that matters most. Once we understand this loop well, most of what an operator does is a variation on it.

A reconciler's entry point is called with just a namespace and a name, nothing else. No spec, status, or diff. The reconciler has to fetch the object itself, compare its spec against what it can observe of the actual state, and decide what to do. That constraint is deliberate, and it's the reason for everything below.

Idempotency

Because the reconciler only ever gets a key, and because it can be called any number of times for the same object (in a row, out of order, or after a long gap), it has to produce the same end result no matter how many times it runs. A reconciler that blindly calls create every time it runs breaks the moment it runs twice, since the second call fails against an object that already exists.

The fix is to always check current state before acting: create only if missing, update only if different, and delete only if it shouldn't exist.

Event-driven reconciliation

A reconcile is triggered by a watch event on the resource being reconciled, and by convention also on anything it owns or otherwise depends on. On top of that, most controllers set a periodic resync so the loop also runs on a schedule even with no watch event at all, which matters once state can drift for reasons a watch would never catch.

Requeues

Sometimes a single pass through reconcile can't finish the job, becuase the work it's waiting on is still in progress elsewhere. A reconciler can ask to be called again after a delay without treating this as a failure. This is how it polls something that takes time to converge, rather than blocking inside a single call.

Error handling

Returning an error does something similar: it requeues, but with exponential backoff instead of a fixed delay. So a persistently failing reconcile doesn't hammer whatever it's failing against.

It's worth distinguishing errors that are worth retrying (like a timeout or lock conflict) from ones that aren't (like a spec that will never be valid, which should be surfaced as a status condition instead of retried forever).

Drift correction

Put all of the above together and the loop is self-healing by construction. Because reconcile recomputes the full diff every time rather than reacting to what specifically changed, it doesn't matter whether the drift came from someone running kubectl edit, another controller, or the underlying system the resource represents changing state on its own. The next reconcile, whether triggered by a watch event or a resync, sees the same gap either way and closes it the same way.

Part 3: Building the Operator

Everything so far has been building toward this. We now know what an operator is, how the terms around it relate, and what pieces a reconciliation loop is made of.

Now we'll put all of it to use and build VMOperator. It's an operator that manages a VirtualMachine Custom Resource backed by a mock cloud provider, a small HTTP service we'll also write that stands in for a real one.

apiVersion: compute.example.com/v1
kind: VirtualMachine
spec:
  image: ubuntu-22.04
  cpu: 2
  memory: 4Gi
status:
  phase: Running
  id: vm-123

Say we want to represent a virtual machine in a Kubernetes-native way, kubectl apply a YAML file, and get a VM – all without touching a cloud console or a separate CLI.

That's the motivation behind VMOperator: something that lives entirely outside Kubernetes becomes just another object the cluster's own tooling (kubectl, RBAC, GitOps pipelines) already knows how to work with.

Setup

We'll need a local cluster, and kind is the easiest way to get one:

kind create cluster --name vmoperator

Beyond that, we'll be using Go in this part for the operator, kubectl pointed at the new cluster, and Python with Flask for the mock provider below.

pip install flask

Mock Provider

Before we write controller code, we need something for it to control. The mock provider is a small HTTP service with three endpoints:

• POST /vms to create one

• GET /vms/{id} to check on it

• DELETE /vms/{id} to remove it

backed by nothing more than a dict in memory.

Every VM it creates starts in Provisioning and flips to Running a few seconds later on its own, which is enough to force our reconciler to actually poll instead of assuming success.

We're writing this one in Python rather than Go. It has nothing to do with the operator's code, as this is only for mock purposes.

import random
import string
import threading
import time

from flask import Flask, jsonify, request

app = Flask(__name__)
vms = {}  # in-memory store, keyed by VM id

def provision(vm):
    time.sleep(5)  # simulate provisioning taking time
    vm["phase"] = "Running"

@app.post("/vms")
def create_vm():
    body = request.get_json()
    vm_id = "vm-" + "".join(random.choices(string.digits, k=6))
    vm = {"id": vm_id, "image": body["image"], "phase": "Provisioning"}
    vms[vm_id] = vm

    threading.Thread(target=provision, args=(vm,), daemon=True).start()  # flips to Running in the background

    return jsonify(vm)

@app.get("/vms/<vm_id>")
def get_vm(vm_id):
    vm = vms.get(vm_id)
    if vm is None:
        return "", 404
    return jsonify(vm)

@app.delete("/vms/<vm_id>")
def delete_vm(vm_id):
    vms.pop(vm_id, None)
    return "", 204

if __name__ == "__main__":
    app.run(port=8080, threaded=True)

We'll run this as its own process, alongside the cluster, listening on the port the operator will be configured to call. Nothing about it knows Kubernetes exists, which is the point: it's standing in for a real cloud API.

Defining the VirtualMachine CRD

With something to control, we can define what we're controlling. The VirtualMachine type follows exactly the spec and status split from Part 2:

type VirtualMachineSpec struct {
	Image  string `json:"image"`
	CPU    int    `json:"cpu"`
	Memory string `json:"memory"`
}

type VirtualMachineStatus struct {
	ID    string `json:"id,omitempty"`    // provider-assigned id, empty until first provisioned
	Phase string `json:"phase,omitempty"` // mirrors the provider's lifecycle phase
}

type VirtualMachine struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec   VirtualMachineSpec   `json:"spec,omitempty"`
	Status VirtualMachineStatus `json:"status,omitempty"`
}

type VirtualMachineList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata,omitempty"`
	Items           []VirtualMachine `json:"items"`
}

TypeMeta carries kind and apiVersion, the same two fields on every Kubernetes object, built-in or custom, that say what this thing is.

ListMeta is its counterpart for list types, resourceVersion and continue for pagination (instead of name/namespace). This is why VirtualMachineList embeds ListMeta next to its TypeMeta while VirtualMachine itself embeds ObjectMeta.

Every type we register needs to satisfy runtime.Object, which means implementing DeepCopyObject. This is normally generated for us, but since we're doing this by hand, here's what that generated code actually looks like for VirtualMachine. The rest follow the same mechanical pattern:

func (in *VirtualMachine) DeepCopyObject() runtime.Object {
	out := VirtualMachine{
		TypeMeta:   in.TypeMeta,
		ObjectMeta: *in.ObjectMeta.DeepCopy(), // ObjectMeta already knows how to copy itself
		Spec:       in.Spec,                   // no pointers or slices in Spec, a plain copy is safe
		Status:     in.Status,
	}
	return &out
}

And the CRD manifest that teaches the API server about it:

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: virtualmachines.compute.example.com
spec:
  group: compute.example.com
  scope: Namespaced
  names:
    kind: VirtualMachine
    listKind: VirtualMachineList
    plural: virtualmachines
    singular: virtualmachine
    shortNames: [vm] # lets us type `kubectl get vm` instead of the full plural
  versions:
    - name: v1
      served: true
      storage: true
      subresources:
        status: {} # splits status into its own subresource, see Part 2
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              required: [image, cpu, memory]
              properties:
                image: { type: string }
                cpu: { type: integer }
                memory: { type: string }
            status:
              type: object
              properties:
                phase: { type: string }
                id: { type: string }

The subresources.status line matters. It's what makes status a separate subresource with its own update path. This is exactly the boundary we talked about in Part 2 between what a client can write and what only the controller can.

The names block is also what kubectl resolves against, kubectl get virtualmachines works because plural says so. shortNames is why kubectl get vm works too, the same way kubectl get po works for Pods.

Reconciler

The reconciler's job is small on paper, look at a VirtualMachine, and make sure a matching VM exists in the provider and its status reflects reality. We wrap the provider's HTTP API behind a small client so the reconciler itself stays readable:

func (r *VirtualMachineReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
	var vm computev1.VirtualMachine
	if err := r.Get(ctx, req.NamespacedName, &vm); err != nil {
		return ctrl.Result{}, client.IgnoreNotFound(err) // object was deleted, nothing left to do
	}

	if vm.Status.ID == "" {
		// no VM yet, this is the first time we've seen this object
		created, err := r.Provider.Create(ctx, vm.Spec.Image)
		if err != nil {
			return ctrl.Result{}, err
		}

		vm.Status.ID = created.ID
		vm.Status.Phase = created.Phase
		if err := r.Status().Update(ctx, &vm); err != nil {
			return ctrl.Result{}, err
		}

		return ctrl.Result{RequeueAfter: 2 * time.Second}, nil // check back shortly instead of blocking here
	}

	// VM already exists, poll the provider for whatever it knows right now
	current, err := r.Provider.Get(ctx, vm.Status.ID)
	if err != nil {
		return ctrl.Result{}, err
	}

	vm.Status.Phase = current.Phase
	if err := r.Status().Update(ctx, &vm); err != nil {
		return ctrl.Result{}, err
	}

	if current.Phase != "Running" {
		return ctrl.Result{RequeueAfter: 2 * time.Second}, nil // still provisioning, keep polling
	}

	return ctrl.Result{}, nil
}

Two things are worth calling out. First, this is only reachable at all because we've registered a watch on VirtualMachine. The API server tells us the moment one is created or edited, which is what triggers the first call.

Second, every branch ends by writing to vm.Status, mapping whatever the provider told us onto the resource. Kubernetes never talks to the provider directly. The only way anyone finds out a VM is running is because our reconciler wrote it into status.

Failure Handling & Retries

Notice the reconciler above never retries anything itself. When r.Provider.Create or r.Provider.Get fails (like because of a network blip or the mock provider not being up yet), it just returns the error. That's deliberate. Returning an error is how we ask controller-runtime to requeue with exponential backoff on our behalf. This means we don't need to hand-roll a retry loop, and a persistently unreachable provider doesn't get flooded with retries.

The one thing worth being careful about is treating every failure the same way. A timeout talking to the provider is worth retrying. A VirtualMachine whose spec.image the provider will never accept is not. Retrying that forever just produces a busy loop that never succeeds.

We'll leave surfacing that distinction through status conditions to the exercises. The reconciler above only has one failure mode to worry about, since the mock provider never rejects a request outright.

Finalizer

If we delete a VirtualMachine right now, Kubernetes removes the object and we're left with an orphaned VM the provider still thinks is running. A finalizer closes that gap: it's a string on the object that tells Kubernetes "don't actually delete this until I say so."

const vmFinalizer = "compute.example.com/vm-cleanup"

func (r *VirtualMachineReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
	var vm computev1.VirtualMachine
	if err := r.Get(ctx, req.NamespacedName, &vm); err != nil {
		return ctrl.Result{}, client.IgnoreNotFound(err)
	}

	if !vm.DeletionTimestamp.IsZero() {
		// being deleted, deprovision through the provider before letting it go
		if controllerutil.ContainsFinalizer(&vm, vmFinalizer) {
			if vm.Status.ID != "" {
				if err := r.Provider.Delete(ctx, vm.Status.ID); err != nil {
					return ctrl.Result{}, err
				}
			}
			controllerutil.RemoveFinalizer(&vm, vmFinalizer) // safe to let the delete proceed now
			return ctrl.Result{}, r.Update(ctx, &vm)
		}
		return ctrl.Result{}, nil
	}

	if !controllerutil.ContainsFinalizer(&vm, vmFinalizer) {
		controllerutil.AddFinalizer(&vm, vmFinalizer) // register before we ever provision anything
		if err := r.Update(ctx, &vm); err != nil {
			return ctrl.Result{}, err
		}
	}

	// ... provisioning logic from before
	return ctrl.Result{}, nil
}

A kubectl delete on a VirtualMachine with our finalizer present doesn't remove it. Rather, it sets deletionTimestamp and waits.

Our reconciler sees that on the next call, deprovisions the VM through the provider, and only then removes the finalizer. At this point Kubernetes finally deletes the object. If there's no finalizer, there's no guarantee that cleanup ever runs.

Predicate

There's a subtle bug already sitting in the reconciler above. Every time it calls r.Status().Update, that write is itself a change to the object. This triggers our own watch, which calls reconcile again.

Left alone, this doesn't spin forever, since we're recomputing the same status until it settles. But it's still wasted work reconciling in response to writes we made ourselves.

A predicate filters those events that actually enqueue a reconcile, before our code ever runs:

func (r *VirtualMachineReconciler) SetupWithManager(mgr ctrl.Manager) error {
	return ctrl.NewControllerManagedBy(mgr).
		For(&computev1.VirtualMachine{}, builder.WithPredicates(predicate.GenerationChangedPredicate{})). // drop status only events
		Complete(r)
}

generation only increments when spec changes. Status updates don't touch it. GenerationChangedPredicate uses that to drop events where nothing but status moved, so our own writes stop retriggering us. Then we're back to reconciling only when something meaningful changed, or when we explicitly ask to be requeued.

Owned Resources

A VirtualMachine being Running somewhere isn't very useful on its own, so let's make the operator also create a Secret holding the VM's connection details in-cluster:

func (r *VirtualMachineReconciler) reconcileConnectionSecret(ctx context.Context, vm *computev1.VirtualMachine) error {
	secret := &corev1.Secret{
		ObjectMeta: metav1.ObjectMeta{
			Name:      vm.Name + "-connection",
			Namespace: vm.Namespace,
		},
		StringData: map[string]string{"id": vm.Status.ID},
	}

	if err := controllerutil.SetControllerReference(vm, secret, r.Scheme); err != nil {
		return err // ties the Secret's lifecycle to this VirtualMachine
	}

	return r.Patch(ctx, secret, client.Apply, client.ForceOwnership, client.FieldOwner("vmoperator")) // create or update, either way
}

SetControllerReference is what makes this an owned resource, it stamps an owner reference onto the Secret pointing back at the VirtualMachine.

Two things fall out of that for free. Deleting the VirtualMachine now cascades, Kubernetes garbage collects the Secret automatically, and there's no finalizer needed since it's an in-cluster object, not an external one.

And if we add Owns(&corev1.Secret{}) alongside For(&computev1.VirtualMachine{}) in SetupWithManager, an edit or deletion of the Secret itself re-triggers reconciliation of its owning VirtualMachine. So if someone deletes it by hand, we notice and recreate it.

The same pattern, SetControllerReference call, and Owns() registration creates a second owned resource: a Service fronting the VM in-cluster. That's two different resource kinds owned by one VirtualMachine, which is all multiple owned resources means in practice. There's nothing more to it than calling the same pattern twice for different types.

Cross-Resource Reconciliation

Every VirtualMachine so far talks to one hardcoded provider endpoint. Real deployments need that to be configurable, and it's rarely a one-off: fifty VirtualMachines in the same AWS account share the same endpoint and credentials, and a hundred more might live in Azure instead.

We could put an endpoint field directly on VirtualMachineSpec, but rotating a credential or fixing a typo would then mean editing every VirtualMachine that uses it, one at a time. Pulling that into its own object lets many VirtualMachines reference it by name instead, so a single edit propagates to all of them.

Now let's add a second, small CRD:

type ProviderConfigSpec struct {
	Endpoint string `json:"endpoint"`
}

And a providerRef field on VirtualMachineSpec pointing at one by name. The interesting part isn't the new type. It's what happens when a ProviderConfig changes.

A VirtualMachine doesn't watch ProviderConfig directly, and there's no owner reference between them, so a plain Owns() won't do it. Instead, we watch the type and map each event onto every VirtualMachine that references it:

func (r *VirtualMachineReconciler) SetupWithManager(mgr ctrl.Manager) error {
	return ctrl.NewControllerManagedBy(mgr).
		For(&computev1.VirtualMachine{}, builder.WithPredicates(predicate.GenerationChangedPredicate{})).
		Owns(&corev1.Secret{}).
		Owns(&corev1.Service{}).
		Watches(
			&computev1.ProviderConfig{}, // not owned, so Owns() won't catch its changes
			handler.EnqueueRequestsFromMapFunc(r.findVirtualMachinesForProviderConfig),
		).
		Complete(r)
}

func (r *VirtualMachineReconciler) findVirtualMachinesForProviderConfig(ctx context.Context, obj client.Object) []reconcile.Request {
	var vms computev1.VirtualMachineList
	if err := r.List(ctx, &vms, client.InNamespace(obj.GetNamespace())); err != nil {
		return nil
	}

	var requests []reconcile.Request
	for _, vm := range vms.Items {
		if vm.Spec.ProviderRef == obj.GetName() { // only re-enqueue VMs that actually reference this config
			requests = append(requests, reconcile.Request{NamespacedName: client.ObjectKeyFromObject(&vm)})
		}
	}
	return requests
}

This is cross-resource reconciliation: one resource's change causing a different resource type entirely to reconcile, connected only by a field value rather than ownership.

It's also where the theme of this whole project comes back around. ProviderConfig is what would hold real credentials and a real endpoint for AWS, Azure, or GCP in a production version of this operator. The mock provider is standing in for exactly that boundary.

RBAC

None of the above works without permission to act on it. The manifest just has to list what we actually touch: VirtualMachine and ProviderConfig objects, the VirtualMachine status subresource separately, and the Secret/Service objects we create:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: vmoperator-manager-role
rules:
  - apiGroups: ['compute.example.com']
    resources: ['virtualmachines', 'providerconfigs']
    verbs: ['get', 'list', 'watch', 'create', 'update', 'patch', 'delete']
  - apiGroups: ['compute.example.com']
    resources: ['virtualmachines/status'] # separate rule, it's a separate subresource
    verbs: ['get', 'update', 'patch']
  - apiGroups: ['']
    resources: ['secrets', 'services']
    verbs: ['get', 'list', 'watch', 'create', 'update', 'patch', 'delete']
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: vmoperator-manager-rolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: vmoperator-manager-role
subjects:
  - kind: ServiceAccount
    name: vmoperator-controller-manager
    namespace: vmoperator-system

Note: VMOperator manages a resource entirely outside the cluster through a hand-rolled HTTP client, but it's not a novel pattern. Crossplane, AWS Controllers for Kubernetes, Cluster API, and cert-manager all reconcile external or non-Kubernetes state through CRDs the same way. These resources are worth reading once this pattern feels familiar.

Another note: we're keeping VMOperator's scope narrow on purpose. Resizing a running VM, stopping and restarting one, taking snapshots, and supporting more than one real provider behind ProviderConfig are all natural extensions of what's here, and a reasonable next step once the core loop feels solid.

Part 4: Production & Deployment

Now that VMOperator works, let's see how to package and deploy it and improve it for production.

Packaging & Deployment

Everything so far has run as a binary on our own machine. go run against whatever cluster kubectl happens to be pointed at.

A Deployment needs an image instead, so the operator gets a multi-stage Dockerfile: one stage to compile it, and a second, much smaller image to actually run it:

FROM golang:1.26 AS build
WORKDIR /src
COPY . .
RUN CGO_ENABLED=0 go build -o /vmoperator ./cmd/manager

FROM gcr.io/distroless/static-debian12
COPY --from=build /vmoperator /vmoperator
USER 65532:65532 # nonroot, matches the security context on the Deployment below
ENTRYPOINT ["/vmoperator"]

The build stage has the full Go toolchain and every source file, none of which need to ship. The final image only has the compiled binary, which covers most of what a container security context later in this part would otherwise have to ask for. There's no shell to get a foothold in even before runAsNonRoot is set.

docker build -t registry.example.com/vmoperator:v0.1.0 .
docker push registry.example.com/vmoperator:v0.1.0

That image is what the Deployment manifest under config/manager/ actually references. With it pushed somewhere the cluster can pull from, the rest of the manifests can go on: the CRDs, RBAC, the operator's Deployment, and a Deployment and Service for the mock provider. So it's no longer something we run as a side process on our own machine either:

kubectl apply -f config/crd/
kubectl apply -f config/rbac/
kubectl apply -f config/manager/

Installing the CRDs before anything else matters. Otherwise the operator's Deployment will crash-loop if it starts and immediately (it tries to watch a resource type the API server has never heard of).

Schema changes are the part that hand-written manifests make us feel directly. Adding a field to VirtualMachineSpec is harmless, but existing objects just don't have it set. Renaming or restructuring one isn't: every stored VirtualMachine was serialized against the old shape.

The CRD's versions list is built for exactly this, as more than one version can be served at once. One is marked storage to say which shape objects are actually persisted as, and a conversion webhook translates between them when a client asks for a version that isn't the stored one.

We don't need this for VMOperator today, since v1 is the only version that's ever existed. But it's why the versions field was a list and not a single value from the very first manifest we wrote.

None of the above replaces a person running kubectl apply by hand forever. A CI pipeline that builds the operator's image, pushes it, and applies the manifests on merge to main is the natural next step. This is ordinary CI/CD, nothing operator-specific about it once the manifests themselves are in Git.

Performance & Resilience

By default, a controller only processes one reconcile at a time. That's fine while we're the only ones testing it, but with hundreds of VirtualMachine objects it means that most of them sit in the workqueue waiting their turn even though nothing about reconciling one blocks reconciling another.

MaxConcurrentReconciles raises that:

func (r *VirtualMachineReconciler) SetupWithManager(mgr ctrl.Manager) error {
	return ctrl.NewControllerManagedBy(mgr).
		For(&computev1.VirtualMachine{}, builder.WithPredicates(predicate.GenerationChangedPredicate{})).
		Owns(&corev1.Secret{}).
		Owns(&corev1.Service{}).
		Watches(&computev1.ProviderConfig{}, handler.EnqueueRequestsFromMapFunc(r.findVirtualMachinesForProviderConfig)).
		WithOptions(controller.Options{MaxConcurrentReconciles: 5}). // five VMs in flight instead of one
		Complete(r)
}

Caching only helps one side of this reconciler. Reading vm back from r.Get is already fast and local, as informers keep that in memory. But r.Provider.Get is a real HTTP round trip every single time, and there's no cache in front of it.

That asymmetry is worth sitting with, because it's the same one from Part 2: in-cluster reads are cheap because Kubernetes built the caching layer for us, and external reads are exactly as expensive as whatever's on the other end of the wire. We could add a short-lived cache in front of the provider client, but it comes with a real cost: a cached Running for a VM that just failed is a lie our status will repeat until the cache expires.

The provider not having a cache in front of it also means nothing is stopping us from hammering it. A burst of reconciles, say every VirtualMachine getting touched at once after a cluster restart, turns into a burst of HTTP calls with no coordination between them. Wrapping the client in a rate limiter caps that independently of whatever backoff the workqueue is already doing on failures:

type Client struct {
	baseURL string
	http    *http.Client
	limiter *rate.Limiter // shared across every reconcile using this client
}

func (c *Client) Create(ctx context.Context, image string) (*VM, error) {
	if err := c.limiter.Wait(ctx); err != nil {
		return nil, err
	}
	// ... existing HTTP call
}

Leader election is the other half of running more than one replica safely. We turned this down to a concept in Part 2, but in practice it's two fields on the manager:

mgr, err := ctrl.NewManager(cfg, ctrl.Options{
	LeaderElection:   true,
	LeaderElectionID: "vmoperator-leader",
})

With this set, every replica starts up, but only the one holding the lease actually reconciles. The rest sit ready to take over the moment it doesn't renew in time.

Concurrency also surfaces a race we glossed over in Part 3. Say the reconciler calls r.Provider.Create, the provider creates the VM and returns its id, and then the process crashes before r.Status().Update ever runs. vm.Status.ID is still empty, so the next reconcile sees an object with no VM yet and calls Create again. Now the provider has two VMs for one VirtualMachine.

Nothing about MaxConcurrentReconciles or leader election prevents this. It's a gap in the create step itself, and it only shows up once something can fail between the external call and the write that records it.

Closing it for real means the provider needs to accept an idempotency key, generated once and stored on the object before the first Create call, so a retried create recognizes that it already happened instead of making a second VM.

Security

The ClusterRole from Part 3 works, but it's broader than it needs to be. It grants every verb on secrets and services cluster-wide, when the operator only ever touches the ones it owns.

A tighter version scopes to a single namespace with Role/RoleBinding instead of ClusterRole/ClusterRoleBinding wherever VMOperator is only expected to run in one, and it drops verbs we never call. We never list or watch arbitrary Secrets outside our own, only the ones we create. This is also the RBAC the manifests applied in the previous section were referring to.

Credentials are the other gap. ProviderConfig currently holds a plaintext endpoint, and a real provider needs an API key alongside it, which has no business sitting in a CRD spec anyone with read access to the object can see. It belongs in a Secret, referenced by name instead of embedded:

type ProviderConfigSpec struct {
	Endpoint  string                      `json:"endpoint"`
	SecretRef corev1.LocalObjectReference `json:"secretRef"` // Secret holding the provider's API key
}

The reconciler resolves SecretRef at the point it builds the provider client, reads the key out of the Secret's data, and never logs it or writes it back to anything with wider read access, including the VirtualMachine's own status.

The last piece is the operator's own pod. A container security context that runs as a non-root user sets a read-only root filesystem, drops Linux capabilities it doesn't need, and shrinks what's possible if the binary itself is ever compromised. This is standard practice for any workload, not something specific to operators.

If we'd added an admission webhook anywhere in this guide, its certificates would belong here too. We didn't need one for VMOperator, so we'll leave that as a pointer rather than something to configure.

Observability

The manager exposes a Prometheus endpoint without us writing anything for it. Workqueue depth, reconcile duration, and reconcile error counts are already there per controller.

What isn't there automatically is anything about the provider, so we add a metric the same way any Go service would:

var providerCallDuration = prometheus.NewHistogramVec(
	prometheus.HistogramOpts{
		Name: "vmoperator_provider_call_duration_seconds",
		Help: "Duration of calls to the VM provider, by operation",
	},
	[]string{"operation"},
)

func init() {
	metrics.Registry.MustRegister(providerCallDuration) // shares the manager's existing /metrics endpoint
}

Wrapping each provider call with a timer around this turns "is the provider slow" from a question we'd have to guess at into one we can graph.

Logging benefits from the same instinct. log.FromContext(ctx) inside Reconcile already carries the VirtualMachine's name and namespace on every line if we set that up once in SetupWithManager. Adding vm.Status.ID to that logger right after it's set means every subsequent log line for that reconcile also carries the provider's own identifier for the VM. That one field is what makes it possible to grep a mock provider log and an operator log for the same request and find both sides of the same failure.

Next Steps

In this guide, you learned what a Kubernetes operator is, how to build one from scratch, and how to prepare it for production. You also learned about finalizers, predicates, owned resources, and cross-resource reconciliation along the way.

None of this is specific to managing VMs. The next operator, whatever it manages, is the same shape.

You can also review the resources below to keep learning:

• K8s custom resources

• client-go

• controller-runtime

• Docker docs

You're running a causal inference experiment whether you acknowledge it or not, and your routing rules are quietly poisoning your performance estimates.

Consider a gateway that routes incoming queries to either a premium model or a faster, cheaper alternative based on a confidence threshold. Queries with a confidence score below a certain threshold get routed premium, while queries above that threshold go cheap.

You pull the logs, run a regression of task_completed on the routing decision, and find that premium routing yields a 14-percentage-point lift. Based on this number, your infrastructure team might start drafting a proposal to route everything premium.

Stop before you send that proposal. The routing rule correlates strongly with query complexity, which directly determines whether a task is completed. Complex queries are harder and fail more often, regardless of which model handles them.

When you regress task completion on premium routing, you measure two entangled phenomena simultaneously: the causal effect of sending a query to the premium model, and the inherent difference in difficulty between the queries each model receives.

Standard regression blends those two signals into a single coefficient, and the observed lift reflects query difficulty just as much as it reflects model quality.

The routing confounder arises whenever assignment correlates with query characteristics, as is to be expected in any routing system doing its job. The assignment rule ensures that the two treatment arms contain systematically different queries, invalidating the naïve comparison as a causal estimate.

Instrumental variable analysis is the method that breaks this deadlock. You need a third variable that influences routing for reasons completely unrelated to query quality.

Rate-limit-triggered fallbacks are exactly that. When the premium model hits a rate limit, the gateway reroutes the query to the cheaper model regardless of the query's characteristics. The rate limit fires for infrastructure reasons, independent of what a user actually asked. That randomness is an instrument, and two-stage least squares (2SLS) lets you extract a clean causal estimate from it.

This tutorial walks through the full diagnosis-to-fix sequence in Python: why the routing confounder biases OLS, how to build 2SLS from scratch across two chained regressions, how to check instrument strength with the first-stage F-statistic, and how to recover the local average treatment effect that 2SLS actually estimates rather than mistaking it for the average treatment effect. By the end, you'll know how to spot a confounded routing decision in your own logs, construct a valid instrument from an infrastructure signal like rate-limit fallbacks, and produce a causal estimate with correctly sized confidence intervals instead of the overconfident ones manual 2SLS gives you by default.

Companion notebook: every code block in this article runs end-to-end in iv_demo.ipynb in the companion repo at github.com/RudrenduPaul/product-experimentation-causal-inference-genai-llm/tree/main/11_instrumental_variables/.

Table of Contents

• Why Routing Confounds Regression

• What an Instrumental Variable is

• Prerequisites

• Setting Up the Working Example

• Step 1: Naïve OLS (Biased Baseline)

• Step 2: Two-Stage Least Squares (2SLS) from Scratch

• Step 3: Weak-Instrument Diagnostics

• Step 4: The LATE is the Quantity You Actually Care About

• Step 5: Bootstrap Confidence Intervals

• When Instrumental Variables Fail

• What to Do Next

Why Routing Confounds Regression

A routing system makes a correlated decision. Queries that arrive with low confidence scores, long token counts, or complex multi-step intent get routed to premium. Queries that are short, clear, and well within the cheap model's capability get routed cheap. That correlation is the whole point of the routing layer.

The problem is that the same features driving the routing decision also affect the outcome you care about.

Task completion is harder for complex queries, independent of which model processes them. When you write task_completed ~ routed_to_premium + controls, the controls term can absorb the observable dimensions of complexity: query length, user engagement tier, and whatever you logged.

The unobservable dimensions stay embedded in the routed_to_premium coefficient, and they bias the estimate downward (complex queries routed premium complete less often, making premium look worse than it is) or upward, depending on the direction of the confound.

In the synthetic dataset used in this tutorial, the OLS estimate lands at +3.3 percentage points even though the true causal effect is +6 percentage points. This is a downward bias of 2.7 pp driven entirely by unobserved query complexity.

The regression looks confident, the p-value looks significant, and nothing in the standard OLS output flags the problem. That's what makes this failure mode dangerous: it's invisible in standard regression diagnostics.

2SLS is built for exactly this structure. You need an external source of variation in routing that is uncorrelated with query quality. Rate-limit-triggered fallbacks provide it.

Figure 1: The IV causal structure. The instrument (Z = rate-limit fallback) satisfies relevance (Z predicts routing), exclusion (no direct Z to outcome path), and independence (Z is uncorrelated with unobserved query complexity). The dashed red arrows show the confounder paths that bias naïve OLS.

What an Instrumental Variable is

An instrument is a variable that shifts your endogenous variable (routing decision) without any other direct path to your outcome (task completion).

Four assumptions define a valid instrument.

Relevance

The instrument must actually influence the endogenous variable. A rate-limit fallback indicator that fires on 15 percent of premium-eligible queries will meaningfully affect whether those queries get routed to premium.

This assumption is testable: check it with the first-stage F-statistic. The conventional threshold is F > 10, established by Staiger and Stock (1997), corresponding to approximately a 10% maximum bias in the 2SLS estimator relative to OLS in the worst case.

Note that more recent work by Andrews, Stock, and Sun (2019) suggests this threshold may be too permissive in settings with smaller samples or multiple instruments. For production analyses with limited fallback data, treat F > 10 as a minimum floor and verify with additional sensitivity checks before reporting results. Below 10, the instrument is definitively weak, and the estimate is unreliable.

Exclusion Restriction

The instrument must affect the outcome solely through its effect on routing. The rate-limit fallback completes the task entirely by changing which model handles the query, with no separate direct path.

This assumption requires logical business reasoning and can't be verified from data alone. A fallback triggered by aggregate infrastructure load is unrelated to what a user asked or how hard their task was.

Independence

The instrument must be independent of all confounders. Rate-limit events are driven by aggregate API traffic and are unrelated to the characteristics of any individual query. The probability that a given query triggers a rate-limit fallback is uncorrelated with query complexity, user tier, or any other confounder. This assumption too must be argued logically.

Monotonicity

The instrument must move all affected units in the same direction. For rate-limit fallbacks, every affected query switches from premium to cheap, but no query switches from cheap to premium due to a fallback. This rules out defiers and is required for the LATE interpretation to hold.

When all four hold, 2SLS extracts a causal estimate of routing's effect on task completion by using only the exogenous variation in routing generated by the instrument.

Prerequisites

You need:

• Python 3.11 or newer

• Comfort with pandas and statsmodels OLS

• Rough familiarity with linear regression (2SLS is two OLS regressions chained together)

Install the packages for this tutorial:

pip install numpy pandas statsmodels scipy

This installs the four packages used in the tutorial. statsmodels provides OLS and the formula API, and scipy is used for statistical computations in the bootstrap step.

Clone the companion repo to get the synthetic dataset:

git clone https://github.com/RudrenduPaul/product-experimentation-causal-inference-genai-llm.git
cd product-experimentation-causal-inference-genai-llm
python data/generate_data.py --seed 42 --n-users 50000 --out data/synthetic_llm_logs.csv

You clone the companion repo and regenerate the shared 50,000-user synthetic dataset with a fixed seed so your results match the expected outputs in this article.

Setting Up the Working Example

This tutorial adds three simulated variables on top of the shared dataset's user covariates, constructing the full IV causal graph in code:

• rate_limit_fallback: the instrument Z. Sampled as a pure Bernoulli(0.15), completely independent of all query characteristics.

• routed_to_premium_actual: the endogenous treatment D. Routing is driven by both query_confidence (observable) and query_complexity (unobservable), so OLS is biased.

• task_completed_iv: the outcome Y. Re-simulated from the IV causal graph with a known +6 pp premium routing effect, letting you verify that the estimator recovers the ground truth.

A transparency note: in a real production analysis, the rate-limit fallback events come from your API gateway logs. Your user telemetry table won't have them. You'd join those two sources to construct the instrument.

The simulation here preserves the structural properties of a real instrument: it fires for infrastructure reasons, independent of query quality, without requiring production gateway logs.

import numpy as np
import pandas as pd
import statsmodels.formula.api as smf

np.random.seed(42)

df = pd.read_csv("data/synthetic_llm_logs.csv")
rng = np.random.default_rng(99)
n = len(df)

# Unobserved confounder: complex queries route premium AND complete less often
query_complexity = rng.normal(0, 1, n)

# Endogenous routing: depends on query_confidence (observable)
# and query_complexity (unobserved): this is the confounding structure
log_odds = -2.0 + 4.0 * (1.0 - df["query_confidence"]) + 0.6 * query_complexity
premium_prob = 1.0 / (1.0 + np.exp(-log_odds))
df["routed_to_premium_iv"] = rng.binomial(1, premium_prob).astype(int)

# Instrument: pure Bernoulli(0.15), independent of all query characteristics
df["rate_limit_fallback"] = rng.binomial(1, 0.15, n)

# Actual routing: premium if intended, unless fallback overrides
df["routed_to_premium_actual"] = (
    df["routed_to_premium_iv"] * (1 - df["rate_limit_fallback"])
).astype(int)

# Outcome: known causal structure with +0.06 premium effect
engagement_base = np.where(df.engagement_tier == "heavy", 0.70,
                  np.where(df.engagement_tier == "medium", 0.55, 0.35))
completion_prob = np.clip(
    engagement_base
    + 0.06 * df["routed_to_premium_actual"]  # true causal effect
    - 0.04 * query_complexity                 # unobserved confounder
    + rng.normal(0, 0.02, n),
    0.01, 0.99
)
df["task_completed_iv"] = rng.binomial(1, completion_prob).astype(int)

# Encode engagement tier as dummies
df = pd.get_dummies(df, columns=["engagement_tier"], drop_first=True)
tier_dummies = [c for c in df.columns if c.startswith("engagement_tier_")]
covariate_str = " + ".join(["query_confidence"] + tier_dummies)

print(f"Rate-limit fallback rate:      {df.rate_limit_fallback.mean():.3f}")
print(f"Premium routing rate (actual): {df.routed_to_premium_actual.mean():.3f}")
print(f"Mean confidence | fallback=1:  {df[df.rate_limit_fallback==1].query_confidence.mean():.3f}")
print(f"Mean confidence | fallback=0:  {df[df.rate_limit_fallback==0].query_confidence.mean():.3f}")

Expected output:

Rate-limit fallback rate:      0.151
Premium routing rate (actual): 0.271
Mean confidence | fallback=1:  0.716
Mean confidence | fallback=0:  0.715

In the above code, the nearly identical mean confidence scores between the fallback=1 and fallback=0 groups confirm that the instrument is independent of the observable routing signal. This is the independence assumption check you can run on any proposed instrument. query_complexity is available in this simulation but would be unobserved in production. The regression never receives it.

Step 1: Naïve OLS (Biased Baseline)

Running a standard regression first establishes the biased baseline you'd encounter without accounting for the confounding structure. Most engineering teams report this number without realizing it's mathematically compromised.

ols_formula = f"task_completed_iv ~ routed_to_premium_actual + {covariate_str}"
ols_model = smf.ols(ols_formula, data=df).fit(cov_type="HC3")

ols_coef = ols_model.params["routed_to_premium_actual"]
ols_se   = ols_model.bse["routed_to_premium_actual"]
ols_pval = ols_model.pvalues["routed_to_premium_actual"]
print(f"OLS estimate of premium routing effect: {ols_coef:+.4f}")
print(f"HC3 standard error:                      {ols_se:.4f}")
print(f"p-value:                                 {ols_pval:.4f}")

Expected output:

OLS estimate of premium routing effect: +0.0327
HC3 standard error:                      0.0050
p-value:                                 0.0000

Here's what's happening: OLS recovers +3.3 percentage points (probability units, since task_completed_iv is a 0/1 binary outcome in a linear probability model). The true causal effect is +6.0 pp. The 2.7 pp bias comes from unobserved query-complexity routing: harder queries are routed to premium, and they complete less often, which is a downward confounding mechanism. The p-value looks significant, and the standard error looks precise. Nothing in this output tells you the estimate is wrong.

Keep this number in mind: the 2SLS result in Step 2 will reveal the gap.

Step 2: Two-Stage Least Squares (2SLS) from Scratch

Two-stage least squares corrects the bias by isolating the exogenous routing variation generated by rate-limit fallbacks, using only that variation to estimate the causal effect.

Stage 1: Predict Routing from the Instrument and Covariates.

stage1_formula = f"routed_to_premium_actual ~ rate_limit_fallback + {covariate_str}"
stage1 = smf.ols(stage1_formula, data=df).fit(cov_type="HC3")

print(f"Stage 1 instrument coefficient: {stage1.params['rate_limit_fallback']:+.4f}")
print(f"p-value:                         {stage1.pvalues['rate_limit_fallback']:.4f}")

df["rtp_hat"] = stage1.fittedvalues

Expected output:

Stage 1 instrument coefficient: -0.3190
p-value:                         0.0000

In this code, you regress the endogenous routing variable on the instrument and the same observed covariates you'll use in Stage 2.

The fitted values rtp_hat contain two components: the exogenous variation the instrument explains, and the exogenous variation the covariates explain.

The endogenous component (the variation correlated with unobserved query complexity) stays in the residuals and drops out of rtp_hat. The negative coefficient on rate_limit_fallback confirms the relevance assumption: when the fallback fires, premium routing probability drops by about 32 percentage points.

Stage 2: Regress Outcome on the Predicted Routing.

stage2_formula = f"task_completed_iv ~ rtp_hat + {covariate_str}"
stage2 = smf.ols(stage2_formula, data=df).fit(cov_type="HC3")

tsls_coef = stage2.params["rtp_hat"]
tsls_se   = stage2.bse["rtp_hat"]
print(f"2SLS estimate:              {tsls_coef:+.4f}")
print(f"Stage-2 SE (underestimate): {tsls_se:.4f}")

Expected output:

2SLS estimate:              +0.0599
Stage-2 SE (underestimate): 0.0188

Here's what's happening: replacing routed_to_premium_actual with rtp_hat removes the endogenous part of the routing variation. The Stage 2 coefficient (+0.0599) is the 2SLS estimate of the causal effect of premium routing on task completion, almost exactly the +0.06 ground truth.

Here's an important caveat on standard errors: manual 2SLS produces Stage 2 SEs that are too small. Stage 2 OLS treats rtp_hat as a fixed, known regressor, when in fact it was estimated from the data in Stage 1. That estimation error adds a variance component that Stage 2's residuals never see.

For any result you report to stakeholders, use linearmodels.IV2SLS (shown in "What to do next"), which computes the correct sandwich variance.

Compare OLS and 2SLS Side by Side:

print(f"OLS estimate (biased):  {ols_coef:+.4f}")
print(f"2SLS estimate (IV):     {tsls_coef:+.4f}")
print(f"True premium effect:   +0.0600")
print(f"OLS bias:               {ols_coef - 0.06:+.4f}")

Expected output:

OLS estimate (biased):  +0.0327
2SLS estimate (IV):     +0.0599
True premium effect:   +0.0600
OLS bias:               -0.0273

Here, OLS misses the true effect by 2.7 pp, a 45% underestimate. 2SLS recovers it to within 0.01 pp. The direction of the gap matches the confounding mechanism: unobserved query complexity routes hard queries to premium and reduces their completion, pulling the OLS coefficient downward.

Figure 2: Data-driven results on the 50,000-user synthetic dataset. Left panel: routing rates by fallback group confirm the first-stage relationship: fallback=0 queries route premium at 39.1%, fallback=1 queries at 0% (complete override). Right panel: OLS CI (red) misses the true +0.06 pp effect entirely. 2SLS CI (green) covers it. The wider 2SLS interval reflects the variance cost of relying solely on the instrument's exogenous variation.

Step 3: Weak-Instrument Diagnostics

A valid instrument that has little effect on outcomes is a weak instrument. Weak instruments produce 2SLS estimates with enormous variance that drift toward the OLS estimate in small samples, which defeats the purpose. The standard diagnostic is the first-stage F-statistic.

stage1_restricted = smf.ols(
    f"routed_to_premium_actual ~ {covariate_str}", data=df
).fit()

f_stat, f_pval, _ = stage1.compare_f_test(stage1_restricted)
print(f"First-stage F-statistic (instrument): {f_stat:.2f}")
print(f"p-value:                               {f_pval:.4f}")

if f_stat > 10:
    print("Instrument is STRONG (F > 10). 2SLS estimates are reliable.")
elif f_stat > 4:
    print("Instrument is BORDERLINE WEAK (4 < F < 10). Interpret with caution.")
else:
    print("Instrument is WEAK (F < 4). 2SLS estimates are unreliable.")

print(f"\nFirst-stage coefficient on instrument: "
      f"{stage1.params['rate_limit_fallback']:+.4f}")

Expected output:

First-stage F-statistic (instrument): 3780.94
p-value:                               0.0000
Instrument is STRONG (F > 10). 2SLS estimates are reliable.

First-stage coefficient on instrument: -0.3190

In the above code, you compare the full Stage 1 model (with the instrument) to a restricted model (without it) using an F-test. An F of 3780 is overwhelmingly above the Staiger-Stock rule of thumb. The 15% fallback rate applied to 50,000 observations yields a large, precisely estimated first-stage effect.

On a real production dataset with lower fallback rates or a smaller dataset, the F-statistic will be lower. If you get an F-statistic below 10, either find a stronger instrument or add more fallback data before drawing conclusions.

There's a trade-off between instrument strength and exclusion validity that's worth flagging explicitly. You can make an instrument stronger by increasing the fallback rate, but if you push it high enough to affect user experience, the fallback starts to directly affect task completion through satisfaction and retry behavior, which violates the exclusion restriction. A strong instrument that satisfies both relevance and exclusion is the goal.

The endogeneity direction check:

gap = ols_coef - tsls_coef
print(f"OLS minus 2SLS gap: {gap:+.4f}")
if abs(gap) > 0.005:
    print("Gap suggests endogeneity bias is present in OLS.")
else:
    print("Small gap: OLS and 2SLS broadly agree.")
print("For a formal Hausman endogeneity test, use linearmodels IV2SLS.")

Expected output:

OLS minus 2SLS gap: -0.0272
Gap suggests endogeneity bias is present in OLS.
For a formal Hausman endogeneity test, use linearmodels IV2SLS.

Here's what's happening: the gap between OLS and 2SLS is the diagnostic for endogeneity. A gap of 2.7 pp confirms that the routing variable is genuinely correlated with unobserved confounders, and that OLS was absorbing part of the confounder's effect.

For a formally valid Hausman test (one that produces a chi-squared statistic with a known distribution under the null), use linearmodels.IV2SLS's built-in test. The direction check above is a quick diagnostic only.

Step 4: The LATE is the Quantity You Actually Care About

2SLS estimates the Local Average Treatment Effect (LATE), also called the Complier Average Causal Effect (CACE). The LATE applies only to compliers: the specific subset of queries whose routing actually changes when the instrument fires. Rate-limit fallbacks affect only premium-eligible queries that experience a fallback, so the LATE is specific to that subpopulation.

compliers_mask = df["rate_limit_fallback"] == 1
complier_count = compliers_mask.sum()
complier_pct   = complier_count / n * 100

print(f"Approximate complier population: {complier_count:,} ({complier_pct:.1f}% of queries)")
print(f"\nComplier mean confidence:     {df[compliers_mask]['query_confidence'].mean():.3f}")
print(f"Non-complier mean confidence: {df[~compliers_mask]['query_confidence'].mean():.3f}")
print(f"\n2SLS LATE estimate: {tsls_coef:+.4f}")
print("This is the causal effect of premium routing for queries rerouted")
print("by rate-limit fallbacks, not all queries in the dataset.")

Expected output:

Approximate complier population: 7,575 (15.2% of queries)

Complier mean confidence:     0.716
Non-complier mean confidence: 0.715

2SLS LATE estimate: +0.0599
This is the causal effect of premium routing for queries rerouted
by rate-limit fallbacks, not all queries in the dataset.

In this code, the complier population is 7,575 queries (those that experienced a rate-limit fallback and were rerouted from premium to cheap). Their mean confidence (0.716) is nearly identical to the non-complier group (0.715), confirming that the fallback fired independently of query characteristics.

When compliers look like a representative slice of all queries on observables, the LATE is often a reasonable approximation of the average treatment effect (ATE).

Observable representativeness meets the minimum diagnostic standard. The formal LATE-to-ATE condition requires either homogeneous treatment effects across all units or a valid instrument for every unit in the population. If your routing effect is heterogeneous across query types (premium routing helps complex queries far more than simple ones, for instance), the LATE can diverge substantially from the ATE, even when the complier mean confidence looks similar to that of the non-complier group.

For strategic capacity planning, this is exactly the metric you need. When you ask whether to invest in greater premium model capacity or adjust rate limits, you're asking a specific question about the queries currently constrained by your infrastructure. 2SLS answers that question directly.

Step 5: Bootstrap Confidence Intervals

Manual 2SLS produces Stage 2 standard errors that are too small, as explained in Step 2. Bootstrap CIs give you reliable uncertainty estimates without needing to derive the correct analytic variance formula. The bootstrap resamples the full two-stage procedure together, capturing the sampling variance from both stages.

rng_boot = np.random.default_rng(7)
ols_boot, tsls_boot = [], []

for _ in range(500):
    samp = df.sample(len(df), replace=True,
                     random_state=int(rng_boot.integers(1_000_000_000)))

    # OLS bootstrap
    ols_b = smf.ols(
        f"task_completed_iv ~ routed_to_premium_actual + {covariate_str}",
        data=samp
    ).fit()
    ols_boot.append(ols_b.params["routed_to_premium_actual"])

    # 2SLS bootstrap (two stages together)
    s1b = smf.ols(
        f"routed_to_premium_actual ~ rate_limit_fallback + {covariate_str}",
        data=samp
    ).fit()
    samp = samp.copy()
    samp["rtp_hat"] = s1b.fittedvalues
    s2b = smf.ols(
        f"task_completed_iv ~ rtp_hat + {covariate_str}",
        data=samp
    ).fit()
    tsls_boot.append(s2b.params["rtp_hat"])

ols_ci  = (np.percentile(ols_boot, 2.5),  np.percentile(ols_boot, 97.5))
tsls_ci = (np.percentile(tsls_boot, 2.5), np.percentile(tsls_boot, 97.5))
true_eff = 0.0600

print(f"OLS  95% CI: [{ols_ci[0]:+.4f}, {ols_ci[1]:+.4f}]")
print(f"2SLS 95% CI: [{tsls_ci[0]:+.4f}, {tsls_ci[1]:+.4f}]")
print(f"Ground truth: +{true_eff:.4f}")
print(f"OLS CI covers ground truth:  {ols_ci[0] <= true_eff <= ols_ci[1]}")
print(f"2SLS CI covers ground truth: {tsls_ci[0] <= true_eff <= tsls_ci[1]}")

Expected output:

OLS  95% CI: [+0.0227, +0.0426]
2SLS 95% CI: [+0.0247, +0.0969]
Ground truth: +0.0600
OLS CI covers ground truth:  False
2SLS CI covers ground truth: True

In this code, the OLS 95% CI ([+0.023, +0.043]) entirely misses the true +0.06 effect. Every value in that interval is below the ground truth: OLS is confidently wrong. The 2SLS CI ([+0.025, +0.097]) covers the ground truth. It's wider than the OLS interval, reflecting the variance cost of IV estimation: you pay in precision to gain in validity.

The bootstrap resamples both stages in each iteration, so the uncertainty correctly accounts for the two-stage structure. Use bootstrap CIs when reporting 2SLS results from a manual implementation, as they're more reliable than the Stage 2 parametric SE.

When Instrumental Variables Fail

IV analysis has failure modes more insidious than those of propensity scores or regression discontinuity, because two of the four assumptions are untestable from data alone.

Weak Instruments

A first-stage F below 10 signals a serious identification problem. Weak instruments cause the 2SLS estimator to have large variance and drift toward OLS in finite samples, replicating the biased baseline while appearing to do something more sophisticated. Check the F-statistic before interpreting any IV result.

If F is below 10, find a stronger instrument or report the estimate with an explicit weak-instrument warning. The instrument here is strong (F = 3780) because the 15% fallback rate applied to 50,000 queries yields 7,500+ routing changes.

Exclusion Restriction Violations

If the rate-limit fallback affects task completion through any channel other than the routing decision, exclusion fails.

There are two plausible violations: fallback events cluster during high-traffic periods when users are also more likely to be doing complex batch jobs, making the instrument correlated with query difficulty after all. Or users who experience a fallback notice the degraded response quality and abandon the session, creating a direct Z to Y path through user frustration.

Both violate exclusion while leaving relevance intact. You can't test them from data. You have to argue from system knowledge.

LATE vs. ATE Confusion

Using the LATE estimate to justify a broad routing policy change is wrong if compliers are atypical. If rate-limit fallbacks disproportionately hit complex queries (because complex queries take longer and are more likely to hit a rate limit mid-session), the LATE covers the causal effect of premium routing for that complex-query subpopulation.

Reporting it as if it were the ATE overstates the benefit of routing all queries premium. The complier characteristics table in Step 4 is the diagnostic: if compliers and non-compliers look similar on observables, the LATE is a credible approximation of the ATE.

Defiers and the Monotonicity Assumption

The LATE interpretation requires monotonicity: the instrument moves all affected units in the same direction. For rate-limit fallbacks, this is almost certainly satisfied, since a fallback always reduces the probability of premium routing for the affected query.

If some compensating mechanism exists (say, a fallback: one query triggers a priority boost on the next), you have defiers, and the monotonicity assumption breaks down. Verify directional consistency before trusting the LATE.

What to Do Next

The manual 2SLS implementation in this tutorial is transparent about the mechanism but produces incorrect standard errors. For any result you report to stakeholders or include in a published analysis, use linearmodels.IV2SLS:

# Production-grade 2SLS with correct standard errors
# pip install linearmodels
from linearmodels.iv import IV2SLS

exog_vars = ["query_confidence"] + tier_dummies
iv_model = IV2SLS.from_formula(
    f"task_completed_iv ~ 1 + {' + '.join(exog_vars)} "
    f"[routed_to_premium_actual ~ rate_limit_fallback]",
    data=df
).fit(cov_type="robust")

print(iv_model.summary)

Here's what's happening: linearmodels computes the correct 2SLS variance that accounts for the two-stage structure, runs a proper first-stage diagnostic summary, and provides a formal Hausman endogeneity test. The syntax brackets the endogenous variable and instrument: [D ~ Z].

The full implementation (including bootstrap confidence intervals and the visualization in Figure 2) is in the companion notebook at github.com/RudrenduPaul/product-experimentation-causal-inference-genai-llm/tree/main/11_instrumental_variables/. Clone the repo, generate the synthetic dataset, and run iv_demo.ipynb to reproduce every code block end-to-end.

One final note on when to reach for IV at all: if your system supports forced routing randomization (randomly assigning a fraction of queries to premium regardless of confidence score), a standard A/B test is simpler and produces a full-fleet ATE estimate.

IV is the right tool when randomization is infeasible: when the routing rule is baked into production logic, when you can't afford to deliberately route queries suboptimally, or when you need to use historical observational data. If you can run a true experiment, run it.

Confounding is the structural default for any optimized routing system. Standard regression folds model quality and inherent query difficulty into a single coefficient, measuring both at once when you need them separated.

Rate-limit fallbacks provide the clean, natural instrument that filters infrastructure noise from routing signal. This approach gives your team a defensible causal estimate of how your model architecture actually drives business value.

But relying on a single AI model for every user request creates serious production risks. API outages happen. Proprietary models can be expensive for simple tasks. And cheaper open-source models might struggle with complex logical reasoning.

When my team and I built an enterprise-grade AI engine for our customer support platform, we relied on a single top-tier model for everything.

Within a month, we faced two massive issues: a widespread API outage completely froze our app, and our monthly API bill rose because we used expensive reasoning models to answer simple FAQs.

To fix this, I built a resilient, multi-model orchestrator. In this guide, you'll learn how to build an intelligent, multi-tiered AI application using Python that routes prompts dynamically and handles model fallbacks automatically.

• What We'll Cover

• Prerequisites and Environment Setup

  • Package Installation

  • Local Directory Structure

  • Environment Configuration

• The Problem with Single-Model Architectures

• Understanding the Dynamic Model Routing Lifecycle

• Step 1: Implementing Tier 1 – Prompt Complexity & Intent Analysis

  • Breaking Down the Code Logic for Tier 1

• Step 2: Implementing Tier2– Dynamic Model Routing Logic

  • Breaking Down the Code Logic for Tier 2

• Step 3: Implementing Tier3 – Automatic Fallbacks

  • Breaking Down the Code Logic for Tier 3

  • Combining the Architecture into a Unified Execution Pipeline

  • Breaking Down the Code Logic

• Lessons Learnt from Dynamic Model Switching in Production

• Conclusion

  • Thank You for Reading!

Prerequisites and Environment Setup

To follow along with this tutorial, you should have the following setup:

• Basic proficiency with Python and asynchronous programming.

• Python 3.9 or higher installed on your system.

• A code editor such as Visual Studio Code.

• API keys for at least two model providers (for example, OpenAI and Anthropic), or local models running via Ollama.

Package Installation

Open your terminal and install the required dependencies:

pip install openai anthropic python-dotenv pydantic

Local Directory Structure

Organize your project directory like this to keep your code clean:

ai-model-router/

│

├── .env

├── README.md

└── app.py

Environment Configuration

Create a .env file in the root of your project directory and add your credentials:

Ini, TOML

OPENAI_API_KEY=your_openai_api_key_here ANTHROPIC_API_KEY=your_anthropic_api_key_here ENVIRONMENT=development

The Problem with Single-Model Architectures

If you route every query to a flagship model like GPT-4o or Claude 3.5 Sonnet, you'd be overspending on simple tasks. Conversely, if you route everything to a smaller, faster model like GPT-4o-mini or Claude 3.5 Haiku to save money, your system will fail when users submit complex code-generation or analytical tasks.

On top of cost concerns, single-model systems suffer from single points of failure. When an API provider goes down or rate-limits your account, your entire application crashes.

To solve this, you need an orchestration layer that evaluates prompt complexity before invoking an LLM, routes the request to the most cost-effective model, and falls back to a secondary provider if the primary provider fails.

Understanding the Dynamic Model Routing Lifecycle

Here's how a user request journeys through a dynamic multi-model system:

First, you have the complexity analysis. The system inspects the incoming prompt using lightweight metrics to assign a task tier (Simple, Medium, or Complex).

Second, you have the model routing. The system maps the tier to the appropriate model (for example, lightweight tasks go to Haiku/Mini while heavy reasoning goes to Sonnet/GPT-4o).

You also have an automatic fallback: if the primary provider times out or throws an API error, the system automatically redirects the query to an equivalent fallback model.

Step 1: Implementing Tier 1 – Prompt Complexity & Intent Analysis

First, you need a deterministic, fast way to classify prompts without making an expensive API call just to decide which model to use.

Before spending money on an LLM API call just to figure out what the user wants, we can look at the text directly in code. Think of this step as a smart gatekeeper. By checking simple things like text length, code snippets, or tricky keywords, we can figure out how hard the task is in milliseconds and for free.

Here's how we set up our classification rules inside app.py:

import re
from enum import Enum
from pydantic import BaseModel


class TaskComplexity(Enum):
    SIMPLE = "simple"      # FAQs, short summaries, basic translation
    MEDIUM = "medium"      # Standard text generation, content rewriting
    COMPLEX = "complex"    # Code writing, math logic, structural analysis


class PromptAnalyzer:
    def __init__(self):
        # Regex patterns indicative of complex tasks
        self.complex_keywords = [
            r"\brefactor\b",
            r"\bdebug\b",
            r"\bwrite code\b",
            r"\banalyze\b",
            r"\balgorithm\b",
            r"\barchitecture\b",
        ]

    def analyze_complexity(self, prompt: str) -> TaskComplexity:
        """
        Evaluates input text deterministically to output
        a TaskComplexity rating.
        """
        normalized = prompt.lower().strip()
        word_count = len(normalized.split())

        # Check for code blocks or complex request patterns
        contains_code = "```" in prompt
        has_complex_keyword = any(
            re.search(pattern, normalized)
            for pattern in self.complex_keywords
        )

        if contains_code or has_complex_keyword or word_count > 300:
            return TaskComplexity.COMPLEX
        elif word_count > 80:
            return TaskComplexity.MEDIUM
        else:
            return TaskComplexity.SIMPLE


# Example Usage
if __name__ == "__main__":
    analyzer = PromptAnalyzer()

    test_prompt = (
        "Write a Python script that implements a trie "
        "data structure with autocomplete."
    )

    complexity = analyzer.analyze_complexity(test_prompt)
    print(f"Prompt Complexity Tier: {complexity.value}")

Breaking Down the Code Logic for Tier 1

• TaskComplexity Enum: Defines explicit categories for incoming requests (SIMPLE, MEDIUM, COMPLEX), giving us type safety across our pipeline.

• Keyword Matching: The PromptAnalyzer class sets up regex patterns looking for action words like refactor, debug, or algorithm that signal a heavy reasoning task.

• Deterministic Rules in analyze_complexity:

• Formatting & Length Check: We clean the string, check for Markdown code blocks (```), and calculate word counts.

• Tier Allocation:

  • If the prompt contains code blocks, trigger words, or exceeds 300 words, it immediately escalates to COMPLEX.

  • If it is between 80 and 300 words without code keywords, it maps to MEDIUM.

  • Anything shorter defaults to SIMPLE.

Running this snippet with a complex query checks the text, spots "write code," and outputs:

Prompt Complexity Tier: complex

Step 2: Implementing Tier2– Dynamic Model Routing Logic

Now that we can successfully label a prompt as simple, medium, or complex, we need a rulebook to decide which AI model actually handles it.

This layer maps each complexity tier to a primary model and a secondary fallback model. For instance, simple queries route to budget models (gpt-4o-mini), while complex requests route to heavyweights (claude-3-5-sonnet).

Add this configuration also:

class ModelConfig(BaseModel):
    provider: str
    model_name: str


class ModelRouter:
    def __init__(self):
        # Map task complexity tiers to primary and fallback models
        self.routing_table = {
            TaskComplexity.SIMPLE: {
                "primary": ModelConfig(
                    provider="openai",
                    model_name="gpt-4o-mini",
                ),
                "fallback": ModelConfig(
                    provider="anthropic",
                    model_name="claude-3-5-haiku-20241022",
                ),
            },
            TaskComplexity.MEDIUM: {
                "primary": ModelConfig(
                    provider="openai",
                    model_name="gpt-4o-mini",
                ),
                "fallback": ModelConfig(
                    provider="anthropic",
                    model_name="claude-3-5-haiku-20241022",
                ),
            },
            TaskComplexity.COMPLEX: {
                "primary": ModelConfig(
                    provider="anthropic",
                    model_name="claude-3-5-sonnet-20241022",
                ),
                "fallback": ModelConfig(
                    provider="openai",
                    model_name="gpt-4o",
                ),
            },
        }

    def get_models_for_tier(
        self, complexity: TaskComplexity
    ) -> tuple[ModelConfig, ModelConfig]:
        """
        Returns the primary and fallback models for a given
        task complexity tier.
        """
        config = self.routing_table[complexity]
        return config["primary"], config["fallback"]

Breaking Down the Code Logic for Tier 2

• ModelConfig Schema: Uses Pydantic to ensure every model definition includes both a provider (for example, "openai") and a specific model_name string.

• self.routing_table Mapping: This dictionary acts as our single source of truth for model assignments:

  • SIMPLE & MEDIUM Tiers: Primary target is gpt-4o-mini for high-throughput, low-cost output. If OpenAI fails, it falls back to Anthropic's claude-3-5-haiku-20241022.

  • COMPLEX Tier: Primary target flips to claude-3-5-sonnet-20241022 for top-tier code generation and reasoning, with gpt-4o as the backup.

• get_models_for_tier: A helper function that takes the analyzed tier and safely returns a tuple of (PrimaryModel, FallbackModel).

Step 3: Implementing Tier3 – Automatic Fallbacks

Even the best AI providers experience downtime, rate limits, or unexpected timeouts. A production-ready app can't just throw an error screen at the user when this happens. We need an execution engine that attempts to call the primary model provider and automatically catches errors. If anything goes wrong, it instantly pivots to the secondary fallback model without breaking the workflow .

Add the execution engine code to the script:

import os
import time

from anthropic import Anthropic, APIError as AnthropicAPIError
from dotenv import load_dotenv
from openai import OpenAI, APIError as OpenAIAPIError

load_dotenv()


class ResilientModelEngine:
    def __init__(self):
        self.openai_client = OpenAI(
            api_key=os.getenv("OPENAI_API_KEY", "dummy")
        )
        self.anthropic_client = Anthropic(
            api_key=os.getenv("ANTHROPIC_API_KEY", "dummy")
        )

    def _call_openai(self, model: str, prompt: str) -> str:
        response = self.openai_client.chat.completions.create(
            model=model,
            messages=[
                {
                    "role": "user",
                    "content": prompt,
                }
            ],
            timeout=10.0,
        )
        return response.choices[0].message.content

    def _call_anthropic(self, model: str, prompt: str) -> str:
        response = self.anthropic_client.messages.create(
            model=model,
            max_tokens=1024,
            messages=[
                {
                    "role": "user",
                    "content": prompt,
                }
            ],
            timeout=10.0,
        )
        return response.content[0].text

    def execute_provider_call(
        self,
        config: ModelConfig,
        prompt: str,
    ) -> str:
        """
        Dispatches prompt execution to the correct provider SDK.
        """
        if config.provider == "openai":
            return self._call_openai(config.model_name, prompt)
        elif config.provider == "anthropic":
            return self._call_anthropic(config.model_name, prompt)
        else:
            raise ValueError(
                f"Unsupported provider: {config.provider}"
            )

    def execute_with_fallback(
        self,
        primary: ModelConfig,
        fallback: ModelConfig,
        prompt: str,
    ) -> tuple[str, str]:
        """
        Attempts execution on the primary model and switches to the
        fallback model if the primary provider fails.

        Returns:
            tuple[str, str]: (Response text, Model used)
        """
        try:
            print(
                f"[Attempt] Calling Primary Provider: "
                f"{primary.provider} ({primary.model_name})"
            )

            result = self.execute_provider_call(primary, prompt)

            return result, (
                f"{primary.provider}:{primary.model_name}"
            )

        except (
            OpenAIAPIError,
            AnthropicAPIError,
            Exception,
        ) as e:
            print(f"[WARNING] Primary call failed due to: {e}")

            print(
                f"[Fallback] Switching to Secondary Provider: "
                f"{fallback.provider} ({fallback.model_name})"
            )

            try:
                result = self.execute_provider_call(
                    fallback,
                    prompt,
                )

                return result, (
                    f"{fallback.provider}:"
                    f"{fallback.model_name} (Fallback)"
                )

            except Exception as fallback_error:
                raise RuntimeError(
                    "Both primary and fallback systems failed. "
                    f"Error: {fallback_error}"
                )

Breaking Down the Code Logic for Tier 3

Provider Clients (_call_openai & _call_anthropic): Helper methods wrap provider SDK calls, establishing a unified strict 10-second timeout. If an API hangs, it aborts fast so the fallback can kick in without making the user wait.

execute_provider_call Dispatcher: Acts as an abstraction bridge, matching the requested provider string to its respective API method.

execute_with_fallback Resiliency Logic: Executes the primary provider first inside a try block. Catches API errors, rate limits, or network timeouts via provider-specific exceptions (OpenAIAPIError, AnthropicAPIError). Logically redirects execution to the fallback provider inside the except block. Only raises an unrecoverable RuntimeError if both primary and fallback providers fail. If your primary provider encounters issues, your console tracks the recovery process transparently:

[Attempt] Calling Primary Provider: anthropic (claude-3-5-sonnet-20241022)

[WARNING] Primary call failed due to: Connection timeout

[Fallback] Switching to Secondary Provider: openai (gpt-4o)

Combining the Architecture into a Unified Execution Pipeline

Now you can combine all three layers into a unified pipeline.

Complete your app.py script with this orchestration class:

class SmartAIEngine:
    def __init__(self):
        self.analyzer = PromptAnalyzer()
        self.router = ModelRouter()
        self.executor = ResilientModelEngine()

    def process_request(self, user_prompt: str) -> dict:
        print("\n==========================================")
        print("Processing New Request")
        print("==========================================")

        # Step 1: Analyze prompt complexity
        complexity = self.analyzer.analyze_complexity(
            user_prompt
        )
        print(
            f"[Step 1] Prompt classified as: "
            f"{complexity.value.upper()}"
        )

        # Step 2: Determine routing target
        primary_model, fallback_model = (
            self.router.get_models_for_tier(
                complexity
            )
        )

        print(
            f"[Step 2] Selected Primary: "
            f"{primary_model.model_name}"
        )

        # Step 3: Execute request with resilient fallbacks
        response_text, executed_model = (
            self.executor.execute_with_fallback(
                primary=primary_model,
                fallback=fallback_model,
                prompt=user_prompt,
            )
        )

        return {
            "status": "success",
            "complexity_tier": complexity.value,
            "model_used": executed_model,
            "response": response_text,
        }


# Execution Pipeline Test
if __name__ == "__main__":
    engine = SmartAIEngine()

    # Query 1: Simple task
    simple_query = (
        "What is the capital of Japan? "
        "Answer in one word."
    )

    result_1 = engine.process_request(
        simple_query
    )

    print(f"Model Used: {result_1['model_used']}")
    print(f"Response: {result_1['response']}")

    # Query 2: Complex task
    complex_query = (
        "Write a Python function to debug a "
        "memory leak in a multithreaded "
        "application."
    )

    result_2 = engine.process_request(
        complex_query
    )

    print(f"Model Used: {result_2['model_used']}")
    print(
        f"Response Snippet: "
        f"{result_2['response'][:100]}..."
    )

Breaking Down the Code Logic

• Unified Orchestration (SmartAIEngine): Initializes all three modular components—PromptAnalyzer, ModelRouter, and ResilientModelEngine—as instance properties.

• The Pipeline Steps:

  • Analyze: Evaluates the prompt string offline to get the complexity tier.

  • Route: Resolves primary and secondary model pairs based on that tier.

  • Execute: Calls the models resiliently and catches failure scenarios.

• Normalized Response Payload: Wraps execution details into a consistent output dictionary, keeping track of model usage, complexity categorization, and output text.

Lessons Learnt from Dynamic Model Switching in Production

Building a dynamic AI routing system taught our team critical lessons about enterprise LLM architectures:

First, keep classification light. Never use a large LLM call to classify prompts for small tasks. Use regex, keyword matching, and token-length rules. Your classifier should run in under 5 milliseconds.

Second, normalize system outputs. Different model providers structure outputs differently. Make sure your application wraps responses in a consistent schema before returning data to the user interface.

Third, set a tight timeout. Provider APIs often hang instead of throwing immediate errors. Set tight request timeouts (5 to 10 seconds) on your primary model calls so your fallback triggers quickly without frustrating the end user.

And finally, track usage metrics. Log every routing decision, model fallback, and cost delta. This data will reveal whether your complexity thresholds are properly tuned over time.

Conclusion

As AI applications scale, relying on a single, monolithic LLM becomes unsustainable. Intelligent model routing allows you to balance performance, latency, and cost without sacrificing response quality.

By decoupling your application from specific model providers and introducing automated routing layers, input evaluation, provider abstraction, and resilient fallbacks, you can build production AI systems that are cost-effective, fast, and resilient.

As you deploy your own applications, treat LLM providers as dynamic utilities. Use lightweight models for everyday processing, reserve flagship models for complex tasks, and handle provider transitions cleanly in code.

Thank You for Reading!

I hope this article has given you a practical understanding of how multi-model orchestrators and dynamic routing work in real-world applications and how you can begin implementing them in your own projects.

If you'd like to discuss AI engineering, Agentic AI, LLMs, RAG, MLOps, enterprise AI architecture, or AI governance, feel free to follow, like, share, and connect with me:

• LinkedIn

• Explore my Github repositories

But these skills have a hidden limitation: they're static. If you download a skill someone else wrote and you want it to behave a little differently, you'll have to copy the whole thing and edit it by hand. And as you may have noticed lately, there are many "skills" forks floating around that are difficult to maintain.

In this tutorial, I'll show you I built a small convention that fixes this. It lets any Agent Skill read a per-project config file, so you can adopt any skill and customize how it behaves by editing a few lines of YAML (without ever touching the skill itself).

You'll build it step by step, test it, and see how to share it so other people can plug into it.

Table of Contents

• What You Will Build

• Prerequisites

• What Are Antigravity Agent Skills?

• Why Static Skills Are a Problem

• The Configurable Skills Solution

• How to Build the Config Loader

• How to Make a Skill Configurable

• How to Add Project Overrides

• How to Test Your Configurable Skill

• Two More Example Skills

• How to Share Your Agent Skills With Others

• Wrapping Up

What You Will Build

You will build a tiny, reusable layer called Configurable Agent Skills. It has three parts:

1. A small Python script, resolve_config.py, that merges a skill's default settings with your project settings and prints the result.

2. A convention: each skill ships 2 files, a config.default.yaml file with its "knobs" and a SKILL.md file. They both guide the agent's behavior.

3. A per-project file, .agent/skills.config.yaml, where anyone using your skill sets their own values.

By the end, you'll have a working git-commit-formatter skill that one team can run in Conventional Commits mode and another team can switch to gitmoji mode, all using the exact same skill files with no forking.

Prerequisites

To follow along, you'll need:

• Google Antigravity installed (the IDE, CLI, or SDK. Any of them work, since skills are just files.).

• Python 3 installed, with PyYAML. You can install PyYAML with python -m pip install pyyaml.

• Basic comfort with the terminal and YAML. You don't need to be an expert in either.

If you've never written an Agent Skill before, the next two sections will bring you up to speed.

What Are Antigravity Agent Skills?

A Skill in Antigravity is a folder that contains a SKILL.md file and, optionally, some scripts, templates, or examples. The SKILL.md file has a short block of YAML "frontmatter" at the top (a name and a description), followed by a set of instructions written in plain Markdown.

Here's the important part: skills are loaded on demand. The agent reads only the short description of each skill at first. When your request matches that description, the agent pulls in the full instructions and follows them. This keeps the agent's context small and focused.

A minimal skill that enforces Conventional Commits looks like this:

---
name: git-commit-formatter
description: Formats git commit messages using the Conventional Commits specification. Use this when the user asks to commit changes or write a commit message.
---

# Git Commit Formatter

When writing a commit message, follow the Conventional Commits format:
`type(scope): description`

Allowed types: feat, fix, docs, style, refactor, perf, test, chore.

Drop that in your skills folder, ask the agent to "commit these changes," and it will write a properly formatted message. Simple and useful, right?

Why Static Skills Are a Problem

Now look closely at that skill. The allowed types (feat, fix, docs, and so on) are baked directly into the instructions.

That's fine until someone wants something slightly different. Maybe your team also uses a ci type. Maybe you prefer gitmoji, where each commit starts with an emoji. Maybe you want to require a scope on every commit.

With a static skill, there's only one way to get any of that: copy the whole skill and edit the Markdown. When you do this across a team, everyone ends up with their own private fork. When the original author ships an improvement, none of the forks get it. The skill stops being something you share and becomes something everyone rewrites.

The core issue is that there's no clean line between the skill's logic (which everyone should share) and its settings (which each project wants to control). How do we solve this?

The Configurable Skills Solution

The idea is simple. Instead of hard-coding settings in the instructions, the skill will:

1. Ship its settings and their defaults in a separate config.default.yaml file.

2. Read a merged config (defaults plus any project-level overrides) before it acts.

The project-level overrides live in a file called .agent/skills.config.yaml, which sits at the root of the user's project:

# .agent/skills.config.yaml 
# (edit this file in your project instead of the skill globally)
git-commit-formatter:
  style: gitmoji
  extra_types: [ci, build]
  scope_required: true

That's the easy flow. Drop the skill in, set a few keys, and you're done. The skill's own files never change.

To make this work, you need a script that reads both files, merges them, and hands the result to the agent. Let's build it.

How to Build the Config Loader

Create a file called resolve_config.py. Its job is to take a skill's name, load that skill's config.default.yaml, find the user's .agent/skills.config.yaml, and merge the two so that user values win.

Start with a deep-merge helper. This is the heart of the loader:

def deep_merge(base, override):
    """Recursively merge override onto base.

    Dicts merge key by key. Anything else (scalars, lists) is replaced
    wholesale by the override value.
    """
    if isinstance(base, dict) and isinstance(override, dict):
        merged = dict(base)
        for key, value in override.items():
            merged[key] = deep_merge(merged[key], value) if key in merged else value
        return merged
    return override

Notice the deliberate choice here: dictionaries merge key by key, but lists are replaced, not appended. That keeps the behavior predictable. If you want to handle "defaults plus extras", use the explicit extra_types key in the skill as you'll see in the example below.

Next, you need to find your "per-project" config. The loader walks up from the current directory looking for an .agent/skills.config.yaml file:

from pathlib import Path

def find_project_config(start: Path):
    """Walk upward from start looking for .agent/skills.config.yaml."""
    start = start.resolve()
    for folder in [start, *start.parents]:
        candidate = folder / ".agent" / "skills.config.yaml"
        if candidate.is_file():
            return candidate
    return None

Now put it together. The loader locates the skill's defaults (which sit next to the script), loads your overrides for that skill's name, merges them, and prints the result:

import sys, yaml
from pathlib import Path

def resolve(skill_name, skill_dir, project_root):
    defaults = yaml.safe_load((Path(skill_dir) / "config.default.yaml").read_text()) or {}

    user_path = find_project_config(Path(project_root))
    user_all = yaml.safe_load(user_path.read_text()) if user_path else {}
    user_cfg = (user_all or {}).get(skill_name, {}) or {}

    return deep_merge(defaults, user_cfg)

That completes the whole idea. The full version in the sample repo adds a command-line interface, JSON output, and clear error messages, but the logic above is all you really need.

How to Make a Skill Configurable

Now you'll convert the static commit skill into a configurable one. This takes two files.

First, create config.default.yaml next to the skill. It lists every setting and a safe default, so the skill works even when the user has no config at all:

# Default configuration for the git-commit-formatter skill.
style: conventional          # conventional | gitmoji
types:                       # base set of allowed commit types
  - feat
  - fix
  - docs
  - style
  - refactor
  - perf
  - test
  - chore
extra_types: []              # additional types, merged on top of `types`
scope_required: false        # if true, require a scope: type(scope): ...
max_subject_length: 72       # hard cap on the subject line

Second, update SKILL.md so that its very first instruction is to resolve the config and apply it. This is the key move: you're telling the agent to read the settings before it does anything else:

---
name: git-commit-formatter
description: Formats git commit messages to a team's chosen convention (Conventional Commits or gitmoji). Use this when the user asks to commit changes or write a commit message. Reads per-project settings so teams customize commit style without editing this skill.
---

# Git Commit Formatter (Configurable)

## Step 1 - Resolve configuration (always do this first)

Run the loader and read its output:

`python scripts/resolve_config.py git-commit-formatter --project-root .`

Apply exactly those settings:

- `style`: `conventional` or `gitmoji`.
- `types` + `extra_types`: the full set of allowed commit types.
- `scope_required`: if true, a scope is mandatory.
- `max_subject_length`: hard cap on the subject line.

## Step 2 - Compose the message

Pick the primary type from `types` + `extra_types`, build the subject in the
chosen `style`, and enforce `scope_required` and `max_subject_length`.

This pattern ("make the agent run a script and obey its output") is the same one Antigravity's own validation skills use. It keeps the behavior deterministic instead of leaving it to the model's memory.

Notice how extra_types solves the additive-list question. The default list stays put, and the user's extras are simply added on top by the skill. No fork is required to add a ci type.

How to Add Project Overrides

Let's say you want gitmoji commits with two extra types. Create a single file in your project:

# .agent/skills.config.yaml
git-commit-formatter:
  style: gitmoji
  extra_types: [ci, build]
  scope_required: true

You just changed three lines of config and didn't open the skill or fork any code. The next time the agent commits, it will use this project settings.

And a different project, with no config file at all, keeps getting the sensible Conventional Commits defaults. You have one skill with many behaviors.

How to Test Your Configurable Skill

You don't need the agent to check that the merge works. Run the loader directly and read the output.

With no overrides, you get the defaults:

$ python scripts/resolve_config.py git-commit-formatter --project-root .
style: conventional
scope_required: false
...

Now add the .agent/skills.config.yaml override from the last section and run it again:

$ python scripts/resolve_config.py git-commit-formatter --project-root . --print-sources
style: gitmoji
scope_required: true
extra_types:
- ci
- build
types:
- feat
- fix
- docs
...

The style flipped to gitmoji, scope_required became true, and your extra types appeared (while the base types list stayed intact). That confirms the merge does exactly what you want.

It's worth writing a small automated test too, so a future change to the loader can't silently break the merge. A test can create a fake skill and a fake project config in a temp folder, run the loader, and assert that user values override defaults while untouched defaults survive.

Two More Example Skills

The same pattern works for any skill. Here are two more to show the range.

A Changelog Generator

Its config.default.yaml exposes the output format (like Keep a Changelog), which commit types to include, and whether to link commit hashes to a repo URL. One project can generate a formal changelog grouped by type, while another can generate a simple bulleted list. It's the same skill with a different config.

# changelog-generator config.default.yaml (excerpt)
format: keepachangelog       # keepachangelog | conventional | simple
include_types: [feat, fix, perf]
include_authors: false
repo_url: ""                 # if set, hashes link to commits

A License-Header Adder

Its config exposes the license (Apache-2.0, MIT, or custom), the holder, and a map of file extensions to comment styles. A company sets the holder once in their project config, and every new file gets the right header in the right comment style, without editing the skill.

# license-header-adder config.default.yaml (excerpt)
license: apache-2.0          # apache-2.0 | mit | custom
holder: "Your Name or Org"
year: auto                   # auto = current year

The lesson is that almost any skill has a few decisions baked into it. When you pull those decisions into a config.default.yaml, you convert a one-off skill into a tool that anyone can reuse and tune.

How to Share Your Agent Skills With Others

Once your agent skills follow the convention, they compose into something bigger. To make your agent skills easy for others to adopt, you have to:

• Keep each skill self-contained: Vendor a copy of resolve_config.py inside each skill's scripts/ folder, so someone can copy a single skill folder anywhere and it just works.

• Document every config key in the SKILL.md, so users know exactly what they can tune.

• Publish a small index: A simple index.json that lists each skill's name, path, and config keys makes it easy for others to discover what you've built and contribute their own.

Because the convention is just "read a config file first," anyone can publish a compatible skill. Each new configurable skill makes the whole ecosystem more useful. In addition to shipping a skill, you're shipping a small standard that other people can build on.

Wrapping Up

You started with a static skill whose behavior was frozen in Markdown, and you turned it into a configurable one that anyone can tune from a single project file.

The entire setup is relatively small. It has the merge function, one convention, and a config.default.yaml per skill.

It also changes how skills are shared. Instead of forking a skill to change one setting, you can keep the shared logic and adjust your own config. Improvements to the skill flow to everyone, and everyone still gets the behavior they want.

If you want to try it, build the git-commit-formatter skill from this tutorial, drop it into your Antigravity skills folder, and add an .agent/skills.config.yaml to a project. Then flip style from conventional to gitmoji and watch the same skill behave differently.

From there, make one of your own skills configurable. Find the settings you baked into the instructions, move them into a config.default.yaml, and let your users take it from there.

The full sample code (the loader, its tests, and all three example skills) is on GitHub at github.com/keepdeploying/configurable-agent-skills.

Thanks for reading. If you build a configurable skill of your own, share it. Let's keep the ecosystem growing.