Optimizing data center operation — Part 1
Introduction
Data centers play a critical role in the infrastructure of any modern cloud business. As they grow in complexity, optimizing operations becomes a key concern to ensure efficiency, reduce costs, and manage resources. In this article, we’ll explore a method for optimizing data center operations using Linear Programming, and provide a step-by-step guide on how we can implement similar strategies.
For readers who are encountering this series for the first time, my purpose is to share my passion for mathematics to inspire people of all backgrounds. My code will be published on my GitHub.
✨Please give me a 👏 if you are interested in my sharing, leave a comment 💬 to share your views/thoughts. ✨
Solver Introduction
Optimization problems are central to many fields, from economics and operations research to engineering and machine learning. These problems involve finding the best solution from a set of feasible options, typically aiming to minimize or maximize an objective function, such as cost or profit. The challenge is that real-world problems often come with a variety of constraints, making them complex to solve manually. This is where optimization solvers come in.
Solvers are specialized software tools designed to find optimal or near-optimal solutions to mathematical problems efficiently. They use algorithms such as linear programming (LP), mixed-integer programming (MIP), and constraint programming (CP) to tackle different types of optimization problems. Some popular solvers include CP solvers, CPLEX, Gurobi, Pymoo, Z3, or custom solvers, each offering a variety of capabilities to address problems ranging from small-scale linear problems to highly complex, large-scale optimization issues.
✨ I have a series on solving Jane Street’s Puzzles using Z3 optimisers. I hope the materials in this series will help us practice using the solver 🎁.✨
In this article, we use Gurobi to solve the optimisation. Gurobi is one of the most powerful and widely-used solvers for solving mathematical optimization problems. It is highly efficient for linear programming (LP), mixed-integer programming (MIP), quadratic programming (QP), and various other optimization models. For more information about Gurobi, please refer to https://www.gurobi.com/
Note: The optimization problem presented below requires substantial computational resources to run efficiently. To ensure you have access to the necessary capabilities, I recommend downloading the Gurobi Academic WLS License if you’re affiliated with an academic institution. Gurobi provides this license for free to academic users, offering full access to their high-performance optimization solver.
However, you can use any solver to solve this problem once you understand the strategies we’ve outlined.
Problem Overview
Now, back in our problem. Imagine you are managing a network of data centers (DCs). These data centers host servers of various generations (SG), each with a certain lifespan. Each data center can contain two types of servers: CPU servers and GPU servers. Your job is to maximize profit while deciding when to buy new ones, when to move or remove them between data centers, and when to dismiss servers that have reached the end of their useful life. All of these actions must be balanced with the goal of meeting demand while respecting physical space constraints in the data centers.
In this article, we will model this decision-making problem using mathematical optimization. The first part of the article explores the dataset, which includes demand at each time step, data center capacity, selling prices for different server generations, and server specifications.
The second part of the article introduces the three components of a mathematical optimization model: the objective function, the decision variables, and the constraints. In our data center operation problem, the objective is to maximize overall profit. The decision variables represent the actions (buy, move, hold, dismiss) applied to a certain number of servers. The constraints ensure basic requirements are met, such as conducting four actions respecting the capacity limits of each data center. By determining the optimal actions at each of the 168 time steps, the model will recommend how many servers of each type should be deployed in each data center to maximize profit.
Let’s get started.
Data Exploration
In order to effectively solve the challenges of data center operations, it’s essential to have a clear understanding of the data involved. This section provides an overview of the critical data points that will be analyzed. The data includes market demand for renting servers, as well as the resources available to the business, such as the number of servers and the capacity of data centers. By exploring these data sets, we can better understand the operational landscape and make informed decisions for optimizing resource usage.
1. Data center information
datacenters = pd.read_csv('https://github.com/Tayerquach/data_center_operations/blob/master/data/datacenters.csv?raw=true')
datacenters
Insights:
- Each server will have a specific latency sensitivity (high, medium, low) depending on the data center in which it is placed.
- Datacenters exhibit varying levels of latency sensitivity. A higher latency sensitivity typically correlates with higher energy costs.
- Datacenters with lower latency tend to have a higher slot capacity.
2. Server Information
servers = pd.read_csv('https://github.com/Tayerquach/data_center_operations/blob/master/data/servers.csv?raw=true')
serversIn order to get insights from servers dataframe, we will visualise its attributes as follows.
2. 1. Purchase Price and Energy Consumption by Server Generation
Purchase Price by Server Generation
- GPU servers are significantly more expensive than CPU servers, with the price increasing for each generation of GPU and CPU servers.
- CPU.S4 is the most expensive among the CPU servers, while GPU.S3 is the highest-priced GPU server.
Energy Consumption by Server Generation
- GPU servers consume significantly more energy than CPU servers
- GPU servers consume far more energy, particularly GPU.S3, which consumes the most energy among all server types.
3. Demand Information
Insights:
- Shift to Newer Generations: Across all latency levels, demand moves from servers (S1, S2) to newer ones (S3, S4) over time. Servers S1 and S2 can be considered older generations, while S3 and S4 represent the newest technologies. This transition reflects the growing need for more advanced server capabilities as operational demands increase.
- Increasing GPU Demand: Demand for newer GPU servers (especially S3) grows steadily, highlighting the rising importance of GPUs in modern workloads.
- Decrease in Demand Across Latency Sensitivity Levels: The demand gradually decreases from low latency sensitivity to high latency sensitivity over time. This could be because of the higher selling price for the servers with higher latency sensitivity.
Problem Formulation
Let’s define our notations for the Mathematical Optimisation model.
First, let’s define the following sets:
Sets
- DC: Set of data center IDs
- T: Set of timesteps
- LS: Latency sensitivity types
- SG: Server generations
- Lifespan: Set of life ages for each servers (from 1 to life expectancy).
Parameters
Decision Variables
First, we define four actions as our decision variables: the number of servers bought, moved, removed, and dismissed. The ‘Hold’ action is automatically applied if no dismissal action is taken for a given server.
Another variable is the inventory of servers at each datacenter and is updated at the end of the time-step after all actions are applied.
With these initial decision variables, we can start the Mathematical Operation model.
First of all, initialise the model
model = gp.Model("Data_center_operation", env = env)
model.setParam('Seed', seed)# Decision variables
# Buy action
buy = model.addVars(time_steps, server_types, datacenters_id, lb = 0, vtype=GRB.CONTINUOUS, name="buy") # Number of servers bought
# Move action
move_to = model.addVars(time_steps, server_types, datacenters_id, lifespan, lb = 0, vtype=GRB.CONTINUOUS, name="move_to") # Number of servers moved
remove_from = model.addVars(time_steps, server_types, datacenters_id, lifespan, lb = 0, vtype=GRB.CONTINUOUS, name="remove_from") # Number of servers moved
#Dismissing servers
dismiss = model.addVars(time_steps, server_types, datacenters_id, lifespan, lb = 0, vtype=GRB.CONTINUOUS, name="dismiss") # Number of servers moved
# Decision variables to track server lifespan in inventory
I = model.addVars(range(start_ts, end_ts + 1), server_types, datacenters_id, lifespan, lb = 0, vtype=GRB.CONTINUOUS, name="inventory") # Inventory begins before time step = 1Constraints
- Setting Up Initial Inventory:
At the beginning of the planning period, no servers are in place. The initial inventory for each server generation, data center, and lifespan stage is set to zero. This is represented mathematically as:
2. Buying servers
At each time step, you may need to buy new servers. When you buy a server, it starts with a “life” value of 1, meaning it is newly deployed. However, servers can only be bought when their generation is released. This is controlled by a condition that checks whether the current time step matches the release time of a specific server generation.
3. Dismiss servers
The term “Dismiss” can be understood in two ways:
- Dismiss (lifespan > life expectancy): This refers to decommissioning a server once it exceeds its expected lifespan.
- Dismiss (stop using at a time period): This refers to halting the use of a server during a specific time period, even if it hasn’t reached the end of its life expectancy.
Dismissal occurs first in the decision process, and only servers that are already in the inventory from a previous time step can be dismissed.
For servers in their first life stage, dismiss action is not allowed.
4. Move action
To effectively solve a “move” action, it can be broken down into two sub-actions to efficiently handle the logistics of moving resources.
- Move to: This refers to relocating a certain number of servers from one data center to another.
- Remove to: This refers to removing a certain number of servers from their current data center in preparation for the move.
After dismissal, you can decide to move servers between data centers or remove them altogether. However, you can only remove or move servers that have already existed in the previous time step
Servers in their first life stage cannot be moved or removed. This is because we cannot purchase a server, place it in one data center, and then immediately move it to another. Such a process would not be logical or appropriate.
To ensure no servers are lost during moves, the total number of servers moved into a data center must equal the number moved out at the same time step with same lifespan and server generation:
5. Updating Inventory
Get Boi Mai Quach’s stories in your inbox
Join Medium for free to get updates from this writer.
Once you’ve handled dismissal, moving, and removing servers, you update the inventory for the next time step. The new inventory consists of the servers left from the previous period, plus those moved in, minus those removed and dismissed. This ensures that the server count is updated properly as they age.
6. Slot capacity constraints
Each data center has a limited number of physical slots that can hold servers. You must ensure that the total number of server slots used in each data center doesn’t exceed its capacity. This includes all servers from different generations and lifespans.
7. Meeting Demand with available capacity
We try to meet demand with this available capacity, which is modeled using a variable min_var that represents the minimum of the available capacity and the demand:
Now, let’s look at the code for these constraints.
# Auxiliary variable for min(I, demand)
min_var = model.addVars(time_steps, server_types, latency_sensitivity, vtype=GRB.CONTINUOUS,lb = 0, name="min_var")
Zf_var = model.addVars(time_steps, server_types, latency_sensitivity, vtype=GRB.CONTINUOUS,lb = 0, name="Zf_var")- Setting Up Initial Inventory
# Initialise inventory
for sg in server_types:
for dc in datacenters_id:
for life in lifespan:
model.addConstr(I[start_ts, sg, dc, life] == 0,
name=f"set_up_initial_inventory_{start_ts}_{sg}_{dc}_life_{life}")2. All actions and slot constraints
for ts in time_steps:
# 1. Buying constraint respecting the release window
for sg in server_types:
for dc in datacenters_id:
# Only allow purchases during the server's release time window
if (ts >= release_time[sg][0]) and (ts <= release_time[sg][1]):
model.addConstr(I[ts, sg, dc, 1] == buy[ts, sg, dc], name=f"buy_constraint_{ts}_{sg}_{dc}")
else:
# Prevent buying servers outside the release window
model.addConstr(buy[ts, sg, dc] == 0, name=f"no_buy_{ts}_{sg}_{dc}")
model.addConstr(I[ts, sg, dc, 1] == 0, name=f"no_buy_{ts}_{sg}_{dc}")
# 2. Dismiss servers constraint
for sg in server_types:
for dc in datacenters_id:
for life in range(2, life_expectancy + 1):
model.addConstr(dismiss[ts, sg, dc, life] <= I[ts - 1, sg, dc, life - 1], name=f"dismiss_constraint_{ts}_{sg}_{dc}_life_{life}")
# No dismiss servers right after buying that servers
model.addConstr(dismiss[ts, sg, dc, 1] == 0, name=f"no_dismiss_at_beginning_{start_ts}_{sg}_{dc}")
# 3.1 Remove and move constraint: available resources
for sg in server_types:
for dc in datacenters_id:
for life in range(2, life_expectancy + 1):
model.addConstr(remove_from[ts, sg, dc, life] <= I[ts - 1, sg, dc, life - 1] - dismiss[ts, sg, dc, life], # Dismiss before moving
name = f"remove_constraint_availabel_resource_{ts - 1}_{sg}_{dc}_{life}")
# No remove or move servers right after buying that servers
model.addConstr(remove_from[ts, sg, dc, 1] == 0, name=f"no_remove_at_beginning_{start_ts}_{sg}_{dc}")
model.addConstr(move_to[ts, sg, dc, 1] == 0, name=f"no_move_at_beginning_{start_ts}_{sg}_{dc}")
# 3.2 Balance move_to and remove_from
for sg in server_types:
for life in range(2, life_expectancy + 1):
model.addConstr(gp.quicksum(move_to[ts, sg, dc, life] for dc in datacenters_id) == gp.quicksum(remove_from[ts, sg, dc, life] for dc in datacenters_id),
name=f"balance_move_to_remove_from_{ts}_{sg}_life_{life}")
# 4. Update inventory after moving servers
for sg in server_types:
for dc in datacenters_id:
for life in range(2, life_expectancy + 1):
model.addConstr(I[ts, sg, dc, life] == I[ts - 1, sg, dc, life - 1] + move_to[ts, sg, dc, life] - remove_from[ts, sg, dc, life] - dismiss[ts, sg, dc, life],
name=f"update_inventory_after_all_actions_{ts}_{sg}_{dc}_life_{life}")
# 5. Constraint: Slot capacity of each datacenter
for dc in datacenters_id:
model.addConstr(
gp.quicksum(I[ts, sg, dc, life] * server_slots[sg] for sg in server_types for life in lifespan) <= datacenter_slots[dc],
name=f"slot_capacity_{ts}_{dc}"
)
# 6. Add constraint to enforce min_var[ts, sg, ls] = min(I[ts, sg, ls], demand[ts, sg, ls])
for sg in server_types:
for ls in latency_sensitivity:
# Total inventory across all lifespans
total_inventory = gp.quicksum(I[ts, sg, dc, life] for dc in datacenters_id if datacenter_latency_map[dc] == ls for life in lifespan)
capacity = total_inventory * server_capacity[sg]
model.addConstr(Zf_var[ts, sg, ls] == adjust_capacity_by_failure_rate(capacity), name=f"custom_capacity_{ts}_{sg}_{ls}")
model.addConstr(min_var[ts, sg, ls] <= Zf_var.get((ts, sg, ls), 0), name=f"min_capacity_{ts}_{sg}_{ls}")
model.addConstr(min_var[ts, sg, ls] <= demand.get((ts, sg, ls), 0), name=f"min_demand_{ts}_{sg}_{ls}")Objective
The goal of this optimization problem is to maximize the total profit over 168 time steps. The profit at each time step is calculated by taking the revenue generated from renting servers and subtracting the total costs, which include buying costs, energy consumption, maintenance, and moving costs. Below, we break down the key components involved in this calculation.
Objective function
The primary objective is to maximize the total profit over all time steps:
Revenue at Time Step ts
The revenue at a given time step is calculated as the sum of the rented servers across different server generations and latency sensitivities, multiplied by their respective renting prices:
Buying Cost at Time Step ts
The buying cost reflects the cost of purchasing new servers at each time step. It is calculated by summing the purchase cost for each server generation across all data centers:
Energy Cost at Time Step ts
The energy cost reflects the power consumption of servers based on their generation and data center location. It is calculated by summing the energy prices for each server generation at each data center, multiplied by the number of active servers at their respective life stages.
Maintenance Cost at Time Step ts
Maintenance costs depend on the server’s generation and lifespan. First we have a function to calculate the maintenance cost as follows
def get_maintenance_cost(b, x=1, xhat=96):
# CALCULATE THE CURRENT MAINTENANCE COST
return int(b * (1 + (((1.5)*(x))/xhat * np.log2(((1.5)*(x))/xhat))))The cost is derived by applying the maintenance cost function for each server and multiplying it by the number of servers in use at that time step.
Moving Cost at Time Step ts
The moving cost occurs when servers are relocated between data centers. It is calculated by multiplying a fixed moving cost by the number of servers moved during the time step:
Total Cost at Time Step ts
The total cost at each time step is the sum of all incurred costs, including buying, energy, maintenance, and moving costs:
Profit at Time Step ts
The profit for each time step is calculated by subtracting the total cost from the revenue at that time step:
Now, let’s look at the code for the objective.
for ts in time_steps:
# Revenue at time step ts
revenue_ts = gp.quicksum(
min_var[ts, sg, ls] * renting_prices.get((sg, ls), 0)
for sg in server_types for ls in latency_sensitivity
)
R_recorded[ts] = revenue_ts
# Cost at time step ts
buy_cost_ts = gp.quicksum(purchase_cost[sg] * buy[ts, sg, dc] for sg in server_types for dc in datacenters_id)
energy_cost_ts = gp.quicksum(energy_prices[sg, dc] * gp.quicksum(I[ts, sg, dc, life] for life in lifespan) for sg in server_types for dc in datacenters_id)
# Maintenance cost at time step ts:
maintenance_cost_ts = gp.quicksum(
get_maintenance_cost(maintain_cost[sg], life) * I[ts, sg, dc, life]
for sg in server_types for dc in datacenters_id for life in lifespan
)
moving_cost_ts = moving_cost * gp.quicksum(move_to[ts, sg, dc, life] for life in range(2, life_expectancy + 1) for sg in server_types for dc in datacenters_id)
# Total cost at time step ts
total_cost_ts = buy_cost_ts + energy_cost_ts + maintenance_cost_ts + moving_cost_ts
C_recorded[ts] = total_cost_ts
# Profit at time step t
profit_ts = revenue_ts - total_cost_ts
P_recorded[ts] = profit_ts
# Accumulate the profit
total_profit += profit_tsmodel.setObjective(total_profit, GRB.MAXIMIZE)
model.update()We can configure certain parameters in Gurobi to improve speed and performance.
# Gurobi optimization settings to improve speed and performance
opt = {
"Presolve": 2, # Aggressive presolve
"PreSparsify": 2, # Remove redundant constraints
"Aggregate": 2, # Aggressive aggregation
"OutputFlag": 1, # Enable output to track progress
}
# Set model parameters
model.setParam('Presolve', opt["Presolve"])
model.setParam('PreSparsify', opt["PreSparsify"])
model.setParam('Aggregate', opt["Aggregate"])
model.setParam('OutputFlag', opt["OutputFlag"])We can now tell Gurobi that the model is complete and it can solve the problem.
# Optimize model
model.optimize()
rich.print(f"Profit: {total_profit.getValue():0,.2f}")Output:
Note: The running time depends on the machine you use to run the optimisation.
Post Processing
After completing the optimisation, the next step is to analyse and interpret the results. In this phase, we will save key outputs, including all actions, server utilization, and business metrics such as revenue, cost, and profit. This data provides crucial insights into how the system performed during the optimisation process.
Once the data is saved, we will visualise both the Utilization of server resources and the Business Metrics (revenue, cost, and profit). These visualisations will allow us to better understand the efficiency of the optimization and assess its impact on the business.
Now, running code to generate necessary dataframes.
action_dict = {
'server_generation': [],
'datacenter_id': [],
'time_step': [],
'lifespan': [],
'buying_servers': [],
'remove': [],
'move_to': [],
'dismiss': []
}
utilisation_dict = {
'server_generation': [],
'latency_sensitivity': [],
'time_step': [],
'capacity': [],
'demand': [],
}
business_metric_dict = {
'time_step': [],
'revenue': [],
'cost': [],
'profit': []
}
if model.status == GRB.OPTIMAL:
for ts in tqdm(time_steps):
business_metric_dict['time_step'].append(ts)
revenue_at_ts = float(R_recorded[ts].getValue())
cost_at_ts = float(C_recorded[ts].getValue())
profit_at_ts = float(P_recorded[ts].getValue())
business_metric_dict['revenue'].append(revenue_at_ts)
business_metric_dict['cost'].append(cost_at_ts)
business_metric_dict['profit'].append(profit_at_ts)
# Buying servers
for sg in server_types:
for dc in datacenters_id:
bought_qty = buy[ts, sg, dc].x
action_dict['server_generation'].append(sg)
action_dict['datacenter_id'].append(dc)
action_dict['time_step'].append(ts)
action_dict['lifespan'].append(1)
action_dict['buying_servers'].append(bought_qty)
action_dict['remove'].append(0)
action_dict['move_to'].append(0)
action_dict['dismiss'].append(0)
for ls in latency_sensitivity:
capacity = Zf_var[ts, sg, ls].x
demand_ts = demand.get((ts, sg, ls), 0)
utilisation_dict['server_generation'].append(sg)
utilisation_dict['latency_sensitivity'].append(ls)
utilisation_dict['time_step'].append(ts)
utilisation_dict['capacity'].append(capacity)
utilisation_dict['demand'].append(demand_ts)
# Moving and dismissing servers
for sg in server_types:
for dc in datacenters_id:
for life in range(2, life_expectancy + 1):
remove_qty = remove_from[ts, sg, dc, life].x
moveto_qty = move_to[ts, sg, dc, life].x
dismiss_qty = dismiss[ts, sg, dc, life].x
action_dict['server_generation'].append(sg)
action_dict['datacenter_id'].append(dc)
action_dict['time_step'].append(ts)
action_dict['lifespan'].append(life)
action_dict['buying_servers'].append(0)
action_dict['remove'].append(remove_qty)
action_dict['move_to'].append(moveto_qty)
action_dict['dismiss'].append(dismiss_qty)
action_df = pd.DataFrame(action_dict)
action_df.to_csv(f'result/all_action_df_{seed}.csv', index=False)
utilisation_df = pd.DataFrame(utilisation_dict)
utilisation_df.to_csv(f'result/utilisation_df_{seed}.csv', index=False)
business_metric = pd.DataFrame(business_metric_dict)
business_metric.to_csv(f'result/business_metric_{seed}.csv', index=False)
print("All actions have been saved!")Visualise Utilisation
- Older Generations Struggle with High Demand: CPU.S1 and CPU.S2 show significant under-provisioning in low and medium latency environments, with demand often exceeding capacity
- Better Utilization of Newer Generations: CPU.S3 and CPU.S4 handle demand more efficiently.
- In high latency sensitivity, all server generations effectively meet the demand, which could be due to the fact that servers deployed for high latency workloads are more profitable. As a result, more resources may have been allocated to ensure sufficient capacity, leading to better alignment between demand and capacity across all server generations in this scenario.
- Stable Resource Allocation: Across all latency sensitivities, GPU.S2 and GPU.S3 demonstrate good alignment between capacity and demand, particularly in later time steps, indicating efficient resource allocation.
- GPU.S1 appears to be over capacity compared to market demand in low latency sensitivity. This over-provisioning may result from the accumulation of servers due to moving actions between data centers, shifting from low latency environments to medium or high latency ones, rather than purchasing new servers, which would incur higher costs.
- High Demand for GPU.S3: GPU.S3 handles a significant increase in demand toward the end of the timeline across all sensitivity levels.
Conclusion
CPU: Generally struggles to meet demand for older generations, with more fluctuation between under-provisioning and over-provisioning. Newer generations handle high-demand environments better but still exhibit some inefficiencies in high latency sensitivity.
GPU: Shows better alignment between capacity and demand, particularly for newer generations like GPU.S3. Over-provisioning is observed in GPU.S1, but newer GPUs efficiently meet growing demand, especially in low and medium latency scenarios.
Visualise Business Metric
This chart illustrates a progression toward profitability. Initially, the business experiences volatility in both revenue and costs, leading to some periods of loss. However, as time progresses, revenue grows steadily, while costs stabilize, resulting in increasing and sustained profitability. The overall trend indicates successful business growth.
- Before step 25, there are some periods of loss (negative profit), but these become less frequent over time as revenue grows and costs become more controlled.
- Around time step 100 and 140, there is a sudden decrease in profit, likely due to the dismissal of a certain number of servers that exceeded their life expectancy. This led to a temporary spike in costs, as new servers had to be purchased to meet ongoing demand.
Conclusion
Scenario Analysis
An important requirements in many mathematical optimisation problems is scenario analysis, also known as what-if analysis. Generally speaking, in what-if analysis, we’re interested in understanding how the solution changes under various scenarios. Think about our case here:
- How does the solution change if each server generation has a different life expectancy? In reality, servers have varying life expectancies.
- What if utilization is prioritized over profit?
- What if a new data center is added, or an existing data center is expanded?
- What if we can adjust the selling prices to increase profit?
Our model here is still a simple model. But you can imagine the value that the scenario analysis can provide. It can enable you to answer many questions by creating and comparing different scenarios and evaluating their outcomes, so that you can assess their impacts on the business goals.
Next step
As can be seen, the utilisation of each server generation at various latency sensitivity levels has room for improvement. One potential way to enhance this is by optimising resource allocation and operational strategies.
In the next step, we’ll tackle this data center operation problem by introducing an additional action: a price strategy. By adjusting pricing based on server generation and latency sensitivity, we can potentially improve utilisation and profitability.
For now, try to explore and solve this problem on your own. Think about how different pricing strategies might impact server utilisation and overall business performance.
See you next time, when we’ll dive into a detailed solution and explanation for this enhanced problem.
Source code
You can find the entire notebook with the code developed for this article in the link to my GitHub repository below.
I sincerely hope you found this article useful and entertaining. If so, I’d love to hear your thoughts! And if you’re interested in staying updated on my latest articles, consider following me on Medium. Your support and feedback are what drive me to keep exploring and sharing in this fascinating field. Thank you for taking the time to read, and stay tuned for more insights in my next article!
