A Peek into the Future of Critical Digital Infrastructure
The student team for this project was composed of: Vaidehi Mehra, Gavin Liao, Tiger Li, Rory Landau, Anna Bernsteiner.
The more integrated our critical infrastructure becomes with digital services, the more our digital networks must be able to handle complex physical environments. Networks in a datacenter do not need to care about rain or snow and mobile devices don’t usually care if there is an event drawing in thousands. But in the real world, they must. In preparation for this future, 99P Labs approached us, a team of graduate students at Carnegie Mellon University’s Information Networking Institute, to prototype a testbed that can account for exactly these complexities. In hunting for answers, we explored the landscape of tools to understand the pros and cons of utilizing them and to discover pain points in potential solutions.
Initial Prototype
To hit the ground running, we chose to develop a pipeline to automate testing built on top of Mininet-WiFi, a fork of the Mininet research tool common in wireless networking. We used a pipeline architecture because it allows each stage to operate independently. This means each stage can be updated, maintained, and swapped out easily, for example, changing the backend from Mininet-WiFi to ns-3 or feeding the results into a database instead of visualizing them with Grafana.
With the initial design in place, we began fleshing out user inputs. What would a researcher want to know and how do we represent these quandaries in our testing suite?
Over time, we developed a JSON schema capable of describing a network topology, complete with nodes, links, and the capabilities/constraints of the aforementioned, as well as test in the form of movements and connectivity tests. These tests make up the experiments themselves, able to describe the movements of nodes in a 3D space and test how connectivity changes over time.
Roadblock 1: Movement over Time
This led to our first major roadblock: How do we represent movement over time?
Does movement occur constantly and in parallel to the connectivity tests, or do we pause the simulation to ensure all movements and tests occur within a precise window? The former is more realistic, but loses repeatability and degrades the precision of our timekeeping. Put simply, we cannot guarantee when a test/event starts or completes, or how multiple overlap.
Mininet-WiFi has limited native support for mobility (setting how nodes move over time) as well as a simulation of the progression of time, but does it support the granularity and control we need, or do we need to implement our own handling? Ultimately, we opted for usability, precision, and control, choosing to roll our own.
One of the team members suggested custom handling in the form of timeframes. Each action or test would be assigned a sequence number, and time would only be measured by the number of elapsed actions. This provides total ordering, meaning that there is no ambiguity in when an action occurs; it always occurs after exactly one action and precisely before the start of the next. While disallowing us from using the native mobility options, this solution provides repeatability and applies to any backend, rather than being deeply integrated with the capabilities and limitations of Mininet-WiFi.
However, this solution comes at the cost of realism: nodes do not pause while communicating or take turns moving in a real network. To assuage this, we upgraded the pipeline to support timeframes, sets of actions that occur simultaneously and before the next connectivity test, as a middle ground between simultaneous testing and motion, and perfectly repeatable sequences of actions.
Roadblock 2: Environmental Simulation
With time handled, we moved ahead to environmental factors such as weather, crowding, and obstacles. Questions we ran into early were how can Mininet simulate constraints and how do other people model adverse effects on wireless networks. Another team member explored existing research and found models simple enough to support our prototype. We landed on Log-Normal Shadowing as our underlying model and came away with suggested parameters for a handful of external conditions.
One of our stretch goals was to model line of sight and account for obstacles: radio waves suffer energy loss as they pass through different surfaces, and higher frequency waves may bounce off completely.
We explored integrating OpenStreetMaps, a community-maintained map database, as it would allow us to plot devices on a real map and visualize lines of sight (or the lack thereof) against real structures. However, a feature of this scope could be another practicum project entirely, so we shifted back to modeling using Mininet-WiFi’s propagation models as they have some support for obstacle density in a given environment.
Get R Landau’s stories in your inbox
Join Medium for free to get updates from this writer.
Mininet-WiFi natively supports Friis, LogDistance, and Log-Normal Shadowing, with the latter being the most realistic, able to support path loss for modelling obstructed environments and a randomness component. We then came up with suggested parameters for a variety of external conditions, such as rain, sleet, and crowded environments:
Coupled with suggested noise thresholds (the energy threshold below which radio waves are considered too weak to understand), we were able to draw usable insights from the experiments.
Final Pipeline
By the end of our third milestone, the prototype pipeline had been completed:
Given a single JSON input file, it is capable of generating the described network topology, moving nodes within it, executing connectivity tests at the end of every timeframe, coalescing Mininet-WiFi’s raw output (from its pingall utility and Linux’s iw command) into CSVs, and generating test visualizations in Grafana from these CSVs.
Building a prototypical testing platform for wireless networks keyed us in on some of the pain points that will emerge as 99P and others move forward with their own, more advanced platforms and sandboxes. Specifically, we came to the conclusion that Mininet-WiFi is not an appropriate backend for a sandbox of the desired scope. While feature-rich and easy to get started with, Mininet-WiFi suffers from a critical lack of documentation for its more advanced features and has a handful of common gotchas. The best example of this is a 3-second sleep call before most test commands (e.g., pingall). While a rationale certainly exists, it is nowhere to be found, and the calls cannot be configured externally.
Research Findings
To suggest an alternative to Mininet-WiFi, we spent our fourth and final milestone exploring other simulators. Over those two weeks, the team explored Shadow, ns-3, CORE, and CrowNet.
All four candidates and Mininet-WiFi are powerful simulator/emulator platforms and excel in their respective use-cases. However, we believe ns-3 is the strongest candidate for future wireless sandbox projects. It has native support for distributed computing, which is a must-have to achieve the kind of scale a truly representative test suite would demand. The project is stable and battle-tested, proffering many a research paper. Finally, ns-3 uses true physical and MAC layer emulation, which, while expensive, offers accuracy that abstracted simulators cannot.
Final Takeaways
- We recommend building with a pipeline or microservice architecture to keep the project streamlined and efficient.
- The backend simulator’s status is key: a backend rife with technical debt will make maintenance and extension untenable.
- Regarding handling time: both “Live” and “Artificial” clocks are valid, however Live simulations are more realistic, but less repeatable while Artificial clocks guarantee precision and repeatability, but are a step removed from reality.
- Simulating the effects of weather is difficult and under active research. Ensure whatever mechanism simulates radio signals is easily swappable.
- Line of sight calculations are a mandatory addition going forward, but will likely be the single heaviest lift to integrate.
- Mininet-WiFi is a great tool, however we advise using a more advanced simulator such as ns-3.
Thank you again to Tony Fontana, Ryan Lingo, and the rest of 99P for working with us throughout the project. It was an absolute pleasure.
Thank you to Professor Pedro Bustamante for your invaluable guidance and support. Thank you to Professors Cynthia Kuo and Quinn Jacobson for organizing the practicum projects.
