Your Data Product Standard Is Useless If the Organization Around It Does Not Work
A Data Product Specification Is Not Enough
Open Data Product Specification started from a fairly simple problem. If organisations want to manage data as products, those products need a clear contract. People need to know what a product contains, who provides it, how it is accessed, what quality to expect, what conditions apply, and how the product changes over time. Applications need much of the same information in a structured form. AI agents need even more explicit context because they cannot safely depend on tribal knowledge, undocumented conventions, or a meeting someone had six months ago.
A specification addresses an important part of that problem, but it does not solve the organisational problem around it. Once an organisation starts applying data product standards at scale, the difficult questions quickly move beyond metadata.
- Who decides which products should exist?
- Which business problems justify the investment?
- Who owns the product after the original project ends?
- How are products connected to use cases and objectives?
- How does a consumer find the right one?
- What happens when a product changes?
- How does anyone know whether the promised quality and service levels are being met? And after all of that,
- Did the product create enough value to justify keeping it?
Those questions led us to start building the Data Product Operating Framework around the ODPS standards family. The aim is not to produce another maturity model with a colourful diagram and a long list of things organisations should aspire to do.
The aim is to connect business demand, product definition, governance, delivery, evidence and value through one traceable operating model.
There Are Already Many Good Pieces
There is no shortage of good thinking around data products.
- Data Mesh made product thinking, domain ownership, self-service and federated governance part of the mainstream discussion.
- Thoughtworks has published practical guidance on starting from business use cases and working backwards toward the data products needed to support them.
- Open Data Mesh has developed a machine-readable data product descriptor.
- DCAM provides a serious capability and evidence model for data management.
- COBIT connects enterprise goals to governance practices and controls.
- NIST AI RMF shows how a stable Core can be combined with profiles and implementation guidance.
Data platforms and catalog vendors are also moving toward product-oriented operating models. Many now support ownership, quality, contracts, discovery and some form of product lifecycle management. There is useful work happening across the market, and I do not think another framework adds value simply because it rearranges those ideas into fourteen boxes.
The gap I keep seeing is elsewhere. I have not found another public model that connects the complete chain from a business objective to a use case, from that use case to an investment decision and a machine-readable product contract, then through delivery and consumption to operational evidence, outcomes and eventually another investment decision. Individual parts of that chain exist in several places. Keeping the whole chain intact is where this framework starts to become interesting.
ODPS Defines the Product, the Framework Defines How the Organization Operates It
This distinction became increasingly important while developing ODPS. A product specification tells us what a data product is. An operating framework needs to explain what the organisation does around that product.
The ODPS family already gives us several pieces of the machine-readable foundation. ODPS defines the individual data product contract. ODPC describes portfolio and discovery objects. ODPV provides shared vocabulary. ODPG represents relationships between products, objectives, use cases and other objects. ODPR describes repeatable workflow contracts.
None of those specifications should try to become an enterprise operating model on their own. The framework therefore sits above the specifications.
- DIRECT covers why an organisation should invest and who makes that decision.
- DEFINE covers what the organisation is committing to provide.
- OPERATE covers discovery, access, consumption, change and repeatable work.
- ASSURE covers whether the product performs as promised, stays within required controls and produces enough value to justify continued investment.
That separation also keeps the framework independent of implementation technology. An organisation should be able to apply it whether the underlying environment is Databricks, Snowflake, Microsoft, AWS, Collibra, Informatica, an internal platform, or some combination of them.
ODPS should provide interoperability between those environments rather than become another platform that organisations have to adopt.
Start From Demand, Not From the Data You Already Have
One of the simplest ideas in the framework is also one of the most important. The existence of data is not sufficient justification for creating a data product.
A surprising amount of data product work still begins from supply. A team has a warehouse table, API, file collection or analytics dataset, so the natural next step becomes packaging it and calling it a product. The result often looks tidy in a catalog, but nobody has answered the more difficult question of why the product should exist.
The framework starts from the other direction. An organisation has an objective. That objective creates problems, decisions, obligations or use cases. Those use cases create demand for information.
Only then should the organisation decide whether an existing product already satisfies the demand, whether an existing product should change, or whether creating a new product deserves investment.
This also forces us to keep use cases and products separate.
- A use case explains why data is needed.
- A data product defines a reusable capability that provides data to consumers.
- One use case might require several products, while
- One well-designed product might support dozens of use cases.
If those relationships remain explicit, reuse becomes measurable and portfolio decisions become much easier to defend.
The Most Important Part Is the Traceability Chain
The framework currently has four functions and fourteen capabilities, which makes it easier to explain. I do not think that is the most important part of it.
The more interesting idea is the traceability chain running through the entire operating model.
A business objective connects to a use case. The use case leads to a product decision. That decision leads to a data product and its contract. The product is delivered and consumed. Consumption produces evidence. Evidence helps us evaluate outcomes and value. That value then feeds back into the next investment decision.
If those relationships are machine-readable, questions that are difficult today start becoming normal queries against the operating model.
- A portfolio owner could ask, “Which production data products support Objective 12, have declining adoption, and cost more this year than last year?”
- A governance team could ask, “Which products have declared access restrictions that do not match current entitlements?”
- A product manager could ask, “Which active products have operating costs but no active use cases or measurable outcomes?”
- An engineer could ask, “If I change Product A, which downstream products, applications, workflows, AI agents and business objectives might be affected?”
The same chain works in reverse. After an incident, an organisation could ask, “Which use cases were affected, which KPIs depend on them, and which business objective is now at risk?”
After a portfolio review, leadership could ask, “Which products support several high-value use cases and deserve more investment?”
When duplication appears, someone could ask, “Why are we funding three products that satisfy essentially the same demand?”
Get Dr. Jarkko Moilanen’s stories in your inbox
Join Medium for free to get updates from this writer.
These are not metadata questions. They are management questions that become answerable because the metadata, relationships, contracts and evidence are connected.
Machine-Readable by Default, Human-Readable by Presentation
This leads to one of the principles I think matters most for where ODPS is heading. The source should increasingly be machine-readable by default, while human-readable material becomes a presentation of that source.
Organisations today often maintain several different versions of the same information. A product has one description in a catalog, another in Confluence, another in an architecture diagram, another in an Excel governance register and yet another inside the implementation. They start aligned and then drift. Six months later, nobody knows which version should be trusted.
The ODPS approach is different. The structured product artifact should carry the authoritative definition. A catalog page, website, PDF, product card or AI interface can present that information differently, but they should derive from the same underlying product context whenever practical. Humans still need good documentation. The difference is that the PDF no longer has to be the source of truth.
This becomes much more important when AI agents enter the environment. An agent should not have to scrape a governance PDF, interpret a paragraph describing access conditions and then guess whether the product is suitable for a particular task. The organisation should provide those things explicitly. Meaning, relationships, policies, interfaces and workflow boundaries should become part of the operating environment rather than hidden knowledge.
Declared State and Observed State Should Not Be Confused
One of the framework ideas I find particularly useful is the distinction between declared state and observed state.
Suppose an ODPS contract says that completeness must remain above 98 percent. That is a useful commitment, but it is not evidence that the product currently satisfies it. If the operational measurement shows completeness at 94.7 percent, we now have something much more useful.
The difference between the declared state and the observed state becomes an assurance signal.
The same logic applies across the operating model. The product contract might promise 99.9 percent availability while runtime evidence shows 99.3 percent. Governance might state that only approved consumers have access while the identity system shows an old entitlement still active. A workflow might require human approval before production deployment while the deployment record contains no approval. The authoritative product definition might say that version 2.4 is in production while the runtime still serves version 2.3.
Writing down a requirement does not mean the requirement is being met.
The framework therefore treats the difference between declared and observed state as something the organisation should measure, retain and act upon.
Governance Should Produce Evidence During Normal Work
Governance often becomes an activity performed around delivery rather than through delivery. Policies sit in one system, approvals in another, control evidence in a spreadsheet and product information somewhere else. When an audit arrives, people spend days or weeks reconstructing what happened.
A machine-readable operating model gives us another option.
- A product definition gets validated and produces evidence.
- An access request is approved and produces evidence.
- A workflow executes and produces evidence.
- A quality rule runs and produces evidence.
- An SLA fails and produces evidence.
- A product version changes and produces evidence.
- A control executes and produces evidence.
The point is not to automate everything. Some decisions require human judgement and accountability, and they should remain that way. The point is that important claims should become traceable. If someone says a product complies with a control, the organisation should be able to show the relevant control evidence. If someone says a product supports a strategic objective, the relationship between the product, use case and objective should exist. If someone says a product creates value, there should be some evidence showing the outcome rather than only a slide saying that the project was successful.
Usage Is Useful, but It Is Not the Same as Value
Data product programs often measure what is easiest to count. API calls, downloads, queries, subscribers and catalog views are all useful numbers, but none of them automatically tells us whether the product was worth building.
The framework therefore treats value as a chain rather than a single metric. The product first needs to be discoverable. The intended consumer needs access. The consumer needs to adopt it. Meaningful consumption needs to continue. The associated use case should improve. Only after that should the organisation start making claims about value.
Even then, value does not always mean revenue. A product might reduce processing time, improve a decision, reduce operational risk, satisfy a regulatory obligation or improve a public service. In some cases, the product has direct financial attribution. In others, it contributes alongside several factors. Sometimes it simply enables an activity that would otherwise be impossible, and pretending to calculate an exact ROI would add more fiction than insight.
This distinction becomes important when an organisation manages hundreds of products. At some point, somebody needs to decide which products deserve more investment, which need intervention, which overlap and which should be retired. Without outcome evidence, those decisions depend too much on whoever argues most convincingly in the portfolio meeting.
AI-Agent-First Changes What a Data Product Needs to Expose
ODPS has increasingly moved toward an AI-agent-first direction, but that does not mean every organisation adopting it needs autonomous agents from day one. The framework therefore has one Core and different profiles rather than separate traditional and AI frameworks.
The Enterprise Data Product Profile supports organisations operating through conventional catalogs, APIs, data platforms, governance systems and human-led processes.
The AI-Agent-First Profile strengthens the same capabilities when agents directly discover, interpret, select, combine and act on data products.
This is an important distinction because making a YAML file readable by an LLM does not make an organisation agent-ready. An agent working with enterprise data needs explicit context. It needs to know what concepts mean, which version is current, which product supports a use case, which access rules apply, what policies govern the action, what tools it is allowed to call, where it must stop, when a human needs to approve something and what evidence should remain after the action.
That pushes semantics, relationships, permissions, provenance, workflows and evidence into the core operating environment. Agent-readable data products are useful, but agent-governed data product operations are a much bigger step. The framework is intended to support that progression without forcing every organisation to start there.
This Does Not Require a Data Mesh Organisation
I also want the framework to avoid becoming tied to one organisational doctrine. A Data Mesh organisation should be able to use it. So should a centrally governed bank. A government with dozens of agencies should be able to use it, as should a company with one enterprise data platform.
An organisation with strong domain ownership might implement the capabilities differently from one with central product teams, but both still need to answer the same questions about demand, ownership, contracts, lifecycle, evidence and value.
ODPS adoption should not require an organisation to restructure itself first. The standards should help different organisational models exchange and manage data products consistently, rather than prescribing one model as the price of entry.
Start With One Complete Chain
There is a temptation with frameworks to begin by cataloging the whole organisation. Hundreds of datasets are inventoried, maturity assessments are run, governance boards are created and roadmaps are produced. A year later, the organisation has generated a lot of material but has still not demonstrated one complete operating loop. I want the ODPS framework to encourage the opposite approach.
Start with one real objective and one real use case. Make one explicit product decision. Define one ODPS product contract. Add one catalog reference. Connect the objective, use case and product in the graph. Run one operational workflow. Collect one service or quality observation. Measure one adoption signal and one outcome. Then take that evidence back into one portfolio review.
At that point, test the chain. Can you travel from the business objective all the way down to the running product? Can you travel from operational evidence back to the investment decision? If you can, you have something worth scaling. If you cannot, another hundred data product definitions will not fix the missing operating model.
Why This Changes the ODPS Adoption Conversation
A specification alone asks an organisation to standardise product metadata. That has value, but it is a difficult transformation story to take to senior management.
An operating framework changes the conversation.
- ODPS becomes the product contract.
- ODPC provides the portfolio and discovery layer.
- ODPV gives the shared meaning.
- ODPG preserves the relationships.
- ODPR describes repeatable work.
Operational systems execute that work and produce evidence. The framework explains how the pieces connect from business demand to measurable outcomes.
For an organisation considering ODPS, that is a much larger proposition than adopting another metadata format. The standards become part of an operating model that connects strategy, product management, governance, engineering, AI and value.
That is the direction I think data product management needs to move toward. The missing part has rarely been another catalog screen or another data-product definition template. The harder problem is continuity between the business decision that started the work, the product that eventually runs in production and the evidence showing whether the investment achieved anything useful.
All above is packaged — version 0.2.0
It now includes the framework Core, all four capability references, assessment standard, implementation playbook, glossary, Enterprise and AI-Agent-First profiles, positioning research, the full LinkedIn article with the example questions, Codex implementation prompt, versioning policy, crosswalk scaffold, and both visuals. Contributions in all forms are welcome. The Framework is CC BY 4.0 licensed.
https://opendataproducts.org/framework/
If you are considering ODPS in your organisation, I am happy to compare your current operating model against the framework and identify where adoption should start. A focused 30-minute discussion is often enough to map the first practical steps.
Book a 30-minute call at https://jarkkomoilanen.com/engage
