Camel as a programming language
Part 1: Origins of the Camel DSL
As of today, no middleware or framework implements the message-oriented programming model in its entirety.
Take Apache Camel, for example. A message in Camel is also part of an exchange class containing properties for internal use, as well as other metadata such as the exchange pattern. The message itself can contain text, XML and JSON, but also Java objects such as maps and lists, and binary data such as images. So the framework is not strictly message-oriented as I defined it.
Disclaimer
The views expressed in this blog are my own personal observations and opinions. There is currently no active debate within the Camel community on whether it should be a framework or a language, nor are there any plans for this.
Apache Camel does this for various reasons:
- Multi-paradigm
Camel isn’t purely message-oriented, but embraces multiple programming models, such as event-driven models, and component-based and object-oriented programming.
2.Pragmatic
Camel uses variables, because it is practical to put data outside the message, rather than burdening the message throughout the whole flow. Another practical aspect is that it can execute ordinary Java and other imperative languages, such as Groovy, Simple and JavaScript.
3. Java
Camel isn’t a standalone programming language, but an integration framework implemented in Java. It can be embedded in Java, and most of its users are Java programmers. It is therefore no surprise that it uses concepts such as Java beans directly.
From the perspective of message-oriented programming, Camel is better understood as exchange-oriented: the message is embedded in an execution context represented by the Exchange. The exchange in Camel is a Java object that contains much more than a message:
Exchange
├── Message
│ ├── body
│ └── headers
│ └── attachments
├── properties
├── variables
├── exception
├── exchange pattern
└── other stateThis differs from a strict message-oriented model in which the message itself is the fundamental unit of computation and state is represented through messages rather than through an external execution context.
Combining High-Level Integration DSLs with Plain Java
To understand why Camel looks the way it does, it helps to go back to the intentions of its creator, James Strachan. In a conversation about Camel’s design, Strachan described the project less as an attempt to invent a new programming model and more as an attempt to make the concepts of enterprise integration directly expressible in Java, as being the main enterprise language.
The key idea was the Enterprise Integration Patterns (EIP) catalog. The patterns already existed as a well-established vocabulary for describing messaging and integration problems. Strachan’s insight was that these patterns could become constructs in a domain-specific language. Instead of merely documenting concepts such as routing, filtering, splitting, aggregation and content-based routing, a DSL could make those concepts directly programmable.
This explains an important characteristic of Camel: the DSL is deliberately separate from the underlying routing engine. Camel can provide Java DSL, XML DSL, YAML DSL and other ways of expressing essentially the same routing model. The syntax can change without fundamentally changing the execution model underneath it. As Strachan put it, this separation was intentional: different DSLs should be able to express the same routing concepts.
But there was another deliberate choice: Camel was not designed as a new messaging-oriented programming language. It was designed as a DSL embedded in Java.
Strachan had already created Groovy with a similar philosophy of pragmatism. His goal with Groovy was to make the Java platform more expressive while remaining familiar to Java programmers and retaining access to the Java ecosystem. He brought a similar philosophy to Camel. Integration developers could describe their integration using EIP constructs, but when they needed something outside the DSL, they could simply drop down into ordinary Java code.
This is particularly important in integration, where performance and interoperability can matter. A route might contain high-level routing constructs, scripting or expression languages, and then call a statically typed Java component for a performance-sensitive transformation. Rather than forcing everything into one programming paradigm, Camel deliberately allows these paradigms to coexist.
Get Raymond Meester’s stories in your inbox
Join Medium for free to get updates from this writer.
As Strachan explained it, this was also a pragmatic response to the environment in which Camel was created. Java was the dominant language for enterprise integration, while Strachan already had extensive experience with messaging and integration systems, including TIBCO, IONA, MQSeries, JMS and ActiveMQ. Java therefore provided both the developer audience and the ecosystem into which Camel could naturally fit.
Why Apache Camel Is Built Around the Exchange, Not the Message
This also explains why Camel’s fundamental abstraction is not simply a message.
From a language-design perspective, Camel could perhaps be described as an exchange-oriented integration language. The message is embedded in a larger execution context containing headers, properties, variables, exceptions, attachments, an exchange pattern and other state. The route operates on that context rather than on an isolated, immutable message.
That choice is closely related to Camel’s origins in enterprise integration. Traditional integration problems are rarely just about transforming one message into another. They involve correlation, transactions, retries, error handling, routing decisions, acknowledgements, state and interaction with external systems. The exchange provides a convenient place to represent that execution context.
Strachan’s view is that this complexity is precisely why messaging and integration are better expressed through frameworks and libraries than through a general-purpose programming language. Distributed messaging is not merely a matter of passing values between functions. Once real systems are involved, questions of transactions, acknowledgements, retries, failures, ordering and exceptions become unavoidable.
In that sense, Camel’s apparent departure from a “pure” message-oriented model is not necessarily a failure to implement one. It reflects a different design goal. Camel was intended to raise the abstraction level of integration programming, not to formalize message passing as a fundamental programming paradigm in the way that object-oriented or functional languages formalize their respective models.
Interestingly, Strachan does not regard this design as something Camel subsequently discovered to be wrong. Looking back at Camel 1.0, he points out how remarkably similar its DSL and API are to those of later versions. In many software projects, the API changes substantially between versions 1.0, 2.0 and 3.0 as developers discover that their original abstraction was incomplete. Camel was unusually fortunate: the EIP and messaging domains were already mature and well understood.
In his words, the problem space was already well defined, and the core concepts and patterns were established. The design therefore “almost fell out naturally” from the domain.
True to the core
Since its initial release, Camel has had to adapt to various architectural styles and technologies. It has always taken a pragmatic approach to this by incorporating new ideas without compromising its core functionality. Examples include service-oriented architecture (SOAP), REST APIs, event-driven architectures (Kafka, etc.), and microservices.
So what started as a thin DSL layer around Java has grown into a thicker layer that addresses all practical use cases for legacy and modern systems, as well as for old and new architectures. In the series on message-oriented programming we discussed how a programming language could look like that uses this model. Especially in the last part where language constructs, taxonomy, and syntax was discussed, you will notice how this programming model was inspired by the paradigms, and
- Dataflow programming (Such as Apache NiFi or Node-Red)
- Actor model (Such as Erlang or Akka)
- Service-oriented architecture (Such as Tibco Businessworks or MuleSoft Anypoint)
- Event Streaming (Such as Apache Kafka, Nats Jetstream, or Apache Beam)
- Enterprise Integration Patterns (Such as Spring Integration and Apache Camel)
Especially those familiar with latest versions of Apache Camel find similarities between these language constructs and Camel DSLs. And although mapping these concepts one-to-one is somewhat of an oversimplification, we can clearly draw parallels:
As Apache Camel is a Java Integration Framework, some concepts are put in a fluent-style Java API as a Domain-Specific-Language, where you can see the route DSL as the main one. These can be expressed in the Java DSL, XML DSL or Yaml DSL. The first is mainly used by Java developers, while the other by non-Java developers (in certain runtimes), low-code platforms or agents.
There are some specialized DSLs for example REST, Data formats or Endpoints, but other things you need to express in Java such as Beans, Processors, Connections (Factories), Marshaling classes, Components or Properties.
Conclusion
So the question is no longer simply how Camel can express Enterprise Integration Patterns. Could Camel evolve from an integration framework into a programming language in its own right — one based on the message-oriented programming model? In the next part, we will examine what is missing from Camel’s current DSLs and how those missing language constructs might be expressed.
