Thursday, November 19, 2026

Agentic Orchestration Is Just SOA with an LLM Control Plane—And That’s a Good Thing

Agentic Orchestration Is Just SOA with an LLM Control Plane—And That’s a Good Thing

If you have spent years building Service-Oriented Architectures (SOA), Enterprise Service Buses (ESBs), or microservice sagas, looking at modern "Agentic AI" diagrams can trigger a strong sense of déjà vu.

You have discrete services with strict schemas. You have a gateway governing routing, rate limits, and fallbacks. You have state persistence for long-running workflows, and telemetry tracking latency and failures.

Recently, I built a hands-on proof of concept to test where traditional SOA ends and true agentic orchestration begins. Using Java 21, Spring Boot, Spring AI (MCP), Bifrost, PostgreSQL, and local Ollama models (alongside Claude), I wanted to answer one question: What actually changes when you put an LLM in the driver's seat of a microservice mesh?

The short answer: The services don't change, the governance doesn't change, and state management doesn't change. What changes is who writes the glue code.


The Trap: Building a Pipeline and Calling It an "Agent"

When developers first experiment with tool-calling, they often build a fixed sequence like this:

  • LocationService(city) → returns County and State.
  • TimeZoneService(state) → returns the IANA Time Zone.
  • TimeService(zoneId) → returns the formatted local time.

If a user queries "Atlanta", the system executes Step 1 → Step 2 → Step 3 and outputs JSON.

As an architect, your immediate reaction should be: Why is there an LLM here? Every request follows the exact same compile-time path. A standard Spring Boot REST controller handles this 100x faster, cheaper, and with zero chance of hallucination.

To justify an LLM control plane, you have to introduce runtime ambiguity, split domain rules, and multi-turn state interruptions where hardcoded if/else trees become brittle.


The Architecture: Classic SOA Meets MCP

Instead of treating the LLM like magic, I structured the stack using classic SOA discipline, exposing Spring Boot services over the Model Context Protocol (MCP):

  • Service Layer (Spring Boot + Spring AI MCP): Exposes Java @Tool methods with strict Java record DTOs (LocationMatch, TimeZoneInfo, LocationTimeReport) backed by live geocoding APIs and local circuit-breaker fallbacks.
  • The ESB / AI Gateway (Bifrost): Sits in front of the MCP server and model providers—handling tool discovery, routing between local Ollama instances and Claude, rate limits, and fallback policies.
  • State & Telemetry Store (PostgreSQL): Persists workflow checkpoints for human-in-the-loop interruptions and records execution metrics.
  • The Control Plane (LLM Orchestrator): Reads the JSON contracts and dynamically decides which tools to call, when to fan out in parallel, when to self-correct, and when to pause.

Four Patterns That Prove Why the LLM Layer Matters

Once the plumbing was live, testing four specific scenarios revealed where an LLM control plane genuinely outperforms static orchestration:

1. Dynamic Supervisor-Worker Fan-Out ("Springfield")

When you query "Atlanta", the agent executes a linear 3-step chain. When you query "Springfield", LocationService returns four distinct U.S. states across three time zones (Illinois, Missouri, Massachusetts, and Oregon). Without writing a single loop in Java, the LLM acts as a Supervisor: it recognizes the ambiguity, emits parallel getTimeZone and getCurrentTime calls across all four states in a single turn (executed concurrently on Java 21 Virtual Threads), and aggregates the results into a unified JSON array.

2. Guardrail Circuit-Breakers ("Franklin")

What happens when a user queries "Franklin"—a city name that exists in over a dozen U.S. states? Without guardrails, an autonomous fan-out triggers dozens of downstream API calls and burns unnecessary tokens. By encoding a Circuit-Breaker policy (> 5 matches), the agent inspects the intermediate result count, halts the parallel fan-out immediately, and shifts strategy.

3. Stateful Human-in-the-Loop Checkpoints ("Franklin" → "Tennessee")

In traditional SOA, long-running workflows waiting on human approval use the Saga / Process Manager pattern to externalize state. I applied the exact same pattern to the agent:

  • Turn 1 (Pause): When "Franklin" hits the > 5 guardrail, the agent calls saveHitlCheckpoint(), saving the candidate list into a PostgreSQL agent_checkpoints table (status = PAUSED_FOR_HITL) and returning a checkpointId (chk-franklin-1360).
  • Turn 2 (Resume): When the user replies "Use Tennessee for checkpoint chk-franklin-1360", the agent calls resumeHitlCheckpoint(). It loads the saved candidate straight from Postgres—skipping external geocoding APIs completely—marks the row RESUMED, and finishes the calculation.

4. Evaluator-Optimizer Self-Reflection ("Gary, Indiana")

Smaller local models running on a workstation (like Gemma or Qwen via Ollama) are fast and private, but they occasionally make domain assumptions—such as assuming all of Indiana uses America/Indiana/Indianapolis, forgetting that Gary sits in Lake County (America/Chicago).

To fix this without escalating every request to a cloud model, I added a deterministic @Tool called validateTimeReport(draft). When I injected a faulty Eastern Time assumption for Gary, Indiana, the Java validator rejected it with a plain-English critique:

"Split-timezone rule violation: Lake County, Indiana must use 'America/Chicago'."

The local model read the error, autonomously queried the Central Time zone, re-validated until valid: true, and documented its own correction in the final JSON output.


How SOA Maps to Agentic Orchestration

If you are an enterprise developer transitioning into AI engineering, your SOA experience isn't obsolete—it is your biggest advantage:

Architectural Layer Traditional SOA / Microservices Agentic Orchestration
Service Contracts WSDL / OpenAPI / Protobuf MCP Tool Schemas & Java record DTOs
Gateway / ESB Kong / Apigee / MuleSoft AI Gateway (Bifrost) for routing & fallbacks
State Management Saga / BPMN Process State in DB Postgres HITL Checkpoints (PAUSED → RESUMED)
Control Flow Compile-time if/else trees Runtime LLM reasoning over tool outputs
Error Handling Static retries & Dead-Letter Queues Evaluator-Optimizer semantic self-correction

The Golden Rule

The biggest mistake engineering teams make today is replacing clean, deterministic microservice pipelines with probabilistic LLM loops.

Never use an agent where a standard REST pipeline will do. Keep your core services fast, strongly typed, and deterministic. Reserve the LLM control plane for the messy edges where rigid code breaks down: translating unstructured human intent, dynamically fanning out across ambiguous results, pausing for human clarification, and self-healing when domain rules fail.

When you treat an LLM not as a replacement for software architecture, but as a dynamic control plane sitting on top of solid SOA, you get the best of both worlds: the adaptability of AI and the reliability of enterprise engineering.