Saturday, November 21, 2026

From Spaghetti Prompts to Composed Orchestration: Agentic RAG-as-a-Tool with Spring AI and Milvus

Transparency Disclaimer: The architectural diagram and the entire codebase in this project were developed by Google AI—I did not write or add a single line of code. My role was strictly that of the architect and systems thinker: defining the architectural boundaries, integrating the infrastructure components, testing edge cases, and guiding the structural refactoring. Full source code is available on GitHub at github.com/htotapally/geo-time-adaptive-agent.

In my previous post, we built a sub-second Adaptive UI on top of Java 21, Spring Boot, Spring AI, Bifrost, PostgreSQL, and local Ollama (Qwen 2.5 7B)—all available in our open-source repository at github.com/htotapally/geo-time-adaptive-agent.

Since that post, we expanded the system with local text embeddings (using embeddinggemma-2:270m and a Milvus vector database) so the agent can resolve cultural city nicknames like "The Big Easy" or "Music City" when exact geocoding lookups return zero matches.

As we added that vector fallback, I noticed a classic software engineering trap creeping into our controller: long, monolithic methods. When intent parsing, external geocoding, vector search, parallel fan-out, and Human-in-the-Loop (HITL) checkpoints all live inside one giant method, the code becomes hard to read and nearly impossible to unit test.

1. Why Monolithic Agent Methods Hurt Engineering Quality

In many AI tutorials, the main request handler turns into a sprawling wall of prompt strings, if/else branches, and inline API calls. Even when the logic works, it violates clean software design:

  • Poor Readability: You cannot tell at a glance what the high-level workflow is doing without tracing every nested conditional.
  • Broken Unit-Testability: Testing whether a >5 match guardrail triggers a PostgreSQL checkpoint forces you to mock the entire world or spin up a live LLM.
  • Hidden Architectural Patterns: Important agentic behaviors—like semantic fallback and evaluator validation—get buried in implementation noise.

2. Refactoring to a 4-Step Composed Orchestration

Applying the Composed Method pattern, we refactored GeoTimeAgentController.queryCity() so the top-level endpoint reads like a clean 4-stage architectural pipeline, delegating execution to small, atomic methods:

@PostMapping("/query")
public LocationTimeReport queryCity(@RequestBody QueryRequest request) {
    String rawQuery = request.query().trim();

    // Step 1: Hybrid Semantic Router / Intent Extraction
    ParsedIntent intent = parseIntent(rawQuery);

    // Step 2: Deterministic Tool Execution (Structured Geocoding)
    List<LocationMatch> matches = findMatchingLocations(intent);

    // Step 3: Agentic RAG-as-a-Tool (Autonomous Vector Fallback on Zero Matches)
    if (matches.isEmpty()) {
        matches = resolveViaRagFallback(rawQuery);
    }

    // Step 4: Dynamic Control Flow (Pipeline vs. Parallel Fan-Out vs. HITL Guardrail)
    String resolvedCity = matches.isEmpty() ? intent.city() : matches.get(0).city();
    return routeByAmbiguity(resolvedCity, matches);
}

4-Stage Composed Agentic Orchestration & Milvus RAG Fallback

3. How Each Atomic Step Maps to an Agentic AI Pattern

The most rewarding outcome of this refactoring was seeing that each atomic Java method evolved into a textbook Agentic AI design pattern:

Orchestration Step Agentic Design Pattern Architectural Role
1. parseIntent(rawQuery) Hybrid Semantic Router Splits structured inputs ("Gary, IN") in <1ms; routes unstructured sentences to Qwen 2.5 7B via Bifrost.
2. findMatchingLocations(intent) Deterministic MCP Tool Execution Queries live Open-Meteo geocoding APIs using strongly typed Java records rather than risking LLM hallucinations.
3. resolveViaRagFallback(rawQuery) Agentic RAG-as-a-Tool Invokes Milvus + embeddinggemma-2:270m only when exact geocoding returns 0 matches (e.g., "The Big Easy" → "New Orleans, Louisiana").
4. routeByAmbiguity(city, matches) Dynamic Control Flow & Evaluator Selects Direct Card + Evaluator (1 match), Virtual-Thread Fan-Out (2–5 matches), or PostgreSQL HITL Checkpoint (>5 matches).

4. Agentic RAG-as-a-Tool vs. Naive RAG

Traditional RAG pipelines run vector similarity search on every single user query—adding unnecessary latency and risking noisy matches even when the user typed an exact city name like "Atlanta, Georgia".

By encapsulating Milvus and local embeddinggemma-2:270m embeddings inside resolveViaRagFallback(), vector search becomes an autonomous fallback microservice: structured APIs handle exact queries in milliseconds, and semantic search triggers strictly when exact lookup comes up empty.

5. Independent Unit Testing for Every Stage

Because every stage is now a focused, single-responsibility method, you can write fast, isolated JUnit tests for each agentic behavior:

  • Test Intent Parsing: Verify parseIntent("Gary, IN") resolves deterministically without touching ChatClient.
  • Test RAG Fallback: Mock lookupLocation("The Big Easy") returning empty and assert resolveViaRagFallback() queries Milvus and resolves New Orleans.
  • Test Guardrail Routing: Pass 10 mock matches into routeByAmbiguity() and verify a PostgreSQL HITL checkpoint is created without fanning out external API calls.

Explore the Full Source Code

Inspect the complete Spring Boot 3.4+, Spring AI MCP, Milvus VectorStore, Bifrost, and Adaptive UI codebase on GitHub:

View Repository on GitHub →

No comments:

Post a Comment