Friday, November 20, 2026

Beyond the Chatbot: Building a Sub-Second Adaptive UI with Spring Boot, Bifrost, and Local LLMs

Beyond the Chatbot: Building a Sub-Second Adaptive UI with Spring Boot, Bifrost, and Local LLMs
Transparency Disclaimer Every single line of Java, SQL, YAML, and HTML/JavaScript code in this project was generated through pair-programming with Google AI. I did not write a single line of code manually. My role was strictly that of the architect and systems thinker—defining the architectural patterns, connecting the infrastructure components, testing edge cases, and diagnosing bottlenecks.

In my previous post, I explored how Agentic Orchestration is really just Service-Oriented Architecture (SOA) with an LLM control plane. Once the backend services, guardrails, and PostgreSQL checkpoints were working via MCP (Model Context Protocol), the next question was obvious: How should an actual end user interact with this system?

Most AI demos today default to a chat-first interface—a blank text box where every interaction requires typing sentences and waiting for streaming paragraphs. As an architect, that feels like a step backward. If a user simply wants the current local time in "Gary, Indiana", they don't want a conversation; they want a clean, structured UI card. They only need a chat interface when the system hits genuine ambiguity and needs human clarification.

To test this idea, I built a self-hosted Adaptive UI on top of Java 21, Spring Boot, Spring AI, Bifrost, PostgreSQL, and Ollama (Qwen 2.5 7B). Along the way, I ran straight into a 20-second latency wall—and discovered the exact hybrid pattern needed to make it 30x faster.

1. What Is an Adaptive UI?

Instead of forcing every request through a chatbot, an Adaptive UI acts like a deterministic web application when inputs are clear, and dynamically pops up a conversational modal only when the agent encounters ambiguity or requires Human-in-the-Loop (HITL) approval.

In our Spring Boot application, a single strongly typed JSON contract (LocationTimeReport) drives three completely different user experiences:

  • Mode 1 — Unambiguous Pipeline ("Gary, Indiana"): The backend resolves Lake County, Indiana (America/Chicago) and validates the offset. No chat window appears. The page immediately renders a clean, green-bordered result card with the verified local time.
  • Mode 2 — Moderate Ambiguity ("Springfield"): The backend finds 4 matching U.S. states across 3 time zones and resolves them concurrently on Java 21 Virtual Threads. A Disambiguation Modal pops up displaying clickable cards for Illinois, Missouri, Massachusetts, and Oregon. Clicking a card promotes that state straight to the main view.
  • Mode 3 — High Ambiguity & HITL Checkpoint ("Franklin" → "Virginia"): The backend hits the > 5 matches guardrail, freezes all candidate coordinates into a PostgreSQL agent_checkpoints table (PAUSED_FOR_HITL), and returns a checkpointId. An Interactive Clarification Chat Drawer pops up where typing "Virginia" resumes the saved Postgres state and transitions smoothly into the verified result card.

2. The 20-Second Latency Wake-Up Call

In my first iteration, I let Spring AI's ChatClient and Qwen 2.5 7B (running locally on Ollama behind the Bifrost AI gateway) orchestrate every single step from scratch. Even for a straightforward query like "Gary, Indiana", the UI took 15 to 25 seconds to respond.

Why? Because the system was paying for four sequential LLM inference round-trips over Docker networking:

  1. Send the prompt + 6 JSON tool schemas to Qwen 2.5 7B → model decides to call lookupLocation.
  2. Send geocoding results back to Qwen 2.5 7B → model decides to call getTimeZone.
  3. Send the IANA zone back to Qwen 2.5 7B → model decides to call getCurrentTime.
  4. Send the timestamp back to Qwen 2.5 7B → model builds the draft report, calls validateTimeReport, and serializes the final JSON.

We were using a 7-billion-parameter neural network to execute a deterministic Step 1 → Step 2 → Step 3 sequence that compiled Java bytecode can run in less than one millisecond.

3. The Fix: Hybrid Neuro-Symbolic Fast-Path

Applying the core lesson from SOA—never pay LLM latency for deterministic control flow—we refactored the Spring Boot orchestrator into a Hybrid Fast-Path:

Execution Concern Pure LLM Loop (~20s) Hybrid Fast-Path (~500ms)
Structured Routing 4 sequential LLM inference turns Compiled Java 21 branching (< 1ms)
Multi-State Fan-Out LLM parses & summarizes all branches Java 21 Virtual Threads (parallelStream)
HITL State Resume LLM re-plans tool calls from scratch Direct PostgreSQL checkpoint lookup (< 5ms)
Role of Qwen 2.5 7B Every single step (heavy bottleneck) Semantic edges only (messy intent & chat replies)

Key Architectural Takeaways

  1. Ditch "Chat-First" for Structured Workflows: Use clean UI cards for deterministic results, clickable pills for bounded choices, and pop up a conversational chat drawer only when ambiguity or human approval demands it.
  2. Code Is Commodity; Architecture Is the Differentiator: With modern AI generating the boilerplate Java, SQL, and Alpine.js/Tailwind frontend in minutes, an engineer's highest-leverage skill is knowing where to place deterministic boundaries, state checkpoints, and LLM reasoning.
  3. Hybrid Beats Pure-Agent Every Time: Pairing compiled Java 21 + PostgreSQL state with a local LLM via an AI gateway like Bifrost gives you sub-second speed on happy paths and genuine natural-language adaptability on messy edge cases.
geo-time-adaptive-agent is the link to the code.

No comments:

Post a Comment