<-- Back to All News

Building real-time agents for EV charging using Gemma and TPUs

Publish Date: November 13, 2026

Executive Overview

As electric vehicle (EV) adoption accelerates across global transportation networks, public charging operators face mounting operational pressure to deliver instantaneous, reliable driver assistance and proactive hardware fault resolution. Stranded drivers experiencing payment failures, connector locks, or charger communication timeouts require immediate resolution in their native language, yet traditional call centers and generic cloud chatbots struggle with high latency, rigid API integrations, and prohibitive operational costs. In a new edition of The Blueprint published on November 12, 2026, Google Cloud detailed how European EV charging AI platform lemonflow engineered real-time voice and network monitoring agents using open Gemma models hosted on dedicated Cloud TPUs.

Operating across more than 40 charging networks in 17 countries and supporting over 250,000 charge points, lemonflow’s agents assist an EV driver every 90 seconds in more than 90 languages. When transitioning from initial prototypes to live voice support, lemonflow discovered that shared, managed LLM APIs introduced unpredictable peak-hour latency spikes and rate-limit bottlenecks that disrupted live human conversations. By migrating inference onto self-hosted open Gemma models running on dedicated Cloud TPUs within European Union regions, lemonflow achieved sub-800-millisecond conversational response times, >99.9% serving reliability, and strict data sovereignty across hundreds of customized operator agents.

Features

The architecture deployed by lemonflow combines open-weight foundation models, dedicated accelerator hardware, and multi-tenant agent orchestration to meet strict real-time voice requirements.

Sub-800ms Real-Time Voice Inference on Cloud TPUs: Live voice conversations break down when response latency exceeds 800 milliseconds, triggering awkward conversational barge-ins where frustrated drivers interrupt the agent. Hosting Gemma models directly on dedicated Google Cloud TPUs eliminates multi-tenant API queuing jitter, delivering deterministic, ultra-low-latency token generation capable of sustaining natural spoken dialogue across more than 90 languages.

Dedicated Capacity Eliminating Rate-Limit Bottlenecks: Relying on shared managed model endpoints previously exposed lemonflow to peak-hour latency deviations (spiking from 1.0–1.2 seconds to over 1.7 seconds) and HTTP 429 rate-limit errors during simultaneous charging surges and live tool calls. Dedicated Cloud TPU allocation guarantees predictable, burst-tolerant throughput during intense concurrent voice interactions.

Unified Abstraction Layer Across 30+ Charge Point Management Systems (CPMS): EV charging infrastructure is deeply fragmented across roughly 30 distinct backend Charge Point Management Systems, each exposing proprietary APIs, telemetry schemas, and hardware control protocols. lemonflow engineered a standardized conversational engine capable of invoking dynamic tool calls across heterogeneous CPMS backends to inspect live charger telemetry, verify payment states, and trigger remote hardware reboots in real time.

Multi-Tenant Per-Operator Agent Customization: Because every charging network enforces unique tariffs, authentication workflows, human escalation paths, and operating hours, lemonflow runs hundreds of tailored agent configurations—segmented by country, brand, and hotline—each driven by specialized system prompts served at full speed without cross-tenant interference.

Sovereign EU Hosting and Proactive Fault Monitoring: All primary TPU inference and fallback buffers execute strictly within European Union cloud regions to satisfy regional data residency mandates. Alongside driver voice support, autonomous AI monitoring agents continuously analyze network telemetry to detect and remediate charger faults proactively before drivers arrive at the station.

Benefits

Transitioning from shared third-party model APIs to open models on dedicated Cloud TPUs delivers decisive operational, financial, and customer experience advantages.

Natural, Interruption-Free Voice Customer Experience: Maintaining serving latency below the critical 800-millisecond threshold prevents conversational collisions, enabling stressed drivers at roadside chargers to receive immediate, empathetic, and accurate troubleshooting in over 90 languages.

99.9%+ Production Availability Without Rate Throttling: Dedicated Cloud TPU infrastructure insulates mission-critical mobility support from external API rate limits, ensuring Gemma serves over 99.9% of all live conversation turns with fallback layers acting purely as zero-downtime deployment buffers.

Rapid Multi-Network Scalability Across Fragmented Hardware: By decoupling conversational reasoning from underlying CPMS API quirks through standardized tool calling, lemonflow onboarded over 250,000 charge points across 17 countries on a single unified platform.

Predictable Unit Economics for High-Frequency Interactions: Serving an EV driver every 90 seconds around the clock generates massive token volumes. Running right-sized open Gemma models on dedicated TPUs provides predictable, fixed infrastructure economics compared to variable per-token pricing on proprietary frontier APIs.

Use Cases

The architectural patterns validated by lemonflow extend across real-time voice AI, industrial IoT, and distributed field operations.

24/7 Multilingual Roadside EV Driver Support: Voice agents authenticate stranded drivers, diagnose charger error codes via live backend API calls, initiate remote charger resets, and guide drivers through payment or cable release procedures in 90+ languages.

Autonomous Charging Network Telemetry and Predictive Remediation: Background monitoring agents continuously poll CPMS health streams, identifying offline connectors, payment gateway timeouts, or firmware hangs and executing automated recovery scripts before customers are impacted.

High-Concurrency Voice Assistants for Distributed Infrastructure: Telecommunications, utilities, and public transit operators deploy open models on dedicated TPUs to power low-latency voice support lines that integrate directly with legacy operational control systems.

Sovereign European Enterprise AI Deployments: Organizations operating under strict GDPR and EU data residency mandates utilize self-hosted Gemma weights on regional Cloud TPUs to retain complete architectural custody over customer voice transcripts and operational telemetry.

Alternatives

Organizations architecting low-latency, multilingual voice agents and industrial IoT assistants evaluate several competing infrastructure approaches.

  • Shared Managed Proprietary LLM APIs (OpenAI / Anthropic / Managed Gemini Endpoints): Consuming serverless commercial model APIs offers the fastest initial prototyping speed with zero infrastructure management. However, as lemonflow experienced in production, shared public endpoints remain vulnerable to peak-hour latency jitter exceeding one second and HTTP 429 rate-limit throttling during sudden traffic bursts, making strict sub-800ms voice SLAs difficult to guarantee without dedicated provisioned throughput.
  • Self-Hosted Open Models on Cloud GPU Instances: Engineering teams frequently deploy open models (such as Llama or Mistral) on virtual machines equipped with NVIDIA GPUs. While offering full model custody, GPU instances often incur higher per-watt infrastructure costs and supply constraints compared to Google Cloud TPUs, which are purpose-built for high-throughput, low-latency transformer matrix operations.
  • Traditional Interactive Voice Response (IVR) and Rule-Based Chatbots: Legacy call centers rely on rigid DTMF phone trees and scripted decision-tree bots. While deterministic, rule-based IVRs fail completely when handling nuanced multilingual descriptions, complex multi-API charger diagnostics, or dynamic hardware state recovery.
  • Off-the-Shelf Contact Center as a Service (CCaaS) AI Add-Ons: Enterprise telephony vendors provide built-in conversational AI modules. While easy to integrate into existing call routing, turnkey CCaaS bots lack the deep, low-latency tool-calling extensibility required to interface simultaneously with 30 distinct industrial Charge Point Management Systems.
An Alternative Perspective

A thorough architectural evaluation requires examining the engineering overhead, operational responsibilities, and trade-offs inherent in self-hosting open models on dedicated accelerator hardware.

First, moving from managed serverless APIs to self-hosted Gemma deployments on dedicated Cloud TPUs shifts operational responsibility squarely onto the engineering team. Instead of consuming an abstracted HTTP endpoint, platform engineers must manage model serving runtimes, container orchestration, TPU slice health monitoring, zero-downtime rolling updates, and traffic load balancing across availability zones. While lemonflow achieved >99.9% reliability, sustaining that availability requires mature MLOps and site reliability engineering capabilities.

Second, dedicated accelerator capacity introduces fixed baseline infrastructure costs regardless of diurnal traffic valleys. Public EV charging usage fluctuates significantly between peak daytime commuter hours and overnight lulls. Unless engineering teams carefully right-size TPU topologies or multiplex batch monitoring workloads onto idle nighttime capacity, dedicated hardware reservations can experience underutilization during off-peak hours.

Finally, relying on compact open models like Gemma requires disciplined prompt engineering and rigorous tool-call schema validation. While smaller open models deliver exceptional inference speed below 800 milliseconds, they possess narrower reasoning headroom than massive frontier models, meaning complex multi-step edge cases across 30 different CPMS backends must be constrained with deterministic API guardrails and clear human escalation triggers.

Final Thoughts

lemonflow’s deployment of Gemma on Cloud TPUs provides a compelling blueprint for production voice AI in physical infrastructure industries. By recognizing that conversational voice interfaces live or die by sub-800-millisecond latency and deterministic throughput, lemonflow avoided the pitfalls of shared API bottlenecks and built a resilient, multilingual platform supporting over a quarter-million charge points.

For technology leaders, startup founders, and AI architects building real-time customer-facing agents, this architecture demonstrates the strategic value of combining efficient open-weights models with dedicated cloud accelerators. Organizations that match model size to latency SLAs and retain direct control over serving infrastructure will deliver seamless, reliable AI experiences at global scale.

Source

https://cloud.google.com/blog/topics/startups/building-real-time-agents-for-ev-charging-using-gemma-and-tpus