The AI infrastructure problem: When conventional monitoring isn’t sufficient
As AI transforms enterprise operations, community infrastructure faces new challenges. The efficiency of the community will influence efficiency of AI for important workloads, but we’re nonetheless utilizing the identical conventional community and repair monitoring approaches.
Community service efficiency has historically been measured utilizing Layer 3 (IP, similar to delay, packet loss, and jitter) and Layer 4 (TCP) and even Layer 7 (HTTP) metrics. As we transfer into the period of agentic AI, a single request from a human or an API may generate lots of of interactions between AI brokers and huge language fashions (LLMs). This implies we have to take a look at what we measure to find out community efficiency and the way we measure it.
Almost seven in ten firms (69%) rank AI as a high IT funds precedence within the Cisco AI Readiness Index survey of worldwide companies (Realizing the Worth of AI: Cisco AI Readiness Index 2025). Rising AI workloads are anticipated to stretch infrastructure for all, with 63% of companies anticipating AI workloads will rise greater than 30% within the subsequent two to a few years.
Why AI site visitors requires community assurance to adapt
As AI turns into more and more central to the operating of a company, the power to make sure end-to-end service ranges of AI workflows turns into important. This can embrace the efficiency of coaching or inferencing clusters inside an information middle, or interactions over a WAN between inferencing and LLMs, to AI brokers which might be collaborating on an autonomous course of.
As AI adoption accelerates, the transport community turns into a mission-critical basis. We have to perceive the position the community performs within the efficiency of AI workloads, and guarantee the community is delivering the efficiency required in order that these workloads are performing as much as buyer expectations. Conventional monitoring instruments don’t present sufficient granularity to detect points that can influence the efficiency calls for of AI site visitors, and so they don’t measure the efficiency of the AI brokers and the LLMs they’re interacting with over the community.
5 methods to make your community assurance AI-ready
How prepared is your community assurance to deal with the calls for of AI site visitors? Listed below are the 5 methods proactive assurance options can adapt to guarantee AI workload efficiency and guarantee your different community site visitors shouldn’t be negatively impacted.
1. Set up AI-specific efficiency baselines
Steady proactive assurance of conventional community metrics at Layer 3 supplies latency, jitter, throughput, and packet loss metrics to benchmark efficiency for inference, coaching, and agent site visitors. It flags anomalies early earlier than AI processing is disrupted. As well as, we have to perceive how the community situations influence the efficiency of brokers and LLMs throughout the community. This requires that we develop new LLM metrics to make sure you’re measuring the request latency, time to first token, and time per output token (i.e., the period of time taken to generate every token after the primary token or inter-token latency).
2. Present AI-centric WAN and path analytics
With real-time path visibility and clever telemetry, assurance verifies that AI site visitors throughout knowledge facilities, edge nodes, and public or non-public clouds meets service degree agreements (SLAs). That is important for retrieval-augmented technology (RAG) workflows, mannequin sync, and distributed AI operations.
Determine 1. As AI brokers and LLMs evolve, community latency turns into extra important
3. Correlate LLM agent efficiency with community situations
Assurance sensors observe how the interactions between brokers and LLMs behave underneath various community situations. When efficiency is impacted, assurance analytics helps pinpoint whether or not the efficiency problem pertains to the mannequin or whether or not the community is the basis trigger. This hurries up imply time to decision (MTTR) and mitigates blame-shifting amongst distributors or groups.
4. Optimize assets and implement SLAs dynamically
Utilizing policy-based automation, assurance will help in intelligently routing AI workloads in keeping with efficiency wants. It mitigates microbursts, enforces high quality of service, and ensures inference site visitors will get precedence. Optimization additionally consists of clever routing of requests to the LLM that greatest meets the efficiency necessities, similar to time to first token, request latency, tokens per second, failure fee, and so forth. This visibility is important however so, too, is guaranteeing you could have full observability of LLM transactions, LLM redundancy and switchover, load balancing, semantic and value optimization, regulatory compliance, and guardrails and safety.
5. Future-proof operations with open, automated assurance
Designed for agility, assurance options have to be adaptive and open to assist new telemetry sources and assurance fashions. Assurance AIOps, cloud orchestrators, and federated cross-domain knowledge collectively allow closed-loop, AI-aware community automation. Rising AI agent frameworks will additional drive autonomous networks with minimal human oversight. The foundational factor for agentic AI structure is the pretraining of huge knowledge units that create basic goal LLMs which might be “fine-tuned” utilizing further domain-specific knowledge to create domain-specific or job-specific LLMs.
Measuring the community efficiency and the efficiency of the LLMs and brokers operating over the community ensures that you’ve visibility into the important elements that can influence the efficiency of AI workloads.
How Cisco Supplier Connectivity Assurance allows AI-ready networks
Cisco Supplier Connectivity Assurance helps assess whether or not networks are “AI-ready.” The answer is evolving to include AI-specific WAN efficiency testing for inference, RAG, and agent-based operations. It additionally introduces LLM agent efficiency sensors that allow correlation between massive language mannequin agent habits and underlying community efficiency.
For instance, Supplier Connectivity Assurance will help establish, categorize, and localize community site visitors generated by AI workloads and purposes and supply assurance for AI-focused providers throughout the community. Supplier Connectivity Assurance additionally makes it attainable to simulate consumer or agent behaviors by testing LLM efficiency and correlating this with the underlying efficiency of the community.
Full visibility of the transport community is vital. Supplier Connectivity Assurance AIOps supplies the multilayer community visibility that’s required together with the correlation of the a number of layers: optical, nodes, hyperlinks, and paths along with AI consumer expertise.
Getting began: Assess your AI readiness
Earlier than making your community assurance guidelines, consider the present state of your community:
- Can your monitoring answer detect sub-second site visitors anomalies? AI microbursts occur in milliseconds—conventional five-minute polling intervals received’t catch them.
- Do you could have visibility into LLM-specific efficiency metrics? Metrics like time to first token and inter-token latency are important for AI utility efficiency however invisible to standard instruments.
- Are you able to correlate utility efficiency with community situations in actual time? When LLM efficiency degrades, it’s essential know instantly whether or not it’s a mannequin problem or a community problem.
When you answered “no” to any of those questions, your community assurance is probably not prepared for the calls for of AI workloads. The excellent news? Objective-built options like Cisco Supplier Connectivity Assurance can bridge these gaps and put together your infrastructure for AI at scale.
See Cisco Supplier Connectivity Assurance in motion. Request a reside demo to find the way it makes your community AI-ready.
Associated weblog: Reaching Dependable AI Fashions for Community Efficiency Assurance
