Skip to the yard
Waxsealai

NVIDIA Minerva Connects AI Factory Telemetry to Trace Infrastructure Problems

Shipped
Shipper
Brian Sparks
Readers
938

Minerva links GPU, network, storage and workload data to detect anomalies, group incidents and help teams investigate AI factory performance.

NVIDIA Minerva Connects AI Factory Telemetry to Trace Infrastructure Problems

NVIDIA Minerva is designed to make large AI infrastructure easier to operate by connecting signals that are usually split across separate monitoring views. The AIOps platform combines telemetry from infrastructure and AI workloads, then uses machine learning to surface anomalies and help operators trace problems to their likely causes. Its target is AI factory yield: the useful work produced through GPU hours, goodput, cluster uptime and token production.

That focus reflects a practical challenge for large clusters. Training and serving workloads depend on many components working together, so one slow or degraded device can affect a much larger job. A GPU running hotter than its peers or an NVMe drive losing performance under load may be early signs of trouble. In a system with hundreds of thousands of GPUs, operators need enough context to tell which signals matter and how they connect.

One view across infrastructure and workloads

Minerva brings together data from NVIDIA GPUs, networking, storage, schedulers and workloads. The platform uses those signals to build a dynamic knowledge graph of the factory’s infrastructure and the jobs running on it. The map includes components such as GPUs, SuperNICs, switches and cables, giving operations teams a way to follow a performance issue across the systems involved.

This shared context addresses a limitation of conventional monitoring setups: networking, compute, site reliability and machine learning teams may each see only part of an operation. A slow training job can involve several layers, and a symptom visible in one team’s dashboard may have its cause elsewhere. Minerva is intended to connect those views so operators can trace a workload slowdown to an affected component, including in very large clusters.

The platform analyzes billions of data points per hour with supervised and unsupervised machine learning. It learns typical behavior, looks for anomalies that have not been seen before and predicts emerging failures. It also correlates events using the factory’s topology. The goal is to turn a burst of related alerts into a smaller incident with a likely root cause, reducing the amount of noise operators have to sift through.

From detection to investigation

Minerva includes a native AI assistant and can connect to external agentic AI tools. Operators can ask questions in natural language and investigate cluster health using live telemetry as context. The assistant can also provide recommended remediation, giving teams a route from detecting a problem to deciding what action to take.

Teams retain control over how those recommendations are applied. They can have an operator review and carry out a fix, or connect Minerva to automation through infrastructure application programming interfaces and external systems. That choice matters in complex operations: a recommendation can speed up diagnosis, while the process for making changes can remain aligned with each team’s existing procedures.

The operational measures Minerva is built to support include longer periods between interruptions, faster repair when incidents occur and better visibility into which alerts need attention. Those are intended outcomes, rather than performance figures reported for a measured deployment. The available product description does not provide benchmark results or quantified reductions in downtime or repair time, so teams assessing the platform will need deployment evidence to determine its effect on their own systems.

Deployment plans and operational questions

IREN is among the companies using Minerva to optimize its AI factories. Denis Skrinnikoff, IREN’s chief technology officer, said the platform gives the operations team one view across networks, GPUs and jobs, helping it focus more on proactive maintenance than troubleshooting as its infrastructure scales. His comments describe the intended operational value, but do not include measured results.

Aolani, Bitdeer AI, IBM, Jio, Lambda, Quake AI and Tesla plan to deploy Minerva. Dell Technologies, Lenovo and Supermicro will offer it as part of their AI factory operations portfolios. These plans indicate interest across infrastructure operators and providers, though they do not by themselves establish the platform’s performance at production scale.

For teams evaluating Minerva, the key questions are practical: how well its topology model reflects a particular deployment, whether its anomaly detection produces useful signals, and how its recommendations fit existing response and automation workflows. Connecting telemetry across layers could help operators shorten investigations, especially when workload symptoms and infrastructure faults appear in different places. The value will depend on how accurately Minerva turns that joined-up view into diagnoses teams can act on.

Learn more about NVIDIA Minerva.

Source: Brian Sparks · blogs.nvidia.com

Same vesselRelated cargo