Simplify AI Management With The Nutanix Enterprise AI Dashboard

As we help more customers and partners deploy generative AI applications, the conversation always turns to a critical question: "How do we manage and monitor it all?" Building powerful models is one thing, but operating them efficiently, securely, and at scale is another.

With Nutanix Enterprise AI, we provide a turnkey solution to help you select, deploy, manage, and run your LLMs, and at the heart of managing this solution is the NAI Dashboard. It’s your single pane of glass for complete visibility, offering real-time insights from the underlying infrastructure all the way up to the individual API requests.

Let's take a look at the dashboard and explore how each component gives you the control and observability you need.

At-a-Glance Monitoring Widgets

The dashboard is composed of several key widgets, each designed to give you specific, actionable information.

Endpoint Summary

This widget provides an immediate status check on all your inference endpoints. It tells you how many are active and ready to serve requests, how many are pending, hibernated, or have failed.

In our example dashboard: Total Endpoints: 48 Active: 20 Failed: 0 Pending: 5 Hibernated: 23

This is crucial for understanding your model-serving capacity at a glance. Seeing a high number of hibernated endpoints, for example, could be a normal part of your resource management strategy to save costs.

Infrastructure Summary

AI workloads are demanding, and their performance is tied directly to the health of the underlying infrastructure. This widget displays a summary of the health and resource usage of the Kubernetes cluster that hosts Nutanix Enterprise AI.

It covers: Service Health: A roll-up status of critical services. Resource Usage: Real-time Memory, CPU, and Disk utilization. * GPU Count: The total number of GPUs available in the cluster.

Looking at our dashboard, we see: Service Health: Critical Memory Usage: 21.40% CPU Usage: 0.58% Disk Usage: 24.38% * Number of GPUs: 6

The "Critical" service health is an immediate call to action, prompting an administrator to investigate further.

API Requests Summary

This widget tracks the lifeblood of your AI service: the API requests. It gives you a clean summary of total requests and categorizes them as successful, failed, or invalid. On our sample dashboard, we see 109 Successful Requests and zero failed or invalid ones, indicating that the deployed models are responding correctly.

API Requests Trends

This bar chart visualizes the flow of API requests over time, helping you identify peak usage patterns. The dashboard shows the trend over the "Last 14 mins," with request volume fluctuating between 08:50 AM and 09:04 AM. This view is perfect for understanding when your services are most in demand.

Endpoints (Top 5)

Which models are getting the most use? This table answers that question by displaying the five most-used inference endpoints. In our example, the llama-3-2-1b-cpu (55 requests) and embedcpu (54 requests) endpoints are the most popular, both being used by the 'admin' user. This information is invaluable for understanding which AI services are providing the most value.

API Keys (Top 5)

To get an even more granular view of usage, this widget shows which API keys are driving the most traffic. This is essential for tracking consumption by different applications, departments, or customers. Here, we see a single key, request-generator-key, is responsible for all 109 requests, giving us clear accountability for the workload.

Your Central Hub for AI Operations

The Nutanix Enterprise AI dashboard brings together every critical element of your AI environment into one intuitive interface. By providing clear, correlated insights into your endpoints, infrastructure, and API usage, it empowers you to move from reactive troubleshooting to proactive management. You can easily plan for future capacity, optimize resource allocation, and ensure your AI services are running smoothly, all from a single screen.