Cost-Effective AI with Ollama, GKE GPU Sharing, and vCluster

This post demonstrates how to run cost-effective multi-tenant AI workloads on GKE by combining GPU time-sharing with vCluster isolation.

• •GKE Autopilot is used to dynamically provision GPU nodes (NVIDIA L4) without manual node pool configuration
• •vCluster creates isolated virtual Kubernetes clusters on shared physical nodes, giving each team full admin access without interference
• •Ollama serves open-source LLMs (e.g., Mistral) inside each vCluster, with GPU requests synced to the host cluster automatically
• •GPU time-sharing is configured via node selectors (e.g., time-sharing strategy, max 5 clients per GPU) in the Ollama deployment manifest
• •Two vClusters (demo1, demo2) share a single GPU node provisioned by Autopilot, verifiable via kubectl get nodes on the host cluster