AI Infrastructure Roadmap

The 2026 skillset

LLMs and GPUs the way production runs them: local inference foundations, GPU sharing, the MLOps stack real platforms are built on (Kubeflow, KServe, batch scheduling), and LLMOps/AIOps for day 2.

5 stages · 16 resources ready · 13 in the works, all free · every stage ends with a checkpoint you can verify on a real cluster

Your journey: 16 steps. Check them off as you go, this page remembers.

Local LLM foundations

~1-2 weeks

Before Kubernetes, GPUs, or platforms: understand how inference actually works on hardware you control.

The local LLM revolutionarticle

Anatomy of an inference requestarticle

Quantization demystified: BF16 to NVFP4 to GGUFarticle

Inference engines: vLLM, llama.cpp, and what to pickarticle

NVIDIA DGX Spark series (playlist)playlist

Docker Model Runner: local AI app, no API keys (video)video

✓ Checkpoint: you can now

Run a model locally and explain tokens per second, quantization, and the memory math.

⚡ Prove it

Run the same model at two quantization levels and measure the speed and quality difference.

GPUs on Kubernetes

~2-3 weeks

GPUs are the scarcest resource in the building. Learn to share them without wasting them.

Slicing GPUs with NVIDIA MIG (includes GPU Operator)article

Sharing GPUs with HAMi (software vGPU)article

GPU-Enabled Platforms on Kubernetes (free e-book)e-book

Benchmark your GPU stack (kubemark-ai)tool

DRA: dynamic resource allocation explainedguide coming

✓ Checkpoint: you can now

Share one GPU across workloads and defend your strategy: MIG vs time-slicing vs HAMi.

⚡ Prove it

Slice a GPU with MIG (or simulate with HAMi) and run two isolated workloads on it.

Training & MLOps platform

~2-4 weeks

One model on one GPU is a demo. Teams need notebooks, pipelines, tuning, and fair scheduling: a platform.

Why ML needs a platform: notebooks, pipelines, and experiment trackingguide coming

Batch scheduling: gang scheduling and the Workload API (Kubernetes 1.35 demo)video

Kueue: queues, quotas, and fair sharing for training jobsguide coming

Argo Workflows: pipelines and batch orchestration on Kubernetesguide coming

Kubeflow in 2026: what it is now and when to use itguide coming

Model registries and versioning in practiceguide coming

✓ Checkpoint: you can now

Stand up a real ML platform: notebooks, pipelines, tuning, and batch scheduling with quotas.

⚡ Prove it

Run one training pipeline end to end: train a small model, tune one hyperparameter, and version the result.

Serving & inference in production

~2-3 weeks

Training gets the glory; serving pays the bills. Put models behind reliable, scalable endpoints.

Serving fundamentals: engines, batching, and OpenAI-compatible APIs (from our local LLM series)article

From laptop to cluster: what changes when serving moves to Kubernetesguide coming

vLLM on Kubernetes: production serving deep diveguide coming

KServe: standardized inference endpoints and when you need themguide coming

Scaling inference: autoscaling, caching, and cost per tokenguide coming

✓ Checkpoint: you can now

Serve a model behind a real endpoint and defend your runtime choice (vLLM, Triton, ONNX) like production teams do.

⚡ Prove it

Deploy one model two ways: a standardized serving stack and a raw vLLM deployment; compare latency and ops burden.

LLMOps, AIOps & agents

~ongoing

Day 2: costs, observability, memory, and letting AI help run the infrastructure itself.

K8sGPT: AIOps, when Kubernetes meets AIarticle

MCP tool control with agentgatewayarticle

LLM cost and observabilityarticle

Give your agents verified memory (memwarden)tool

LLMOps: evals, prompt versioning, and safe rolloutsguide coming

RAG infrastructure: vector databases on Kubernetesguide coming

Model and drift monitoring in productionguide coming

✓ Checkpoint: you can now

Run models and agents in production with observability, cost control, and memory hygiene.

⚡ Prove it

Put an agent behind a gateway, cap its spend, audit what it remembers, and let K8sGPT explain your next cluster incident.

Want to know when the 🚧 steps ship?

New articles, videos, and labs land in the newsletter first.

Get updates →