Features

Side-loading Images

Kind allows you to side-load images in your local clusters. Cinder exposes the same functionality via cinder load:

cinder load criticalstack/quake-kube:v1.0.5

This will make the criticalstack/quake-kube:v1.0.5 image from the host available in the Cinder node. Any image that is available on the host can be loaded, and Cinder lazily pulls images that are not found on the host.

Registry Mirrors

Mirrors for container image registries can be setup to effectively "alias" them. The key is the alias, and the value is the full endpoint for the registry:

apiVersion: cinder.crit.sh/v1alpha1
kind: ClusterConfiguration
registryMirrors:
  docker.io: "https://docker.io"

It can be used to alias registries with different names OR it can be used to specify plain http registries:

...
registryMirrors:
  myregistry.dev: "http://myregistry.dev"

Local Registry

An instance of Distribution (aka Docker Registry v2) can be setup for a Cinder cluster by specifying a config file with the LocalRegistry feature gate:

apiVersion: cinder.crit.sh/v1alpha1
kind: ClusterConfiguration
featureGates:
  LocalRegistry: true

This will start a Docker container on the host with the running registry (if not already running). The registry is shared for all Cinder clusters on a host and is available at localhost:5000 (i.e. this is what you docker push to). This registry is then available inside the cluster at cinderegg:5000.

Cinder also creates the local-registry-hosting ConfigMap so that any tooling that supports Local Registry Hosting, such as Tilt, will be able to automatically discover and use the local registry.

apiVersion: v1
kind: ConfigMap
metadata:
  name: local-registry-hosting
  namespace: kube-public
data:
  localRegistryHosting.v1: |
    host: "localhost:{{ .LocalRegistryPort }}"
    hostFromContainerRuntime: "{{ .LocalRegistryName }}:{{ .LocalRegistryPort }}"
    hostFromClusterNetwork: "{{ .LocalRegistryName }}:{{ .LocalRegistryPort }}"
    help: "https://docs.crit.sh/cinder-guide/local-registry.html"`

More information about this Kubernetes standard can be found here.

Krustlet

Krustlet is a tool to run WebAssembly workloads natively on Kubernetes by acting like node in your Kubernetes cluster. It can be enabled for a Cinder cluster using the following configuration:

# config.yaml
apiVersion: cinder.crit.sh/v1alpha1
kind: ClusterConfiguration
featureGates:
  LocalRegistry: true
  Krustlet: true
controlPlaneConfiguration:
  kubeProxy:
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
            - matchExpressions:
              - key: "kubernetes.io/arch"
                operator: NotIn
                values: ["wasm32-wasi", "wasm32-wascc"]

to create a new Cinder cluster:

❯ cinder create cluster -c config.yaml
Creating cluster "cinder" ...
 🔥  Generating certificates
 🔥  Creating control-plane node
 🔥  Installing CNI
 🔥  Installing StorageClass
 🔥  Running post-up commands
Set kubectl context to "kubernetes-admin@cinder". Prithee, be careful.

Note that node affinity is being set for kube-proxy to ensure it does not try to schedule a pod on either the WASI or WASCC nodes

This will start two instances of Krustlet for both WASI and waSCC runtimes:

❯ kubectl get no

NAME           STATUS   ROLES    AGE   VERSION
cinder         Ready    master   2m    v1.18.5
cinder-wascc   Ready    <none>   1m    0.5.0
cinder-wasi    Ready    <none>   1m    0.5.0

With these nodes ready, we can build and push images to our local registry and run them on our Cinder cluster. For example, the Hello World Rust for WASI can be built using cargo and pushed to our local registry using wasm-to-oci:

❯ cargo build --target wasm32-wasi --release
❯ wasm-to-oci push --use-http \
    target/wasm32-wasi/release/hello-world-rust.wasm \
    localhost:5000/hello-world-rust:v0.2.0

The line in k8s.yaml specifying the image to use will need to be modified:

...
spec:
  containers:
    - name: hello-world-wasi-rust
      #image: webassembly.azurecr.io/hello-world-wasi-rust:v0.2.0
      image: cinderegg:5000/hello-world-rust:v0.2.0
...

Finally, the manifest can be applied:

❯ kubectl apply -f k8s.yaml

Which will result in the pod being scheduled on the waSCC Krustlet:

❯ kubectl get po -A

NAMESPACE            NAME                                  READY   STATUS                          RESTARTS   AGE
kube-system          cilium-operator-657978fb5b-frrxj      1/1     Running                         0          8m4s
kube-system          cilium-pqmsc                          1/1     Running                         0          8m4s
kube-system          coredns-pqljz                         1/1     Running                         0          7m57s
kube-system          hello-world-wasi-rust                 0/1     ExitCode:0                      0          1s
kube-system          kube-apiserver-cinder                 1/1     Running                         0          8m18s
kube-system          kube-controller-manager-cinder        1/1     Running                         0          8m18s
kube-system          kube-proxy-85lwd                      1/1     Running                         0          8m4s
kube-system          kube-scheduler-cinder                 1/1     Running                         0          8m18s
local-path-storage   local-path-storage-74cd8967f5-vv2mb   1/1     Running                         0          8m4s

And should produce the following log output:

❯ kubectl logs hello-world-wasi-rust

hello from stdout!
hello from stderr!
POD_NAME=hello-world-wasi-rust
FOO=bar
CONFIG_MAP_VAL=cool stuff
Args are: []

Bacon ipsum dolor amet chuck turducken porchetta, tri-tip spare ribs t-bone ham hock. Meatloaf
pork belly leberkas, ham beef pig corned beef boudin ground round meatball alcatra jerky.
Pancetta brisket pastrami, flank pork chop ball tip short loin burgdoggen. Tri-tip kevin
shoulder cow andouille. Prosciutto chislic cupim, short ribs venison jerky beef ribs ham hock
short loin fatback. Bresaola meatloaf capicola pancetta, prosciutto chicken landjaeger andouille
swine kielbasa drumstick cupim tenderloin chuck shank. Flank jowl leberkas turducken ham tongue
beef ribs shankle meatloaf drumstick pork t-bone frankfurter tri-tip.