K3s Overview¶
What Is K3s¶
K3s is a lightweight, CNCF-certified Kubernetes distribution built for edge computing, IoT, and resource-constrained environments. Developed by Rancher Labs (now part of SUSE), K3s packages the entire Kubernetes control plane into a single binary under 100 MB. Despite its small footprint, K3s is a fully conformant Kubernetes distribution -- every standard Kubernetes tool, including kubectl and Helm, works identically with K3s as it does with upstream Kubernetes.
K3s achieves its reduced footprint by:
- Replacing
etcdwith an embedded SQLite database (or optional external datastore) for cluster state storage - Removing legacy and alpha features that are rarely used in edge deployments
- Bundling commonly needed components (CoreDNS, Traefik ingress, local-path provisioner) directly into the binary
- Compiling the control plane and kubelet into a single process
Why K3s Instead of Full Kubernetes¶
The Astra system runs on Raspberry Pi hardware with limited CPU and RAM resources. Full upstream Kubernetes (kubeadm-based clusters) requires a minimum of 2 GB of RAM for the control plane alone, and typically consumes significantly more in practice. On a Raspberry Pi with 4-8 GB of total RAM that must also run PostgreSQL, Keepalived, the status dashboard, and other services, this overhead is prohibitive.
K3s provides the same core capabilities with substantially lower resource consumption:
| Resource | Full Kubernetes | K3s |
|---|---|---|
| Control plane RAM | 2+ GB minimum | ~512 MB typical |
| Binary size | Multiple binaries, 200+ MB total | Single binary, <100 MB |
| Startup time | 30-60 seconds | 5-15 seconds |
| ARM64 support | Requires custom builds | First-class ARM64 support |
| Installation | Multi-step kubeadm process |
Single command install |
K3s is specifically designed for ARM architectures and runs natively on the ARM64 processors in the Raspberry Pi 4 without any custom compilation or patching. In the Astra system, K3s runs exclusively on the six Pi 4 units (the cluster nodes). The Pi 5 units and Jetson Orin Nano are dedicated to AI inference and do not run K3s.
Version Information¶
Astra runs K3s version v1.34.x across all nodes:
- Server nodes (node1s): v1.34.3+k3s1
- Agent nodes (node2s): v1.34.6+k3s1
Minor version differences between servers and agents
The slight version difference between server and agent nodes is intentional and fully supported. K3s maintains backward compatibility between minor patch versions, and the agent-server protocol is stable across these releases.
What K3s Provides¶
Pod Scheduling and Self-Healing¶
K3s continuously monitors the health of all running pods. When a pod crashes or becomes unresponsive, K3s automatically restarts it. If the node hosting a pod becomes unavailable, K3s reschedules the pod to another available node in the same cluster.
In Astra, this provides intra-cluster fault tolerance: if an OpenWebUI pod crashes, K3s restarts it within approximately 3 minutes. If a node fails, the pod is rescheduled to the other node in the cluster.
Service Discovery and Networking¶
K3s provides Kubernetes Services, which give pods stable network identities and load-balanced endpoints. The OpenWebUI deployment uses a NodePort service on port 30080, making the application accessible on that port from any node in the cluster.
Helm Chart Support¶
K3s includes full support for Helm, the standard Kubernetes package manager. Astra uses Helm charts to deploy and configure OpenWebUI, which simplifies version management, configuration, and upgrades across all three clusters.
Namespace Isolation¶
All Astra application workloads run in the open-webui namespace, keeping them logically separated from K3s system components that run in kube-system.
Key Configuration¶
The K3s server configuration is stored at /etc/rancher/k3s/config.yaml on each node1 (server node).
Network Policy Disabled¶
This setting disables the K3s network policy controller. On ARM hardware (Raspberry Pi), the network policy controller's initialization process can fail during boot, causing the K3s service to enter a restart loop. The error manifests as:
Disabling network policies prevents this boot loop entirely. Since Astra runs on a physically isolated LAN with no untrusted workloads, network policies are not required for security.
ARM-specific boot loop
This is a known issue affecting K3s on ARM-based single-board computers. Do not remove disable-network-policy: true unless you have verified that the network policy controller initializes correctly on your specific hardware and OS version.
TLS SAN for Remote Access¶
The tls-san (TLS Subject Alternative Name) setting adds additional IP addresses to the K3s API server's TLS certificate. This allows remote kubectl access from the operator's laptop via additional network interfaces without TLS certificate verification errors. In the current ground-based prototype, this is used for remote development access.
Without this setting, connecting to the K3s API server via an IP not covered by the default certificate would produce a certificate error because the default certificate only covers the node's primary LAN IP and localhost.
Agent Node Configuration¶
Agent nodes (node2s) do not have a /etc/rancher/k3s/config.yaml file. The disable-network-policy flag is a server-only option and has no effect on agent nodes. Agents connect to their cluster's server node using the K3s agent token generated during server initialization.
Deployment Files¶
| Path | Node Type | Purpose |
|---|---|---|
/etc/rancher/k3s/config.yaml |
Server (node1) | K3s server configuration |
/var/lib/rancher/k3s/ |
All nodes | K3s runtime data, container images, and state |
/etc/rancher/k3s/k3s.yaml |
Server (node1) | Kubeconfig file for kubectl access |