Delta Packages

Delta packages provide a bandwidth-efficient way to upgrade K2s installations by including only the files that have changed between two versions, rather than redistributing the entire package.

Overview

A delta package contains:

Changed files: Files that differ between the source and target versions
Wholesale directories: Complete directories that must be replaced entirely (e.g., bin/kube, bin/docker)
Delta manifest: A JSON file describing the changes and metadata
Debian package changes (optional): Linux package differences for the KubeMaster VM
Apply script: A PowerShell script to apply the delta to an existing installation

Addons are excluded

Delta packages do not include addon files. Addons are managed separately via k2s addons ... and are not modified during a delta upgrade.

Delta Package vs Full Package

Aspect	Full Package	Delta Package
Size	2-4 GB typically	50-1000 MB typically
Contains	All K2s files	Only changed files
Requires	Nothing	Specific source version
Use case	Fresh installs, major upgrades	Minor/patch upgrades

Node packages also support delta mode

In addition to cluster-level delta packages, K2s supports node-specific packages (full and delta) for upgrading bare-metal or VM-based Linux worker nodes independently of the control plane. See Node Packages.

Creating a Delta Package

Prerequisites

Two K2s offline packages (source and target versions)
PowerShell 5.1 or later
Sufficient disk space for extraction (3x package size recommended)

Basic Usage

Bash Session

k2s system package --delta-package `
    -d C:\packages `
    -n k2s-delta-v1.4.0-to-v1.5.0.zip `
    --package-version-from C:\packages\k2s-v1.4.0.zip `
    --package-version-to C:\packages\k2s-v1.5.0.zip

Parameters

Flag	Required	Description
`--delta-package`	Yes	Enables delta package creation mode
`-d, --target-dir`	Yes	Target directory for the output package
`-n, --name`	Yes	Name of the output delta package ZIP file
`--package-version-from`	Yes	Path to the older (source) K2s package ZIP
`--package-version-to`	Yes	Path to the newer (target) K2s package ZIP
`-c, --certificate`	No	Path to code signing certificate (.pfx file)
`-w, --password`	No	Password for the code signing certificate
`-o, --output`	No	Show log output in terminal

Example with Code Signing

Bash Session

k2s system package --delta-package `
    -d C:\packages `
    -n k2s-delta-v1.4.0-to-v1.5.0.zip `
    --package-version-from C:\packages\k2s-v1.4.0.zip `
    --package-version-to C:\packages\k2s-v1.5.0.zip `
    -c path\to\cert.pfx `
    -w mycertpassword

Delta Package Contents

After creation, the delta package contains:

Text Only

k2s-delta-v1.4.0-to-v1.5.0.zip
├── delta-manifest.json          # Metadata and file lists
├── Apply-Delta.ps1              # Application script
├── bin/                         # Changed binaries
│   ├── kube/                   # Kubernetes binaries (wholesale)
│   ├── docker/                 # Docker binaries (wholesale)
│   ├── containerd/             # Containerd binaries (wholesale)
│   └── cni/                    # CNI plugins (wholesale)
├── lib/                         # Changed library files
├── smallsetup/                  # Changed setup scripts
├── scripts/                     # Delta application scripts
│   ├── apply-debian-delta.sh   # Linux package update script
│   └── verify-debian-delta.sh  # Linux package verification
└── debian-delta/                # (Optional) Debian package changes
    ├── added-packages.txt
    ├── removed-packages.txt
    ├── changed-packages.txt
    └── packages/                # Downloaded .deb files

Delta Manifest Structure

The delta-manifest.json file describes the changes between versions. A typical manifest includes:

JSON

{
  "sourceVersion": "1.4.0",
  "targetVersion": "1.5.0",
  "createdAt": "2025-02-03T10:30:00Z",
  "filesAdded": ["lib/new-module.psm1"],
  "filesModified": ["lib/existing-module.psm1"],
  "filesRemoved": ["lib/deprecated-module.psm1"],
  "wholesaleDirectories": ["bin/kube", "bin/docker", "bin/cni", "bin/containerd"],
  "debianDelta": {
    "added": ["new-package"],
    "removed": ["old-package"],
    "changed": ["updated-package"]
  },
  "imageDiff": {
    "added": ["new-image:tag"],
    "removed": ["old-image:tag"]
  }
}

Applying a Delta Package

Prerequisites

An existing K2s installation matching the delta's source version
Administrator privileges
The delta package ZIP file

Application Steps

Extract the delta package to a temporary location:

PowerShell

Expand-Archive -Path "k2s-delta-v1.4.0-to-v1.5.0.zip" -DestinationPath "C:\temp\delta"

Run the upgrade from the extracted delta folder:
Bash Session
```
cd C:\temp\delta
k2s system upgrade
```
Verify the installation:
Bash Session
```
k2s system status
```

Node Packages

Node packages contain the Linux packages and container images required to run K2s worker nodes (kubelet, kubeadm, kubectl, CRI-O, buildah). They are independent of the control-plane installation, allowing you to upgrade bare-metal or VM-based Linux worker nodes without touching the cluster control plane.

Both full node packages and node delta packages are created with the --node-package flag.

Full Node Package

A full node package bundles everything needed for a fresh node installation or a full node upgrade.

Bash Session

k2s system package --node-package --os debian12 `
    -d C:\packages `
    -n debian12-node-v1.8.0.zip

Full Node Package Parameters

Flag	Required	Description
`--node-package`	Yes	Selects node package mode
`--os`	Yes	Target Linux distribution (e.g. `debian12`, `debian13`)
`-d, --target-dir`	Yes	Output directory
`-n, --name`	Yes	Output ZIP file name
`-o, --output`	No	Show log output in terminal

Node Delta Package

A node delta package contains only the Debian packages and container images that changed between two node package versions — significantly reducing transfer size for incremental upgrades.

Bash Session

k2s system package --node-package --delta-package `
    --package-version-from C:\packages\debian12-node-v1.7.0.zip `
    --package-version-to C:\packages\debian12-node-v1.8.0.zip `
    -d C:\packages `
    -n debian12-node-delta-v1.7.0-to-v1.8.0.zip

The --os flag is optional in delta mode — the OS folder is auto-detected from the ZIP structure. Specify it explicitly only when both input ZIPs contain multiple OS directories.

Node Delta Package Parameters

Flag	Required	Description
`--node-package`	Yes	Selects node package mode
`--delta-package`	Yes	Enables delta (diff) mode
`--package-version-from`	Yes	Path to the older (base) node package ZIP
`--package-version-to`	Yes	Path to the newer (target) node package ZIP
`-d, --target-dir`	Yes	Output directory
`-n, --name`	Yes	Output ZIP file name
`--os`	No	OS folder override (e.g. `debian12`); auto-detected if omitted
`-o, --output`	No	Show log output in terminal

Node Delta Package Contents

Text Only

deb12-node-delta-v1.7.0-to-v1.8.0.zip
├── delta-manifest.json      # Metadata and file lists (DeltaType: "node-package")
├── Apply-Delta.ps1          # Manual application helper (Windows host)
├── apply-node-delta.sh      # Manual application helper (Linux node)
├── verify-node-delta.sh     # Manual verification helper (Linux node)
├── packages/
│   └── debian12/            # Changed and added .deb files only
│       ├── kubelet_1.35.2-1.1_amd64.deb
│       └── ...
├── images/                  # Changed and added container image archives
│   ├── pause.tar
│   └── ...
├── packages.removed         # (Optional) Removed .deb filenames, one per line
└── images.removed           # (Optional) Removed image tar filenames, one per line

The delta-manifest.json uses ManifestVersion: "2.0" and DeltaType: "node-package" to distinguish it from a cluster delta manifest.

Applying a Node Upgrade

Use k2s system upgrade --node to upgrade a worker node. The upgrade mode is auto-detected from the ZIP contents:

ZIP contains delta-manifest.json with DeltaType: "node-package" → delta upgrade
(installs only changed/added packages; purges removed ones)
No delta-manifest.json present → full upgrade
(installs all packages present in the ZIP)

Bash Session

k2s system upgrade --node k2s-nodepkg-debian12 --path "C:\ws\DeltaPackage\debian12-node-delta-v1.7.0-to-v1.8.0.zip" -o

The same command works for full node packages:

Bash Session

k2s system upgrade --node k2s-nodepkg-debian12 --path "C:\ws\DeltaPackage\debian12-node-v1.8.0.zip" -o

`k2s system upgrade --node` Flags

Flag	Required	Description
`-n, --node`	Yes	Worker node name as registered in the cluster
`--path`	Yes	Path to the node package ZIP (full or delta)
`-o, --output`	No	Show detailed log output

--node and --path are paired flags

Both flags must always be specified together. Omitting one while providing the other results in an error.

Best Practices

When to Use Delta Packages

✅ Recommended for:

Minor version upgrades (e.g., 1.4.0 → 1.5.0)
Patch releases (e.g., 1.4.0 → 1.4.1)
Bandwidth-constrained environments
Large-scale deployments with many nodes

❌ Not recommended for:

Major version upgrades (e.g., 1.x → 2.x)
Fresh installations
Upgrades spanning multiple minor versions

Version Compatibility

⚠️ Version Matching: Delta packages are version-specific. A delta from v1.4.0 to v1.5.0 can only be applied to installations running exactly v1.4.0.

To upgrade across multiple versions, either:

Create sequential delta packages (v1.4.0 → v1.4.1 → v1.5.0)
Use a full package for the final version

Troubleshooting

Common Issues

Version Mismatch Error

Text Only

Error: Source version mismatch. Expected 1.4.0, found 1.3.0

Solution: Ensure your current installation matches the delta's source version, or use -Force (with caution).

Missing Files After Apply

Check the delta manifest to verify which files were expected:

PowerShell

Get-Content delta-manifest.json | ConvertFrom-Json | Select-Object -ExpandProperty filesAdded

Debian Delta Failures

If Linux package updates fail:

Check network connectivity in the VM
Verify the .deb files exist in debian-delta/packages/
Run verification script to identify missing packages

Logs

Delta package creation logs are written to:

Console output (with -o or --output flag)
K2s log directory: C:\var\log\k2s\

For more detailed logging, use the verbosity flag:

Bash Session

k2s system package --delta-package ... -o -v debug

Integration with CI/CD

Automated Delta Generation

YAML

# Example GitHub Actions workflow snippet
- name: Generate Delta Package
  run: |
    k2s system package --delta-package `
      -d ${{ env.OUTPUT_DIR }} `
      -n "k2s-delta-${{ env.PREV_VERSION }}-to-${{ env.NEW_VERSION }}.zip" `
      --package-version-from "${{ env.PREV_PACKAGE }}" `
      --package-version-to "${{ env.NEW_PACKAGE }}" `
      -o

Artifact Publishing

Delta packages can be published alongside full releases:

Text Only

releases/
├── k2s-v1.5.0.zip                    # Full package
├── k2s-delta-v1.4.0-to-v1.5.0.zip   # Delta from previous minor
└── k2s-delta-v1.4.1-to-v1.5.0.zip   # Delta from previous patch

Delta Packages

Overview

Delta Package vs Full Package

Creating a Delta Package

Prerequisites

Basic Usage

Parameters

Example with Code Signing

Delta Package Contents

Delta Manifest Structure

Applying a Delta Package

Prerequisites

Application Steps

Node Packages

Full Node Package

Full Node Package Parameters

Node Delta Package

Node Delta Package Parameters

Node Delta Package Contents

Applying a Node Upgrade

k2s system upgrade --node Flags

Best Practices

When to Use Delta Packages

Version Compatibility

Troubleshooting

Common Issues

Version Mismatch Error

Missing Files After Apply

Debian Delta Failures

Logs

Integration with CI/CD

Automated Delta Generation

Artifact Publishing

See Also

`k2s system upgrade --node` Flags