Skip to content

Conversation

nirs
Copy link
Contributor

@nirs nirs commented Aug 15, 2025

Newer krunkit allows disabling offloading for faster networking. Keep the Offloading option in case offloading is improved in future versions.

Testing shows 6.7 times faster network performance, and 9p mount is 3 times faster.

Starting cluster:

% minikube start --driver krunkit --container-runtime containerd
😄  minikube v1.36.0 on Darwin 15.6 (arm64)
✨  Using the krunkit (experimental) driver based on user configuration
👍  Starting "minikube" primary control-plane node in "minikube" cluster
🔥  Creating krunkit VM (CPUs=2, Memory=6144MB, Disk=20000MB) ...
📦  Preparing Kubernetes v1.33.2 on containerd 1.7.23 ...
🔗  Configuring bridge CNI (Container Networking Interface) ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: default-storageclass, storage-provisioner
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

Testing iperf3:

% kubectl apply -f iper3-server.yaml
deployment.apps/iperf3 created
service/iperf3 created

% kubectl get deploy iperf3
NAME     READY   UP-TO-DATE   AVAILABLE   AGE
iperf3   1/1     1            1           9s

% kubectl get service iperf3
NAME     TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
iperf3   NodePort   10.105.127.180   <none>        5201:30201/TCP   17s

% iperf3 -c $(minikube ip) -p 30201
Connecting to host 192.168.105.10, port 30201
[  5] local 192.168.105.1 port 50630 connected to 192.168.105.10 port 30201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  1.05 GBytes  9.03 Gbits/sec
[  5]   1.00-2.01   sec  1.09 GBytes  9.29 Gbits/sec
[  5]   2.01-3.01   sec  1.06 GBytes  9.09 Gbits/sec
[  5]   3.01-4.00   sec  1.08 GBytes  9.33 Gbits/sec
[  5]   4.00-5.00   sec  1.08 GBytes  9.31 Gbits/sec
[  5]   5.00-6.00   sec  1.07 GBytes  9.23 Gbits/sec
[  5]   6.00-7.00   sec  1.09 GBytes  9.35 Gbits/sec
[  5]   7.00-8.01   sec  1.08 GBytes  9.20 Gbits/sec
[  5]   8.01-9.00   sec  1.07 GBytes  9.18 Gbits/sec
[  5]   9.00-10.01  sec  1.08 GBytes  9.28 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.01  sec  10.7 GBytes  9.23 Gbits/sec                  sender
[  5]   0.00-10.01  sec  10.7 GBytes  9.23 Gbits/sec                  receiver

Testing 9p mount:

% minikube mount ~/models:/mnt/models
📁  Mounting host path /Users/nir/models into VM as /mnt/models ...
    ▪ Mount type:   9p
    ▪ User ID:      docker
    ▪ Group ID:     docker
    ▪ Version:      9p2000.L
    ▪ Message Size: 262144
    ▪ Options:      map[]
    ▪ Bind Address: 192.168.105.1:50614
🚀  Userspace file server:
ufs starting
✅  Successfully mounted /Users/nir/models to /mnt/models

📌  NOTE: This process must stay alive for the mount to be accessible ...

$ time cat /mnt/models/DeepSeek-R1-0528-Qwen3-8B-Q4_K_M.gguf >/dev/null

real    0m10.219s
user    0m0.007s
sys     0m0.263s

Requires krunkit PR:
containers/krunkit#63

And installing local libkrun-efi from main.

Status

@k8s-ci-robot
Copy link
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Aug 15, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: nirs
Once this PR has been reviewed and has the lgtm label, please assign spowelljr for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot requested review from medyagh and prezha August 15, 2025 17:00
@k8s-ci-robot k8s-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Aug 15, 2025
@nirs nirs force-pushed the krunkit-no-offloading branch from 5d171f8 to dfb2dfd Compare August 21, 2025 22:07
Newer krunkit allows disabling offloading for faster networking. Keep
the Offloading option in case offloading is improved in future versions.

Testing shows 6.7 times faster network performance, and 9p mount
is 3 times faster.

Starting cluster:

    % minikube start --driver krunkit --container-runtime containerd
    😄  minikube v1.36.0 on Darwin 15.6 (arm64)
    ✨  Using the krunkit (experimental) driver based on user configuration
    👍  Starting "minikube" primary control-plane node in "minikube" cluster
    🔥  Creating krunkit VM (CPUs=2, Memory=6144MB, Disk=20000MB) ...
    📦  Preparing Kubernetes v1.33.2 on containerd 1.7.23 ...
    🔗  Configuring bridge CNI (Container Networking Interface) ...
    🔎  Verifying Kubernetes components...
        ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
    🌟  Enabled addons: default-storageclass, storage-provisioner
    🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

Testing iperf3:

    % kubectl apply -f iper3-server.yaml
    deployment.apps/iperf3 created
    service/iperf3 created

    % kubectl get deploy iperf3
    NAME     READY   UP-TO-DATE   AVAILABLE   AGE
    iperf3   1/1     1            1           9s

    % kubectl get service iperf3
    NAME     TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
    iperf3   NodePort   10.105.127.180   <none>        5201:30201/TCP   17s

    % iperf3 -c $(minikube ip) -p 30201
    Connecting to host 192.168.105.10, port 30201
    [  5] local 192.168.105.1 port 50630 connected to 192.168.105.10 port 30201
    [ ID] Interval           Transfer     Bitrate
    [  5]   0.00-1.00   sec  1.05 GBytes  9.03 Gbits/sec
    [  5]   1.00-2.01   sec  1.09 GBytes  9.29 Gbits/sec
    [  5]   2.01-3.01   sec  1.06 GBytes  9.09 Gbits/sec
    [  5]   3.01-4.00   sec  1.08 GBytes  9.33 Gbits/sec
    [  5]   4.00-5.00   sec  1.08 GBytes  9.31 Gbits/sec
    [  5]   5.00-6.00   sec  1.07 GBytes  9.23 Gbits/sec
    [  5]   6.00-7.00   sec  1.09 GBytes  9.35 Gbits/sec
    [  5]   7.00-8.01   sec  1.08 GBytes  9.20 Gbits/sec
    [  5]   8.01-9.00   sec  1.07 GBytes  9.18 Gbits/sec
    [  5]   9.00-10.01  sec  1.08 GBytes  9.28 Gbits/sec
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bitrate
    [  5]   0.00-10.01  sec  10.7 GBytes  9.23 Gbits/sec                  sender
    [  5]   0.00-10.01  sec  10.7 GBytes  9.23 Gbits/sec                  receiver

Testing 9p mount:

    % minikube mount ~/models:/mnt/models
    📁  Mounting host path /Users/nir/models into VM as /mnt/models ...
        ▪ Mount type:   9p
        ▪ User ID:      docker
        ▪ Group ID:     docker
        ▪ Version:      9p2000.L
        ▪ Message Size: 262144
        ▪ Options:      map[]
        ▪ Bind Address: 192.168.105.1:50614
    🚀  Userspace file server:
    ufs starting
    ✅  Successfully mounted /Users/nir/models to /mnt/models

    📌  NOTE: This process must stay alive for the mount to be accessible ...

    $ time cat /mnt/models/DeepSeek-R1-0528-Qwen3-8B-Q4_K_M.gguf >/dev/null

    real    0m10.219s
    user    0m0.007s
    sys     0m0.263s

Requires krunkit PR:
containers/krunkit#63

And installing local libkrun-efi from main.
@nirs nirs force-pushed the krunkit-no-offloading branch from dfb2dfd to 390e6a1 Compare August 24, 2025 14:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants