The source code for this article is available on GitHub as a gist.

DeepSeek Harness (DSH) was officially released on 2026-08-13. Two revolutionary features offered by DSH set it apart from other harnesses such as Claude Code, Codex or OpenCode.

1. Extensive customization. At the heart of DSH is the Cordis plugin framework following the philosophy, “Everything is a plugin”. Each component of DSH can be independently added, evolved, removed or replaced including core functionality such as the agent loop, taking the plugin-centric approach of the Pi coding agent to the next level
2. Agent observability. Unlike Claude Code, Codex or OpenCode which function as black boxes, DSH provides native capabilities to inspect each and every part of the agent loop including the system prompt, intermediate context injection and tool calling. This makes DSH an excellent developer tool for debugging agent behavior and comparing key performance metrics such as time to first token (TTFT) and tokens per second across different model providers

Follow me through this article to set up DSH on Kubernetes with vLLM-Ascend as the model provider running on the OrangePi AI Studio Pro extension dock. The OrangePi AI Studio Pro features 2x Ascend 310P NPUs providing a combined 192G device memory (VRAM) and 352 TOPS (176 TFLOPS) AI computing power.

Hardware and reference setup

You’ll need the OrangePi AI Studio Pro extension dock to follow through the instructions provided in this article. The officially recommended setup by Orange Pi is to pair the extension dock with the GMKtec NucBox K11 Mini PC. To load larger models such as the Qwen3-30B-A3B-Thinking-2507 smoothly, extend the system memory (RAM) of your Mini PC from the default 32G configuration to a maximum of 96G.

Before we proceed, ensure the drivers and firmware for the extension dock are installed and functional by following the official PDF manual from Orange Pi.

Installing K3s

K3s is a lightweight Kubernetes distribution by SUSE which makes it trivial to set up a single-node Kubernetes cluster. Since this is the only PC running Kubernetes, increase the per-node pod limit to 250 from the default value of 110. This allows us to run more apps and services on the same node.

The command below to install K3s is adapted from the official documentation.

curl -sfL https://get.k3s.io | \
    INSTALL_K3S_EXEC="--kubelet-arg=max-pods=250" sh -s -

Installing and configuring Ascend Docker Runtime

Ascend Docker Runtime simplifies NPU passthrough and driver mounting on Docker and Kubernetes. With it, we can simply configure the environment variables ASCEND_VISIBLE_DEVICES, LD_LIBRARY_PATH within the Docker container or Kubernetes Pod and Ascend Docker Runtime will transparently handle the required device and driver mounts.

Download the Ascend-docker-runtime_{version}_linux-{arch}.run installer for Ascend Docker Runtime version 26.1.0 and mark it as executable. The installer is available for download on GitCode: Ascend/mind-cluster

Now pass the following installation options.

1. --install
2. --install-scene=containerd: informs the installer that our container runtime is containerd (default: Docker)
3. --config-file-path=/var/lib/rancher/k3s/agent/etc/containerd/config.toml: K3s stores the configuration files for its embedded containerd instance under a customized path. This option instructs the installer to edit containerd’s config.toml under the correct path

./Ascend-docker-runtime_{version}_linux-{arch}.run --install \
    --install-scene=containerd \
    --config-file-path=/var/lib/rancher/k3s/agent/etc/containerd/config.toml

Furthermore, in K3s, the config.toml is automatically regenerated from a template file config-v3.toml.tmpl under the same directory every time K3s is restarted. Create or edit the file config-v3.toml.tmpl with the content below. It declares a customized containerd runtime ascend invoking the Ascend Docker Runtime shim under the hood which our pods can use by specifying their runtime class.

{{ template "base" . }}
[plugins."io.containerd.cri.v1.runtime".containerd]
  default_runtime_name = "runc"
[plugins."io.containerd.cri.v1.runtime".containerd.runtimes]
  [plugins."io.containerd.cri.v1.runtime".containerd.runtimes.ascend]
    runtime_type = "io.containerd.runc.v2"
    [plugins."io.containerd.cri.v1.runtime".containerd.runtimes.ascend.options]
      BinaryName = "/usr/local/Ascend/Ascend-Docker-Runtime/ascend-docker-runtime"
      SystemdCgroup = true

Restart the K3s service to take effect.

sudo systemctl restart k3s.service

Next, define our RuntimeClass in Kubernetes and apply it to our cluster.

---
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: ascend
handler: ascend

Test it with the following Pod definition. We set ASCEND_VISIBLE_DEVICES=0 to make the NPU #0 available in our pod, plus a minimal LD_LIBRARY_PATH to identify the mounted Ascend NPU drivers so npu-smi info works inside our pod. Notice the runtimeClassName: ascend – it instructs Kubernetes to run the container under Ascend Docker Runtime instead of the default runc binary.

---
apiVersion: v1
kind: Pod
metadata:
  labels:
    run: ubuntu
  name: ubuntu
spec:
  containers:
    - args:
        - sleep
        - infinity
      env:
        - name: ASCEND_VISIBLE_DEVICES
          value: '0'
        - name: LD_LIBRARY_PATH
          value: '/usr/local/Ascend/driver/lib64:/usr/local/Ascend/driver/lib64/common:/usr/local/dcmi:/usr/local/Ascend/driver/lib64/driver'
      image: ubuntu:latest
      name: ubuntu
      resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
  runtimeClassName: ascend
status: {}

The details are available in my comment to k3s-io/k3s#13553 on GitHub.

Deploying Qwen3-30B-A3B-Thinking-2507 with vLLM-Ascend

vLLM is the industry-leading inference engine for LLMs. It natively supports multi-device and multi-node inferencing which allows it to scale across multiple server nodes within datacenter environments for serving frontier models such as the DeepSeek-V4.x and GLM-5.x series efficiently. Support for Huawei’s Ascend NPUs for vLLM are provided through the vLLM-Ascend plugin which operates as a sub-project under the official vLLM project.

We’ll deploy the Qwen3-30B-A3B-Thinking-2507 Mixture-of-Experts (MoE) model from Hugging Face on both available Ascend NPUs provided by our OrangePi AI Studio Pro extension dock with vLLM-Ascend 0.23.0. Due to the sparse nature of this MoE model, only 3 billion parameters are active at any given time despite the full model being composed of 30 billion parameters. This speeds up model inferencing without compromising on its reasoning abilities which makes it perfect for small to mid-scale local agentic and coding tasks.

Use the container image quay.io/ascend/vllm-ascend:v0.23.0-310p for our vLLM pod and define the following environment variables.

1. ASCEND_VISIBLE_DEVICES='0-1': make both NPUs available inside our Pod. The combined model weights and KV cache for utilizing the full 256K context window supported by Qwen3-30B-A3B-Thinking-2507 occupies about 128G of VRAM which will not fit in a single Ascend 310P NPU
2. VLLM_PORT=8000: vLLM-Ascend serves our model under port 8000/tcp exposing it via an OpenAI-compatible interface. The OpenAI-compatible backend is supported by most agentic coding tools including Codex and DSH

Key options for our vllm serve command include:

1. --tensor-parallel-size=2: utilize both NPUs for inferencing
2. --dtype=float16: the original model weights are in BF16 which can be expanded to full FP32 precision. To ensure our model fits on the OrangePi AI Studio Pro nicely with reasonable performance, we downcast them to FP16 to conserve VRAM and memory bandwidth
3. --enforce-eager: prevents VRAM exhaustion on our extension dock
4. --max-model-len=262144: use the full 256K context window supported by our model
5. --tool-call-parser=hermes: the format used by our model for tool calls
6. --reasoning-parser=deepseek_r1: the format used by our model for reasoning
7. --enable-auto-tool-choice: allow our model to decide when to invoke tool calling based on the existing context
8. --served-model-name=qwen3-30b-a3b-thinking-2507: the name of our model

Below we create the following Kubernetes resources.

1. A PersistentVolumeClaim to persist the model weights across container and pod restarts, instead of re-downloading them every time
2. A Deployment for our vLLM instance using the vLLM-Ascend plugin to serve our model via NPU acceleration
3. A Service to enable clients such as DSH to connect to our served model via the OpenAI-compatible backend

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: hf-cache
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 128Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: vllm
  name: vllm
spec:
  replicas: 1
  selector:
    matchLabels:
      app: vllm
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: vllm
    spec:
      containers:
        - command:
            - sh
            - -c
            - |
              rm -rvf /usr/local/python3.12.13/lib/python3.12/site-packages/triton*
              vllm serve Qwen/Qwen3-30B-A3B-Thinking-2507 \
                  --tensor-parallel-size 2 \
                  --dtype float16 \
                  --enforce-eager \
                  --max-model-len 262144 \
                  --tool-call-parser hermes \
                  --reasoning-parser deepseek_r1 \
                  --enable-auto-tool-choice \
                  --served-model-name qwen3-30b-a3b-thinking-2507
          env:
            - name: ASCEND_VISIBLE_DEVICES
              value: '0-1'
            - name: VLLM_PORT
              value: '8000'
          image: quay.io/ascend/vllm-ascend:v0.23.0-310p
          name: vllm-ascend
          ports:
            - containerPort: 8000
              protocol: TCP
          resources: {}
          volumeMounts:
            - name: hf-cache
              mountPath: /root/.cache/huggingface
      runtimeClassName: ascend
      volumes:
        - name: hf-cache
          persistentVolumeClaim:
            claimName: hf-cache
status: {}
---
apiVersion: v1
kind: Service
metadata:
  name: vllm
spec:
  type: ClusterIP
  selector:
    app: vllm
  ports:
    - protocol: TCP
      port: 8000
      targetPort: 8000

Spin up a ephemeral pod with cURL installed and run the following command to confirm our model is available under the OpenAI-compatible backend. Place it in the same namespace as your vLLM instance.

curl http://vllm:8000/v1/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "qwen3-30b-a3b-thinking-2507",
        "prompt": "Write a function in Python `multiply` accepting 2 numbers `a`, `b` as arguments returning their product. CODE ONLY, NO COMMENTS",
        "max_tokens": 1024,
        "temperature": 0
    }' | jq

The sample output shown below.

Deploying DSH with Helm

DSH provides a web interface listening at port 3080/tcp written in Node.js. Here’s how to run it based on the official installation instructions.

npx @deepseek-ai/dsh web

This means we can containerize it and make it run on Kubernetes! In fact there’s already a Helm chart for it on Github: runzhliu/deepseek-harness-docker

Clone the image-v0.1.2-rc.1-r1 branch of the GitHub repository.

git clone https://github.com/runzhliu/deepseek-harness-docker.git \
    -b image-v0.1.2-rc.1-r1

Now create the file dsh-values.yaml with the following Helm chart values.

1. We declare a custom DSH image quay.io/donaldsebleung/deepseek-harness:0.1.2-rc.1-r1 with the additional system package python3-venv installed. This allows DSH to create dedicated virtual environments (venv) to install Python packages without privileged access – it will come in handy for our agent-driven Python bug-fixing demo later 😉
2. The DSH OpenAI-compatible provider expects an OPENAI_API_KEY. Our vLLM instance requires no authentication so fill in a dummy value such as hunter2
3. Optional: specify a trusted host to expose DSH via ingress, e.g. dsh.internal.donaldsebleung.com
4. Optional: add ingress rules to our network policy to expose DSH via the Traefik ingress pre-bundled with K3s itself

image:
  repository: quay.io/donaldsebleung/deepseek-harness
  tag: 0.1.2-rc.1-r1
extraEnv:
  - name: OPENAI_API_KEY
    value: hunter2
args:
  - web
  - --patch
  - /opt/deepseek-harness/web.cordis.patch.yml
  - --no-open
  - --trusted-host
  - dsh.internal.donaldsebleung.com
networkPolicy:
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
          podSelector:
            matchLabels:
              app.kubernetes.io/name: traefik
      ports:
        - protocol: TCP
          port: 3080

Install it to the same namespace as our vLLM instance with Helm and wait for the installation to succeed.

helm install deepseek-harness \
    deepseek-harness-docker/charts/deepseek-harness \
    -f dsh-values.yaml \
    --wait

Optionally expose it via Ingress and serve it over HTTPS with a custom TLS certificate. My AI homelab already has cert-manager and ExternalDNS installed – here’s the extra resources I created to make DSH available via ingress.

---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: deepseek-harness-tls
spec:
  secretName: deepseek-harness-tls
  duration: 8760h
  renewBefore: 720h
  issuerRef:
    name: root-ca-clusterissuer
    kind: ClusterIssuer
  commonName: dsh.internal.donaldsebleung.com
  dnsNames:
    - dsh.internal.donaldsebleung.com
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cluster-name: donaldsebleung-nucbox-k11
  name: deepseek-harness
spec:
  tls:
    - hosts:
        - 'dsh.internal.donaldsebleung.com'
      secretName: deepseek-harness-tls
  rules:
    - host: 'dsh.internal.donaldsebleung.com'
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: deepseek-harness
                port:
                  number: 3080

Let’s make DSH recognize our model provider running on vLLM-Ascend. Create a file settings.yaml with the following content. We just declare our vllm provider and specify the following connection details.

1. Integration type: openai-completions
2. The (dummy) API key for authentication
3. Fill in the model provider base URL as http://vllm:8000/v1
4. Model name: qwen3-30b-a3b-thinking-2507

llm-pi-ai:
  providers:
    vllm:
      apiKeyEnv: OPENAI_API_KEY
      api: openai-completions
      baseURL: http://vllm:8000/v1
      models:
        - id: qwen3-30b-a3b-thinking-2507

Copy the settings.yaml to our pod under $DSH_HOME/settings.yaml to make it available to DSH. DSH hot-reloads its configuration so no pod restart is required.

kubectl cp settings.yaml deepseek-harness-0:/home/node/.dsh/settings.yaml

If you haven’t configured ingress, run a kubectl port-forward to make the web UI available locally and keep the terminal window open.

kubectl port-forward svc/deepseek-harness 3080:3080 6080:6080

In a new terminal tab or window, get the login token for DSH from the pod logs.

kubectl logs sts/deepseek-harness

Here’s what you should see.

dsh web: http://127.0.0.1:3080/?token=XXXX (LAN: http://10.42.0.136:3080/?token=XXXX)

Open the provided link in your browser. The DSH web UI appears.

Running an agent-driven Python debugging workflow with DSH

With DSH installed and connected to our vLLM-Ascend model provider, let’s test our AI agent on a sample Python codebase, see if it can identify the bug and fix it: DonaldKellett/coding-plan-litmus-test

First we need to create a workspace. Find the “Choose workspace” button near the chat window, accept the default workspace directory and click “Open”.

Next, to the bottom right of the chat window, expand the dropdown menu to choose your model provider and select the qwen3-30b-a3b-thinking-2507 model under the vllm provider.

Now type in the following prompt and press Enter.

Clone the GitHub repository DonaldKellett/coding-plan-litmus-test and make it your working directory. The tests and CI are passing but the program doesn’t work. Find the root cause of the issue and fix it. Use the diff command to show me the changes you made.

Watch our AI agent switch between thinking, tool calling and producing intermediate output before arriving at the solution in around 5 minutes. Also notice the metrics displayed near the bottom of the browser window: our time to first token (TTFT) is under 0.5 seconds with an output generation speed of around 24 tokens per second. Not bad for a local AI model running on edge-optimized Ascend 310P NPUs!

Near the top left of the browser window, click on the “Trajectory” tab next to “Chat”. This is where DSH really shines: unlike proprietary harnesses such as Claude Code which function as a black box, DSH provides you complete visibility to each step the agent takes via an immutable append-only audit trail. This includes when and where the system prompt is injected, additional context injection such as a project-wide AGENTS.md file and which tools our agent called, etc.

Congratulations, you’ve successfully deployed a local AI model on the OrangePi AI Studio Pro extension dock with vLLM-Ascend and ran your first agent-driven debugging session with DSH! 😃

Demo animation and screencast

View a GIF animation of running an agent-driven Python debugging session with DSH using our vLLM-Ascend model provider below. Alternatively, the original MP4 recording is also available.

Concluding remarks and going further

vLLM-Ascend enables us to serve LLMs leveraging Huawei Ascend NPUs for hardware-accelerated model inferencing. While our example involves both Ascend 310P NPUs connected to a single consumer-grade PC, vLLM-Ascend effortlessly scales across multiple datacenter NPUs and server nodes with Kubernetes. With models such as the Qwen3-30B-A3B-Thinking-2507 which support both reasoning and tool calling, vLLM-Ascend exposes these capabilities over an OpenAI-compatible API endpoint for seamless integration with agent frameworks and harnesses such as Codex and DSH to empower a wide range of AI-driven workflows, including but not limited to agent-driven software development.

I hope you enjoyed reading this article as much as I did authoring it and stay tuned for updates! 😉