Donald Sebastian Leung

Infrastructure | Ops

Send me an email

  • Fine-tuning ResNet-18 on CIFAR-100 with MindCV and MindSpore

    +

    +

    +

    +

    +

    +

    The notebook source code for this article is available on GitHub.

    Transfer learning is the practice of adapting a pre-trained model for a new but related task. An example of transfer learning is low-rank adaptation (LoRA), a common technique in LLM fine-tuning.

    In this notebook experiment, let’s load ResNet-18 with MindCV for fine-tuning on the CIFAR-100 dataset. ResNet-18 is a modern CNN pre-trained on ImageNet which makes it suitable for our task as both datasets represent image classification tasks. Furthermore, ImageNet contains far more samples and categories than CIFAR-100, meaning the pre-trained filters from ImageNet should translate well to CIFAR-100 with minor adjustments.

    (more…)

    +

    +

    +

    +

    +

    +

    + , , , , , ,

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

  • Upgrading to MindSpore 2.9.0 + CANN 9.0.0 on the OrangePi AIpro (20T)

    +

    +

    +

    +

    +

    +

    The notebook source code for this article is available on GitHub.

    Last time we saw how to upgrade the MindSpore 2.4.10 + CANN 8.0.0 pre-included with the Ubuntu 22.04 desktop image for OrangePi AIpro (20T) to MindSpore 2.8.0 + CANN 8.5.0.

    MindSpore 2.9.0 + CANN 9.0.0 were released on May 2026 with the following major highlights, among others.

    1. Native support for Ascend 950PR inference-optimized NPUs which the DeepSeek-V4 series LLMs were optimized on
    2. Addition of 2 DeepSeek Sparse Attention (DSA) interfaces under mindspore.ops:
      1. mindspore.ops.sparse_flash_attention
      2. mindspore.ops.lightning_indexer

    This notebook article outlines the process of upgrading from MindSpore 2.8.0 + CANN 8.5.0 on the OrangePi AIpro (20T) development board to the latest MindSpore 2.9.0 + CANN 9.0.0 as of May 2026. We assume you have already replaced Conda from the provided Ubuntu 22.04 desktop image with pyenv for modern, efficient Python virtual environment management.

    (more…)

    +

    +

    +

    +

    +

    +

    + , , , ,

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

  • Migrating PyTorch deep learning workloads to the Ascend ecosystem

    +

    +

    +

    +

    +

    +

    The notebook source code for this article is available on GitHub.

    PyTorch is an open source deep learning framework in Python originally developed by Meta, now hosted under the vendor-neutral PyTorch Foundation. It is the de-facto industry standard for training modern deep learning models and large language models (LLMs). Like most other deep learning frameworks, PyTorch natively supports accelerating machine learning workloads on NVIDIA GPUs through its CUDA ecosystem without the need for special plugins or adapters.

    (more…)

    +

    +

    +

    +

    +

    +

    + , , , , , ,

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

  • Accelerating model inference on NVIDIA Tesla T4 with ONNX Runtime

    +

    +

    +

    +

    +

    +

    This Kaggle Notebook is a GPU-enabled variation of my original notebook experiment.

    Open Neural Network Exchange (ONNX) is an open format built to represent machine learning models. It is used by popular deep learning frameworks such as MindSpore, PyTorch and TensorFlow to export machine learning models trained by these frameworks into the portable, unified and vendor-neutral ONNX format. This allows exported models to be loaded by any compatible framework or runtime for inference instead of being locked in to a single framework or provider.

    In this notebook experiment, we will load an ONNX model trained on the CIFAR-10 dataset mounted at /kaggle/input/models/donaldsebleung/cifar10-linear-simple/onnx/default/1/cifar10-linear-simple.onnx and serve it with ONNX Runtime for inference. ONNX Runtime is a cross-platform inference accelerator for ONNX models developed by Microsoft under the permissive MIT license.

    (more…)

    +

    +

    +

    +

    +

    +

    + , ,

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

  • Serving machine learning models for inference with ONNX Runtime

    +

    +

    +

    +

    +

    +

    The notebook source code for this experiment is available on GitHub.

    Open Neural Network Exchange (ONNX) is an open format built to represent machine learning models. It is used by popular deep learning frameworks such as MindSpore, PyTorch and TensorFlow to export machine learning models trained by these frameworks into the portable, unified and vendor-neutral ONNX format. This allows exported models to be loaded by any compatible framework or runtime for inference instead of being locked in to a single framework or provider.

    In this notebook experiment, we will download an ONNX model trained on the CIFAR-10 dataset from Hugging Face and serve it with ONNX Runtime for inference. ONNX Runtime is a cross-platform inference accelerator for ONNX models developed by Microsoft under the permissive MIT license.

    (more…)

    +

    +

    +

    +

    +

    +

    + , ,

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

  • Predicting California house prices with MindSpore

    +

    +

    +

    +

    +

    +

    The notebook source code for this article is available on: GitCode, GitHub

    MindSpore is an open source deep learning framework lead by Huawei and the MindSpore community. It is optimized for performing model training and inference tasks on Huawei’s Ascend series processors (NPUs), though it supports NVIDIA GPUs and CPU-only environments as well. It provides a compelling alternative to PyTorch, the leading open source deep learning framework under the Linux Foundation. Furthermore, it provides utility classes and functions compatible with a subset of PyTorch to ease the migration of training and inference pipelines from PyTorch to MindSpore, which is out of scope for this article.

    In this lab, we will use MindSpore for training a simple linear regression model consisting of a single fully connected layer to predict house prices in California. The dataset we will be using is the California housing dataset which we’ll fetch using scikit-learn’s sklearn.datasets.fetch_california_housing function.

    (more…)

    +

    +

    +

    +

    +

    +

    + , , , , , ,

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

  • Exploring the Ascend ecosystem with OrangePi AIpro (20T)

    +

    +

    +

    +

    +

    +

    OrangePi AIpro (20T) is a development board under the Orange Pi brand owned by Xunlong Software and released in 2024. It features 4 CPU cores plus a neural processing unit (NPU) based on Huawei’s Ascend technology capable of running modern AI/ML workloads. The embedded NPU is capable of performing 20 TOPS or trillion operations per second.

    I purchased the OrangePi AIpro (20T) in January 2026 to explore the emerging field of LLM deployment, inferencing and fine-tuning as an infrastructure and platform engineer, as well as exploring the Ascend CANN ecosystem which is swiftly thriving and positioning itself as a strategic alternative to NVIDIA’s CUDA ecosystem in 2026.

    Follow me as I unpack and set up my OrangePi AIpro (20T) development board for exploring AI/ML applications and use cases.

    (more…)

    +

    +

    +

    +

    +

    +

    + , , , ,

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

  • Deploying DeepSeek-R1 for distributed inferencing with Ray

    +

    +

    +

    +

    +

    +

    Ray is an open source framework for scaling Python applications through distributed computing. The framework focuses on scaling AI/ML training and inference pipelines horizontally through distributed resource allocation and execution, though it can be used in other Python projects as well.

    Since October 2025, Ray joined the PyTorch Foundation alongside leading open source AI/ML projects such as PyTorch and vLLM, ensuring its sustained development in the AI/ML ecosystem and vendor-neutral governance.

    Follow me as I deploy a distilled variant of DeepSeek-R1 across 2 GPU-enabled cloud servers on Huawei Cloud with vLLM and Ray. Leveraging Ray as the framework for distributed inference enables us to scale horizontally when needed and increase the number of concurrent requests our model can handle compared to a single server.

    (more…)

    +

    +

    +

    +

    +

    +

    + , ,

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

  • Running DeepSeek-R1 on a commodity laptop with vLLM

    +

    +

    +

    +

    +

    +

    vLLM is a model serving runtime supporting the most popular families of large language models (LLMs) such as DeepSeek and Llama. It exposes an OpenAI-compatible server implementing common OpenAI REST API endpoints such as the Chat, Completions and Responses API for text generation and inference, allowing existing clients and tools leveraging the OpenAI API to be reused simply by customizing the server URL. One of the main innovations of vLLM at the time it was initially released to the public is its paged attention mechanism addressing the critical bottleneck of key-value (KV) cache memory through its non-contiguous cache implementation, enabling significant efficiency and throughput gains in tasks such as text generation and inference.

    vLLM is an incubation project under LF AI & Data since October 2024 ensuring vendor-neutral governance.

    Follow me as I deploy my first locally hosted distilled variant of the DeepSeek-R1 model on a commodity Redmi Book 14 2024 laptop with vLLM in this exploratory lab. DeepSeek-R1 is a series of models trained via reinforcement learning available on Hugging Face.

    (more…)

    +

    +

    +

    +

    +

    +

    + ,

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

  • Exploring the developer ecosystem on Huawei MateBook Pro

    +

    +

    +

    +

    +

    +

    Back in mid-2025, I stumbled upon the MateBook Pro in a Huawei flagship store in Shenzhen. The laptop caught my attention since it came pre-installed with Huawei’s very own HarmonyOS 5 operating system, otherwise known as HarmonyOS NEXT.

    Unlike previous HarmonyOS releases such as HarmonyOS 4 which were based on Android, specifically the Android Open Source Project (AOSP), HarmonyOS NEXT features a fully independent software stack ranging from Huawei’s proprietary HongMeng kernel, to the Kernel Abstraction Layer (KAL) and userspace based on the OpenHarmony project and HarmonyOS-native applications. It’s not common to find a laptop or PC in the wild which is pre-loaded with an operating system distinct from Windows or macOS so I decided to purchase it and give it a try.

    (more…)

    +

    +

    +

    +

    +

    +

    + ,

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +