Category: Model Deployment and Inferencing
-
This Kaggle Notebook is a GPU-enabled variation of my original notebook experiment. Open Neural Network Exchange (ONNX) is an open format built to represent machine learning models. It is used by popular deep learning frameworks such as MindSpore, PyTorch and TensorFlow to export machine learning models trained by these frameworks into the portable, unified and vendor-neutral ONNX format. This…
-
The notebook source code for this experiment is available on GitHub. Open Neural Network Exchange (ONNX) is an open format built to represent machine learning models. It is used by popular deep learning frameworks such as MindSpore, PyTorch and TensorFlow to export machine learning models trained by these frameworks into the portable, unified and vendor-neutral ONNX format.…
-
Ray is an open source framework for scaling Python applications through distributed computing. The framework focuses on scaling AI/ML training and inference pipelines horizontally through distributed resource allocation and execution, though it can be used in other Python projects as well. Since October 2025, Ray joined the PyTorch Foundation alongside leading open source…
-
vLLM is a model serving runtime supporting the most popular families of large language models (LLMs) such as DeepSeek and Llama. It exposes an OpenAI-compatible server implementing common OpenAI REST API endpoints such as the Chat, Completions and Responses API for text generation and inference, allowing existing clients and tools leveraging the OpenAI API to be…
Donald Sebastian Leung 