← Projects

Hawk Point NPU LLM

Running small transformer models directly on first-generation AMD XDNA NPUs with MLIR-AIE/IRON, plus an OpenAI-compatible local API.

Published
July 29, 2026
Updated
August 29, 2026
Status
active
Source
c8dhjp4tyv-bit/hawkpoint-npu-llm ↗

Problem

The project asks how much useful transformer work can actually be moved onto first-generation AMD XDNA NPUs instead of treating the accelerator as a black box.

Target environment

• AMD XDNA1 / Ryzen AI Phoenix and Hawk Point• AIE2 array• MLIR-AIE / IRON• XRT runtime

Architecture

Tokenization and orchestration stay on the CPU. The native path targets the AIE2 array directly, with decoder work such as projections, RMSNorm, RoPE, attention, KV caches, residuals, and SwiGLU implemented as AIE2 kernels.

Inference boundary

  1. Tokenizer + API
  2. CPU orchestration
  3. XRT / IRON
  4. AIE2 decoder kernels
  5. Sampling + stream

Current result

The repository includes terminal chat, selectable checkpoints, hybrid NPU/CPU layer offload, hardware validation tools, and an OpenAI-compatible API with streaming responses.

Validation

On the documented Hawk Point test system, the native path records matching CPU/NPU first-token argmax results and measured decode benchmarks. Exact numbers and test conditions remain in the repository so they can evolve with the implementation without turning this page into a stale benchmark claim.

Limitations

The project deliberately keeps host orchestration visible. A result is only described as NPU-accelerated when the execution path and correctness evidence support that statement.

Stack

  • Python
  • C++ AIE2 kernels
  • MLIR-AIE / IRON
  • XRT
  • AMD XDNA1 / Ryzen AI
  • OpenAI-compatible HTTP API
View the source, measurements, and hardware notes on GitHub.

Related projects

All projects →
  1. Maintainer of the Samsung Galaxy M20 (m20lte, Exynos 7904) for LineageOS 22.2 (Android 15), now an official device with weekly builds: device trees, a 4.4 kernel with CIP updates, proprietary blobs, and a camera HAL patch for the ultra-wide lens.

    Android · LineageOS · Linux Kernel · Exynos · Samsung

    ★ 1 · 25 commitsNov 19, 2026Repository ↗
  2. XDNA NPU Miner

    Experimental

    An early-stage research project exploring whether suitable cryptocurrency mining kernels can be mapped honestly and measurably onto AMD XDNA1 NPUs.

    Python · AIE2 · MLIR-AIE · XRT · Ryzen AI

    ★ 0 · 45 commitsAug 12, 2026Repository ↗