大模型
dev.to ·
@xanpavle
Windows/Linux 上的 AMD 本地 LLM 配置：ROCm 覆盖与 Vulkan vs. HIP 基准测试
文章针对 AMD GPU 本地运行 LLM 的两个常见障碍——需手动设置 HSA_OVERRIDE_GFX_VERSION 以及 Vulkan 与 ROCm/HIP 后端选择——介绍两个工具：ROCmFix 通过 Windows 注册表或 Linux lspci 查询 PCI ID 并自动设置环境变量，附带 doctor 检查；InferBench 自动化测试不同后端的推理速度，包含预热查询和运行间显存卸载以减少缓存偏差。
AI推荐理由 针对 AMD GPU 本地跑 LLM 的具体痛点给出两个工具：ROCmFix 自动处理 HSA_OVERRIDE_GFX_VERSION，InferBench 在 Vulkan 与 ROCm/HIP 后端间做 tok/s 基准测试，并说明预热与显存卸载等测试方法。对本地推理与 AMD 算力用户有实用价值；但为个人开发者工具介绍，缺少第三方验证与完整基准数据。
70
正文
The Problem with Local AI on AMD Hardware
Running models locally with tools like Ollama or LM Studio on AMD GPUs often leads to two roadblocks:
Missing drivers or unrecognised GPU architectures requiring HSA_OVERRIDE_GFX_VERSION manual tweaks.
Uncertainty over whether to run Vulkan backends or ROCm/HIP backends for maximum tokens per second.
Solution 1: ROCmFix
ROCmFix is a single-file Python utility designed to resolve HSA_OVERRIDE_GFX_VERSION issues automatically.
Key Features:
Query PCI IDs directly using Windows Registry or Linux lspci.
Set permanent or session environment variables for CMD, PowerShell, Bash, Zsh, and Fish.
Run rocmfix doctor to inspect installed HIP SDK and Vulkan components.
# Quick run
python rocmfix.py
Solution 2: InferBench
InferBench automates testing local LLM inference speeds across backend engines.
Benchmarking Workflow:
Executes warm-up queries.
Forces VRAM unloads between runs to eliminate memory fragmentation/caching bias.
Calculates median tok/s and TTFT (Time-to-First-Token).
Check out the repositories on GitHub:
ROCmFix Repository
InferBench Repository