[seed11]

PR draft → tobi/qmd

Title: llm: resolve cached hf: models from local model cache before network metadata check (fixes indefinite hang when huggingface.co is unreachable)

Body

Problem

LlamaCpp.resolveModel() delegates hf: URIs straight to node-llama-cpp's resolveModelFile(), which performs a network metadata check against huggingface.co before looking at the local model cache. On networks where huggingface.co is unreachable (blocked, with silently dropped packets), the TCP connect has no effective timeout and the call hangs forever at 0% CPU — qmd embed / qmd query / model loading never proceed even though the model file is already present in the model cache directory (manually downloaded or previously cached).

This surfaces as the "embed deadlocks at 0% CPU" family (see #735). On our machine (Apple M4 Pro, macOS 15.6) the deadlock was not Metal-related: with the fix below, a locally built CPU-only backend embeds the full corpus in ~35s, and Metal was never implicated. The same 0%-CPU-idle signature can have different root causes per environment; this patch removes one of them for everyone behind unreachable-HF networks.

Fix

resolveModel(): for hf: URIs, check the model cache directory for a file whose name matches the URI's filename first; if present, return it directly and skip the network resolution entirely. Fall back to the existing resolveModelFile() flow otherwise. (qmd's own pullModels() already does filename-based cache matching — this applies the same idea to load-time resolution.)

Also included: in loadCpuCompatibleLlama(), when the CPU-only prebuilt is missing, try a locally built CPU backend (build: "auto", gpu off) before falling back to the packaged Metal prebuilt — machines with a local CPU build get a true CPU path instead of an unintended Metal load.

Validation

• Apple M4 Pro / macOS 15.6 / node 22 & 26 / qmd 2.8.3: before — qmd embed hung indefinitely (0% CPU, all threads in cond-wait); after — 70 docs / 252 chunks embedded in ~31-35s on CPU, queries 4-13ms, across 10 real-world Chinese colloquial queries recall@3 improved from 0/10 (BM25) to 7/10 (vector).

Notes

• The Metal JIT deadlock described in qmd embed deadlocks at 0% CPU on Apple M5 Max + macOS 26.4 — Metal shader JIT compile failure in bundled llama.cpp b8390 (distinct from #673/#724) #735 (macOS 26.2+) is a separate failure mode; this patch does not address it, but unblocking model resolution makes backend issues easier to diagnose.

---

Issue #735 comment draft（数据点回帖）

Suggested comment:

Data point from a different environment that may help triage: Apple M4 Pro, macOS 15.6 (not 26.x), qmd 2.8.3 — identical symptom (qmd embed hangs at 0% CPU after printing the model line, all threads idle in cond-wait). In our case the root cause turned out not to be Metal: it was resolveModel("hf:…") performing a network metadata check against huggingface.co, which is unreachable from our network and hangs indefinitely without a timeout — even when the model file is already in the local cache. Workarounds that did NOT help here: QMD_FORCE_CPU (no such env in 2.8.3; the correct one is QMD_LLAMA_GPU=false), full Xcode, different node versions, CPU-only local builds — because the hang happened before/independent of backend selection. After short-circuiting hf: resolution to use the local cache file directly (2-line change in LlamaCpp.resolveModel), embed completes fine on a locally built CPU backend (70 docs / 252 chunks in ~35s). Suggestion: consider a fail-fast timeout or cache-first resolution for hf: URIs — it would make the "deadlock" reports easier to triage, since unreachable-HF environments currently mimic the Metal JIT hang exactly.