Strata v0.1.40 发布：新增 Strix Halo（gfx1151）实验性 HIP 支持与多项修复

Official release notes (https://github.com/Niko1221/Strata/releases/tag/v0.1.40) **Strix Halo (gfx1151) support, crash, NaN and security fixes, better multi-GPU, batching and resident modes, Q4_0 / Q4_1 and more PLE table formats, AMD fixes, a much faster Linux cold start, and a long list of opt-ins.** **Strix Halo / Ryzen AI Max (gfx1151, experimental):** the HIP engine now runs on the Radeon 8060S / 8050S, and setup recognizes the chip (PCI 1002:1586), sizes memory for it as one shared pool, recommends UD-IQ4_XS, and on Linux picks HIP by itself. The engine has RDNA3.5 matrix-core kernels for the prompt, a hipBLASLt table for ROCm 7.14.1, free-memory accounting for unified-memory chips (#409, a DGX Spark gets it too) and 18 speed switches that turn on by themselves on gfx1151 (`STRATA_GFX1151_DEFAULTS=0` turns them off). All of them gave byte-identical output here. The Windows HIP zip now includes gfx1151 code. It is experimental, and we have not run it on Windows Strix Halo hardware. docs/STRIX_HALO.md has the toolchain (no root needed), a manual build and the opt-in prompt switches. The 8040S mapping in setup is a guess. Measured on a Ryzen AI Max+ 395 with 128 GB, UD-IQ4_XS, medians of interleaved runs: | Context | Prompt tok/s | Output tok/s | |---|---|---| | 8K | 1,293 | 53.8 | | 64K | 1,370 | 46.6 | | 128K | 1,320 | 51.4 | That is +6% prompt and +4% output over our earlier engine branch. UD-Q4_K_XL, IQ3_S and IQ3_XXS are in the doc. The table matches the final defaults, which keep the shared-expert stream fork on for gfx1151 (it is the 18th automatic switch): UD-IQ4_XS at 8K gave 54.07 output against 53.94 in the table. Thanks to StevenChenSE for the RDNA3 matrix-core GEMM kernels (#313). **Crash and NaN fixes:** - **Non-finite values (#838, #550, #871, #606, #879):** the fused SwiGLU q8_1 quantizers keep their scale finite. The residency-table upload waits for its own copy before the verifier reads it. The all-resident verify graph runs only while every expert is in VRAM, and a stale plan fails the window instead of printing `!!!!`. A Stop sent during a prompt read reaches the engine at once. We could not reproduce #879 here, so it stays open for a retry on 0.1.40. - **`--batch` on a split (#776, #845):** a fully resident split stage no longer waits for doorbells that are never rung. - **A dead engine (#748):** a fatal verify timeout is caught before batch routing, and the next request is refused. - **Parking a conversation (#752):** a host out-of-memory skips the park instead of ending the server. - **HIP:** the shared-expert stream fork is off by default on AMD, which brings decode back to 0.1.38 speed (#826, #816). The doorbell kernels fence after the ring store (#697), which may explain the batched-prompt stalls in #579 and #613. - **Windows:** the RAM budget respects the commit limit (#749, #730), a large-page arena is no longer VirtualLocked (#779), and Smart App Control blocks get a clear message (#735). **Security:** - **#553:** network image paths are refused, URLs are capped at 32 MiB, and a page from another origin cannot name a local file as a picture. We added the same check to `/v1/responses`, which the PR did not cover. - We did not merge #434 (a Manager page that any website can drive, plus a proxy that adds the API key) or #696 (the API key becomes a shell on a LAN server). The replies say what each needs. **More than one GPU, batching and resident modes:** - **Resident RAM mode on a layer split (#848, thanks to Francesco Albano):** about 70 tok/s against 32-64 on 32 GB machines. It also fixes a 0.1.39 bug where the adaptive swaps copied back from the wrong card's cache. Setup keeps the resident variant on a split only with an engine that has it, so setup now needs engine 0.1.40. - **A helper GPU's experts are not in the PCIe share (#854):** 40 -> 66 tok/s on a helper rig, the author's number. - **Four GPUs (#887):** the layer split scores every four-way placement instead of guessing. This changes the automatic placement on 4 GPUs. 2 and 3 GPUs keep 0.1.39's, and `STRATA_SPLIT_COVER_B` asks for the new one at any count. - **Stage weights (#880):** `--layer-split auto` can trim the later stages' weights (4 GPUs, prompt 470 -> 1,930 tok/s, the author's number). `--vram-reserve-later-mib` sets a separate reserve for the later cards. - **Batching:** `--batch-mtp` keeps MTP drafting in concurrent slots (#846, +31 to +39% with 2-4 clients, the author's number). `serve` runs without `--mtp` (#708). - **Fusions ported from Eddoursul's fork:** the draft round runs the draft layer's front once, a one-token window commits itself, the draft kernels use the main layer's forms, the final mixer is one fused read, and the argmax runs over many blocks. `--host-core first|last` and the `--spec-follow` / `--window-hashes` test tools come from the same fork. The #783 kernel work (QSA early exit, GDN decode kernels, vec4 router, batched KV append, multi-token GR) is merged with a kill switch for each. Its long-context QSA early exit gives +2.7% decode at 120K (6 pairs). `STRATA_GR_DOWN_MAX4` stays opt-in, because it gave no gain on the 5070. - **Decode against 0.1.39 on NVIDIA (RTX 5070, 12 pairs, 200-token medians):** Q2_0 code -0.3%, story +3.4%; IQ3_XXS code +2.9%, story +2.3%. Prompts are within 2%. **New formats:** - **Q4_0 and Q4_1 experts (#599)** on the GPU kernels, and a Q4_0 PLE table. - **PLE tables in Q8_0, Q5_1 and BF16 (#651, #586, #865):** one table of PLE formats now covers these and Q5_0 and FP8, with bounds checks on the file. Against the BF16 table, Q8_0 is off by 0.53%, FP8 by 2.64% and IQ4_NL by 7.60%. The effect on answers is small (KL about 0.012 for any of them), so setup does not offer a bigger table. `--ple-io ram` faults the table in on 16 threads (8 GB cold in WSL: 15 s -> 6 s). - **Unsloth UD-Q6_K_XL:** the Q8_0 PLE table is read, and the Q6_K expert kernels are an opt-in build (`-DSTRATA_Q6K_EXPERTS=ON`, #860). - **IQ3_XXS and IQ4_XS PLE keys** stay native, so those packs start again (#381). - **`--kv k8v4` streams with `--kv-resident`** (#711, #705), and setup offers it. **AMD:** - **RDNA2 (#835, opt-in):** `STRATA_HIP_PROMPT_F16=1` runs the prompt's 16-bit GEMMs in FP16, and prompts read about 2x faster on an RX 6900 XT. It rounds differently, so it is off for now. The engine prints a tip on gfx103x. - **hipBLASLt tables for gfx1100 (#766, #755, #565)**, RDNA3 / RDNA4 sibling arch names (#695), gfx1034 (#778) and a gfx906 build fix (#808). - **Windows HIP (#873, #654):** one `hipFree(0)` warm-up before the first memory query. - **Opt-in:** `STRATA_HIP_ADAPT_KERNEL_COPY=1` does the adaptive swaps' copies with a kernel instead of SDMA (#884, for the 2x gfx1030 hang). `STRATA_DENSE_MMQ=1` (#820) gave no gain on gfx1151 and stays opt-in. - **Older cards:** the Q4_0 expert kernel overflowed on sm_60 (signed overflow). It is fixed, found on a new P100 test box. - **Intel Arc (#784, #866, #868, #870):** the SYCL port compiles against 0.1.39's sources again, a ring wait over 2 ms is no longer reported as finished, and the Arc Pro B60 ids are known. **Linux cold start:** weights, head, embedding, vision encoder and the RAM copy are now read ahead of use (#699). On an RTX 5090 with 32 GB, Q2_0 at 262K, ready in 70 s instead of about 920 s. `STRATA_READ_AHEAD=0` turns it off. Linux also gets the unbuffered file tier (#773, O_DIRECT, same rule as Windows). On a 16 GB laptop with 30 GB RAM, decode went 4.4 -> 7.9 tok/s and startup 36 -> about 20 s (author's numbers). With transparent huge pages on `always` the arena no longer asks for more (#771, `STRATA_NO_ARENA_THP=1`). **Defaults that changed (the rest of the default output is unchanged):** - **Empty assistant turns (#886):** they are not rendered into the next prompt, so a client's history can't teach the model to answer with nothing. `STRATA_KEEP_EMPTY_TURNS=1` restores the old rendering. - **`tool_choice` (#790):** `none` offers no tools on both routes. `required` and a named function work. - **`json_object` mode (#762):** JSON is taken out of a code fence or the prose around it. - **Stop strings (#454):** OpenAI `stop` and Anthropic `stop_sequences` are honoured on every path. - **Workers on hybrid CPUs (#798):** P and E cores come from the CPU's own lists. A Core Ultra 7 270K Plus is now 8P + 16E, 15 workers. - **Windows (#691):** the server is opted out of power throttling, so a minimized window no longer moves the engine to the E-cores. - **Conversation caches from 0.1.39 are rebuilt on first use:** the cache's version key changed, so the first request after the update reads its prompt again. - Setup's context menu has a 200K step (#608). It is never the recommended one. **Opt-ins (off by default, the output is unchanged):** - Server: `"strata_checkpoint": false` for a one-shot request (#861), `"return_progress": true` (#837), `"reasoning_loop_recovery": "stop" | "recover"` (#869), session files with `--slot-save-path` (#668), `--prompt-cache-tail` (#614), `STRATA_CACHE_MESSAGE_BOUNDARY=1` (#734). - Memory: `--kv-grow`, the K/V takes VRAM as the context grows (#378). - Speed: `STRATA_Q2_BITPLANE=1` (#706), `STRATA_PREFILL_EQUAL=1` (#693), `STRATA_KV_PREFETCH=1` (#732, 4-5% slower on our 5070, so off), `STRATA_IQ3S_MT1=1` (#863), `STRATA_DMA_BATCH=1|2` (#807), `STRATA_ADAPT_LAG=2` (#764). - Multi-GPU: `--adapt-async 1` (#876), `STRATA_EXCHANGE_ROTATE=1` (#864), `--pipeline-windows` (#859), `STRATA_DISJOINT_ADAPT=1` (#731). - Tools: `python setup.py --rollback-engine` puts the previous engine back (#670), and `strata_pack build --force` rebuilds into a used folder (#634). - Turing prompts above about 90K cells take the wide top-k kernel (#743, `STRATA_TOPK_STREAM=0` restores the old one). - `STRATA_DEFERRED_REGISTER=1` (Windows, #285 part 2) registers the arena per layer on a thread ahead of the readers. It cost about 8% decode here, so it is off. **Testers wanted** (we have one RTX 5070 and no second card): - **2 GPUs:** a 30-minute soak of the resident RAM mode on a split (#848, #642), `--batch` on an all-resident stage (#776), and `--pipeline-windows` with 10 or more interleaved pairs, with and without #848 (#859). - **4 GPUs:** the new automatic placement (#887) and `--layer-split auto` with the trim (#880). Tell us if the auto layout got worse. - **Turing (RTX 20):** a prompt at 90K, 131K and 155K, before and after (#743). - **Intel Arc:** the SYCL port on an A- or B-series card (#867, #870). - **Older cards:** Pascal and Volta owners with a result from the CUDA 12 engine. We now have a P100 for sm_60, but not Volta. - **Windows AMD:** the HIP zip's gfx1151 code on a Strix Halo PC, `STRATA_HIP_PROMPT_F16=1` on gfx103x (#835), the SDMA kernel copy on 2x gfx1030 (#884) and the HIP warm-up (#654). - Open reports where a retry helps: #879 (both reporters), #871 at 100% residency, #828 (`MALLOC_CHECK_=3`) and #795 (BIOS microcode update). Thanks to everyone who sent PRs, tests and reports. Special thanks to Eddoursul, whose fork shaped this release, to Francesco Albano for #848, to sergiywith for #838 and #550, and to StevenChenSE for the Strix Halo GEMM work. **Checked before the release:** - **The same answers as 0.1.39 on all four quants** (Q2_0, IQ3_XXS, IQ3_S, the Coder), 10/10 each with a fixed cache, on the final tip. The prompt path's internal state is identical (exact_pp). A Tesla P100 (sm_60) gave 10/10 on 3 quants. - **Speed:** the decode numbers above, interleaved against 0.1.39 on the RTX 5070. - **Strix Halo:** output is byte-identical to the earlier branch. - **AMD and Intel:** the HIP engine compiles, and the Windows HIP zip builds with gfx1151 (598 MiB). Nothing here ran on an AMD or Arc card except gfx1151 on Linux. **Updating:** run `UPDATE.bat` (Linux: `./update.sh`). Setup installs engine 0.1.40. The ready-made Strata engines for Windows, which `START-HERE.bat` fetches by itself: - `strata-windows-x64.zip`: NVIDIA (RTX 20 / 30 / 40 / 50: sm_75, sm_86, sm_89, sm_120 + PTX), CUDA 13.0, needs an NVIDIA driver 580 or newer. Contents: `strata.exe`, `strata-vision.exe` (the optional image encoder), `BUILD.json`. - `strata-windows-x64-cuda12.zip` (**experimental**): older NVIDIA cards (Pascal and Volta: sm_60, sm_61, sm_70; it also has sm_75, sm_80, sm_86, sm_89 + PTX for a mixed PC), CUDA 12.9, needs an NVIDIA driver 528 or newer. Setup fetches it only for a model you put on such a card, or with `--cuda 12`. Contents: `strata.exe`, `strata-vision.exe`, `BUILD.json`. - `strata-windows-x64-hip.zip`: AMD (gfx1100, gfx1101, gfx1102, gfx1200, gfx1201, gfx1030, gfx1151), ROCm 10.2.0a20260930 from AMD's TheRock builds, needs a current AMD driver. gfx1151 (Strix Halo) is in this zip too, experimental and untested on Windows hardware. Contents: `strata.exe`, `strata-device.exe`, the HIP runtime next to them, `BUILD.json`, `rocm\` (the ROCm libraries and their licenses). --- Strata is free and open source. If it runs well on your PC, a coffee keeps the work on it going: Repository README (current repository documentation, not release changes) Strata **English** · [简体中文](README.zh-CN.md) · [日本語](README.ja.md) · [Deutsch](README.de.md) · [Français](README.fr.md) · [Español](README.es.md) · [Português](README.pt-BR.md) Run a 125-billion-parameter AI model on your own gaming PC NVIDIA or AMD graphics card (12 GB or more) · Windows or Linux · free and open source A voxel pagoda garden, 1 shot prompt running on an RTX 5070 with Strata (IQ3_S, 128K context) · full video (49 s) Strata runs **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)** on a normal PC. This is a large, smart AI model that usually needs a server. It chats, writes code, reads pictures and works with your apps and coding agents. Nothing leaves your PC. ## How fast is it? We measured it on two ordinary gaming PCs. A token is about ¾ of a word. - **Writes answers:** how fast the reply appears in a short chat. 60 tokens per second is faster than you can read. - **Reads your prompt:** how fast it takes in what you send (here a 32K-token document, code or chat history). NVIDIA: RTX 5070 (12 GB), Ryzen 5 7600, 64 GB RAMAMD: RX 9070 XT (16 GB), Ryzen 9 3900X, 47 GB RAM | Size | Writes answers | Reads your prompt | | --- | ---: | ---: | | **Q2_0** | 94 tokens/s | 2,650 tokens/s | | **IQ2_XS** | 79 tokens/s | 2,090 tokens/s | | **IQ3_XXS** | 62 tokens/s | 1,750 tokens/s | | **IQ3_S** | 53 tokens/s | 1,620 tokens/s | | **Coder** | 55 tokens/s | 2,180 tokens/s | | Size | Writes answers | Reads your prompt | | --- | ---: | ---: | | **Q2_0** | 60 tokens/s | 1,160 tokens/s | | **IQ2_XS** | 52 tokens/s | 1,110 tokens/s | | **Coder** | 44 tokens/s | 1,420 tokens/s | NVIDIA: Q2_0 with engine 0.1.36, the other rows with 0.1.26 (4K answers, 32K prompts). The full tables are in [DETAILS.md](docs/DETAILS.md#speed-measured). A card with more VRAM is faster: an RTX 3090 (24 GB) should write about 100-140 tokens per second. Long chats and other cards: [speed of each model](docs/MODELS.md#how-fast-is-each-size), [community results](docs/COMMUNITY_BENCHMARKS.md). Strata is free. If it runs well on your PC, a coffee keeps the work on it going. ## What you need | | | | --- | --- | | **Graphics card** | **NVIDIA** GeForce RTX 20, 30, 40 or 50 series, or **AMD** Radeon RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT, RX 9070 / 9070 XT, Radeon AI PRO R9700 or RX 6800 / 6900 series. It needs **12 GB of VRAM or more**. | | **RAM** | 32 GB or more. Your RAM decides [which model](#which-model-should-i-pick) fits. 64 GB runs every size. | | **Disk** | About 80 GB free. Use an SSD if you can: the first start is much faster. | | **System** | Windows 10 / 11 or Linux, and a current graphics driver from NVIDIA or AMD. | The installer sets up everything else. Two or three cards can share the model ([multi-GPU](docs/MULTI_GPU.md)). Experimental, written and tested by community members on their own machines: - **Older graphics cards** (Tesla P40 / V100, GTX 10, Radeon VII / MI50, RX 6700 XT, RX 5500 XT): [Older GPUs](docs/OLDER_GPUS.md). - **Intel Arc**, built from source on Linux: [Intel Arc](docs/INTEL_ARC.md). - **AMD Ryzen AI Max (Strix Halo)**, built from source on Linux: [Strix Halo](docs/STRIX_HALO.md). - **Older processors without AVX2**: they work, but slowly. [Older CPUs](docs/INSTALL.md#older-cpus-experimental). The full list: [docs/INSTALL.md](docs/INSTALL.md#what-you-need). ## Install ### Let your AI set it up Do you use an AI coding assistant (Claude Code, Cursor, Codex, GitHub Copilot, ...)? Paste this into it: ```text Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository. ``` It checks your graphics card, RAM and disk and picks the model that fits. Then it installs and starts it and tells you how to connect your apps. AI tools can also install, start and stop Strata through its [MCP server](docs/MCP_SERVER.md). ### Or do it yourself [Download Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) and unzip it (or `git clone` it). **Windows:** double-click **`START-HERE.bat`**. **Linux:** run **`./setup.sh`** in the Strata folder. The steps are the same for NVIDIA and AMD. The installer finds your card and sets up the right engine for it. It asks you a few questions: - which model and which size, - how much context (how much text the model keeps in mind), - whether it should read pictures. Press Enter each time for the recommended answer. Then it downloads the model (about 70 GB) and starts it. If the download stops, run it again: it continues where it left off. Your browser opens the Strata app at `http://127.0.0.1:8080`. > **While the model starts, your PC can be slow or stop responding for 1-3 minutes** (longest the first time). > Strata loads 35-55 GB into your RAM and locks part of it for the graphics card. This is normal. Wait, and don't > close the window. The window shows what Strata is doing. **Next time**, run `START-HERE.bat` (or `./setup.sh`) again. It starts right away and downloads nothing twice. Close its window to stop the model. `UPDATE.bat` (`./update.sh`) updates Strata without starting it. Updating, Docker, several cards, where the files go and every option: [docs/INSTALL.md](docs/INSTALL.md). ## Which model should I pick? The installer recommends one for your RAM. The same model comes in several sizes, compressed more or less. Smaller sizes are faster. Larger sizes are a bit smarter. | Your RAM | Take | Why | | --- | --- | --- | | **32 GB** | **Coder** | it fits 32 GB, and it is made for code (with a 24 GB card, Q2_0 and IQ2_XS run too) | | **48 GB** | **IQ2_XS** (or Q2_0, the fastest) | the larger sizes do not fit | | **64 GB** | **IQ2_XS** (recommended), or IQ3_XXS / IQ3_S | every size fits; IQ3_S is the best and the slowest | | **96 GB or more** | **IQ3_S**, or Unsloth's UD-IQ4_XS (~4-bit) | room for the largest sizes with everything else open | - **[Coder](docs/MODELS.md#coder):** a coding version with half of the experts removed. It reaches 91% of the full model's SWE-bench Verified score (measured by its authors) and fits 32 GB of RAM. It is weaker outside code, including Chinese and other CJK text (#438). For those, take Q2_0, IQ2_XS or IQ3_S, which keep every expert. - **[Swift 1.5](docs/MODELS.md#swift-15):** a fine-tune that thinks for a much shorter time before it answers. You get the answer sooner, at about the same quality. - **[Unsloth UD-IQ4_XS](docs/MODELS.md#unsloth-ud-iq4_xs):** Unsloth's ~4-bit version, between IQ3_S and UD-Q4_K_XL in quality. A 94 GB download. With less than ~80 GB of RAM, Strata reads part of it from the SSD while it answers, so it is slower there (an NVMe SSD helps). - **[Unsloth UD-Q4_K_XL](docs/MODELS.md#unsloth-ud-q4_k_xl-experimental)** (experimental): the closest to the full model. But Strata reads most of it from the SSD while it answers, so it writes only 7-8.5 tokens/s on a 64 GB PC. - **[OrcaRouter's Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs):** you set it up by hand. It is not in the installer's menu. Sizes, downloads and what fits where: [docs/MODELS.md](docs/MODELS.md). To add another model later, run `SETUP.bat` (Linux: `./setup.sh --setup`). ## Using it The Strata app's Monitor (left) while a coding agent writes the pagoda garden from the video (right) - **In the browser:** open `http://127.0.0.1:8080`. It has **Chat**, a live **Monitor** of the model and your GPU/CPU/RAM, and **About** with the settings and addresses. - **Your apps and coding agents:** add an "OpenAI-compatible" provider with the base URL **`http://127.0.0.1:8080/v1`**. Any API key and any model name work. - Apps that use Anthropic's API: `http://127.0.0.1:8080/v1/messages` (Claude Code: `ANTHROPIC_BASE_URL=http://127.0.0.1:8080`). - Codex CLI and other apps that use the OpenAI Responses API: `/v1/responses` ([setup](docs/DETAILS.md#the-responses-api-and-codex-cli)). - **Thinking:** choose **off, low, medium or high** in the chat menu or in your app's "reasoning effort". Off is the fastest. High is best for hard questions. - **Pictures:** say yes to "Images?" in setup. Then click **Picture** in the chat, or attach pictures in your app. AMD cards read pictures on Linux through the processor; on Windows they can't yet. - **From your phone or another PC:** `START-HERE.bat --setup --host 0.0.0.0 --api-key `. Always set a key. - **One request at a time:** by default Strata answers one request, and the others wait. To answer several at once, set `"parallel": 2` ([BATCHING.md](docs/BATCHING.md)). On a 12 GB card this makes each answer slower. - **Long prompts:** Strata reads the first message of a chat in full, about 1 minute per 30,000 tokens. Follow-up messages start in seconds. More: [where your chats are stored](docs/INSTALL.md#where-things-are-stored), [the API](docs/DETAILS.md#using-it). ## Something went wrong? - **My PC froze the first time Strata started.** This is normal while it loads the model. Wait, and don't close the window. Still frozen after 10 minutes? Restart the PC, close other programs and try again, or pick a smaller size. - **It stopped while downloading or installing.** Run `START-HERE.bat` (or `./setup.sh`) again. It continues where it stopped. - **It's very slow and the disk light keeps blinking, or it says "the engine stopped unexpectedly".** Your PC does not have enough free RAM. Close other programs (browsers use a lot), or pick a smaller size (Q2_0 or IQ2_XS). - **It says port 8080 is already in use.** Strata is already running. Look for its window. More problems and their fixes: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Still stuck? Open an [issue](https://github.com/Niko1221/Strata/issues) and attach `strata-.log` from the Strata folder. Found a security problem? Report it privately: [SECURITY.md](SECURITY.md). ## How does it work? Models like this one usually run on servers with hundreds of gigabytes of graphics memory. Your graphics card has 12-24 GB. Strata makes the model fit by **sharing the work across your whole PC**. Think of a kitchen: the things you use all the time stay on the counter, and the rest waits in the pantry. - **The model is a team of 24,576 small specialists ("experts").** Each word needs only 10 of them. - **Your graphics card** keeps the few thousand experts that are used most often. **Your RAM** holds all of them, and **your processor** works on the rest at the same time. **Your SSD** holds a big lookup table. - **Guess, then check:** a small helper guesses the next few words. The big model checks them all at once. You get the same answer, 1.6-1.8x sooner. - **Long texts are read in big pieces** (up to 8,192 tokens at a time), at over 1,000 tokens per second. The longer explanation: [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Every part and its numbers: [the details](docs/DETAILS.md#how-it-works) and the [paper](docs/paper/Strata-Paper.pdf). ## Credits and license The model is [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) by the Qwen team. It was compressed by [ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5) and Unsloth. Strata uses parts of [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp). All credits: [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata is open source under the [MIT License](LICENSE). A few parts and every model have their own licenses ([which ones](docs/HOW_IT_WORKS.md#license)). ## Support Strata Strata is free and open source. If it is useful to you, you can support its development: