Spark-X2.6-1.7B is the smaller model of XHToken's Spark-X2.6 series, aimed at on-device agents: tool use, coding and long context, with a thinking budget that can be set to high, low or off. This build runs the official INT4 codes through a hand-written WGSL decoder with a 32K context window. The model runs inside a browser tab, and what you type stays on your machine.
How Spark-X2.6-1.7B compares
XHToken reports these results for Spark-X2.6-1.7B against open models of similar size, all with thinking on, with their Spark-X2.6-4B as a reference. Across 30 benchmarks it averages 49.6, ahead of MiniCPM5-2B (46.9) and well ahead of Qwen3.5-2B (22.0) and Gemma4-E2B (22.5). It leads on agent and coding work such as web browsing, MCP tool use and software engineering; MiniCPM5-2B is ahead on competition math, BFCL-V4, AA-LCR and GPQA.
How Spark-X2.6-1.7B compares
All 30 benchmarks
Spark-X2.6-1.7B benchmark results
The browser build scores the same
A different runtime and 4-bit weights can change a model's answers, so the browser build was tested against the original model in transformers: same prompts, same chat template, greedy decoding, thinking off. Every browser number comes from the actual web page, run in headless Chrome.
The browser build scores the same as the official model
The browser build matches the official INT4 checkpoint on all three. On GSM8K the two disagree on 38 and 38 problems in the two directions (McNemar p = 1.00); on BFCL on 4 and 4 cases (p = 1.00). At the level of individual tokens, the build adds a mean KL divergence of 0.00085 on top of the INT4 quantization, and 30 minutes of back-to-back generation produced no errors and no slowdown.
The 4-bit weights themselves cost this model more than they cost the 4B: the official INT4 checkpoint scores 24 points lower on BFCL function calling and 3 points lower on GSM8K than the original bf16 model, mostly because it answers in text more often where the original calls a tool. The browser build reproduces the INT4 checkpoint, including that.
Validation against the original model
Speed
In the earlier ONNX build on a desktop GPU, the model generated about 105 tokens per second and started answering a short prompt in 49 ms. Generation kept that pace as the conversation grew: 107 tokens per second at 16,000 tokens of context. Long prompts took longer to read before the first token: 618 ms for 2,300 tokens and 5.9 s for 16,000.
1.2x to 1.5x the decode speed of the 4B build
Decode speed
tokens per second
Time to first token
seconds
Get started
The model is a folder of static files and the loader is a few JavaScript modules, with no build step and no server code.
-
Chat in the browser
Open the chat in desktop Chrome or Edge. The model downloads once (1.20 GB) and loads from the browser cache afterwards.
-
Run it locally
Serve the static Space files locally. The app loads the model from Hugging Face unless a local bundle exists in
spark/runtime.git clone https://huggingface.co/spaces/leventkoldas/Spark-X2.6-1.7B-WebGPU-Runtime cd Spark-X2.6-1.7B-WebGPU-Runtime python3 -m http.server 8000 -
Use it in your own app
Copy
lib/and the import map fromindex.html.generatestreams text and acceptstemperature,maxNewTokens,effort,toolsand an abortsignal.import { loadSpark } from "./lib/runtime/index.js"; const spark = await loadSpark({ base: "https://huggingface.co/leventkoldas/Spark-X2.6-1.7B-WebGPU-Runtime/resolve/main", }); const messages = [{ role: "user", content: "Hello!" }]; for await (const text of spark.generate(messages, { thinking: false })) { output.textContent += text; } -
Reference ONNX build
The earlier ONNX export remains available in mohdmahmodi/Spark-X2.6-1.7B-ONNX-WebGPU and loads with Transformers.js for comparison. The runtime above is faster at long context, since it keeps the KV cache in one preallocated buffer.
import { pipeline } from "@huggingface/transformers"; const generator = await pipeline("text-generation", "mohdmahmodi/Spark-X2.6-1.7B-ONNX-WebGPU", { device: "webgpu", dtype: "q4f16" }); const out = await generator([{ role: "user", content: "Hello!" }], { max_new_tokens: 256, tokenizer_encode_kwargs: { enable_thinking: false }, }); console.log(out[0].generated_text.at(-1).content);
How it is built
The runtime keeps XHToken's INT4 codes byte-for-byte: most projections are 4-bit with group-32 scales and
zero points, while out_proj and lm_head are 8-bit symmetric. tools/pack_weights.py
copies those payloads into a 256-byte-aligned bundle and writes runtime/manifest.json, which is the only
source of tensor offsets. Decode runs f16 activations and f16 KV through WGSL kernels; RMSNorm statistics, weight
dequantization and softmax accumulate in f32. The ONNX graph is an offline packing input and validation reference only.
A prompt is pre-filled in 256-token GEMM chunks. Full-attention layers keep a 32,768-position KV cache, sliding layers use a 1,024-position ring, and a follow-up message reuses the longest matching prefix of the previous prompt. Each token is one state write, one command submit and a 2 KB readback of the top-256 candidates; greedy decoding reads back 4 bytes. The source repository holds the earlier ONNX export, its validation report, and the chat app history.
Limitations
- It needs desktop Chrome or Edge with WebGPU and
shader-f16, and about 1.8 GB of free GPU memory. - The runtime is specific to this Spark-X2.6-1.7B q4f16 bundle. It does not load arbitrary ONNX, GGUF or other checkpoints.
- The context window is 32,768 tokens; older chat turns are summarized automatically as the prompt approaches that limit.
- The official 4-bit weights score lower than the original model on function calling and math, and this build inherits that.
- Testing so far covers Windows with an NVIDIA RTX 3090 Ti and an Intel Iris Xe. macOS, Android and other GPUs are untested.
- The comparison with other models is XHToken's, measured on the original model with thinking on. It was not re-run for this build. The validation report covers the earlier ONNX build.
Spark-X2.6-1.7B