How Good Is K2-Horizon from Abu Dhabi? On an M5 It Solves the Problems, Then Stumbles over Chinese Prose

How good is K2-Horizon, the new model family from Abu Dhabi? Give it a reasoning problem and then ask it to write an essay, and the two results tell very different stories.

After adding K2-Horizon-MoVA-36B-A4B to zLLM, I ran the IQ3_XS quantization on an Apple M5 Mac with 24 GiB of unified memory. It correctly solved a snail-climbing problem and the classic cats-and-mice rate problem. When asked for a Chinese essay about a market after the rain, however, it began mixing English, Japanese, and Arabic into the prose and eventually hit repetition detection.

Lowering the temperature to zero improved the output, but did not eliminate the problem. Before evaluating the model, though, there is a more basic engineering question to answer: what did zLLM actually have to implement to run it end to end on Metal?

K2-Horizon is not a conventional Transformer that can be supported by swapping in a new set of matrix dimensions. It places sparse experts in the value path of attention, creating two independent routing systems inside a layer. The first half of this article explains how zLLM connects the GGUF, MoVA, MoE, KV cache, and complete decode path. The second half examines the distance between solving a problem and writing consistently good Chinese.

Start with the source: IFM is headquartered in Abu Dhabi

K2-Horizon was developed by the Institute of Foundation Models, or IFM. According to IFM, the institute is part of Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), headquartered in Abu Dhabi, UAE, with research centers in Paris and Silicon Valley. The K2 Horizon family was released on September 3, 2026. See the IFM overview and official announcement.

Calling it simply a European model would therefore be inaccurate. A better description is a model released by a global research team headquartered in Abu Dhabi, with research centers in Paris and Silicon Valley.

36B parameters of capacity, about 4B active per token

The complete model name is K2-Horizon-MoVA-36B-A4B. IFM describes it as a 36B-parameter model that activates about 4B parameters per token, with a native context length of 524,288 tokens. Those are model specifications; this local test did not attempt a 512K context. See the official model card.

Conventional MoE models apply sparse routing mainly to the feed-forward network. K2-Horizon's MoVA, or Mixture-of-Value Attention, extends expert routing into the value path of attention. IFM describes the design in its K2 Horizon architecture overview.

The local GGUF metadata describes 48 layers, a hidden size of 2,560, 32 query heads, and 8 KV heads. The value path selects 4 of 64 experts. After three leading dense layers, the feed-forward path selects 8 of 100 routed experts and also executes a shared expert. zLLM now supports the corresponding grouped RMSNorm, attention output gate, MoVA, MoE, complete Metal prefill and decode, and Q8g64 KV cache.

The file used here was:

K2-Horizon-MoVA-36B-A4B-IQ3_XS.gguf

It is 15,695,083,968 bytes, or about 14.62 GiB. This is a third-party low-bit quantization rather than an official IFM BF16 checkpoint. Its tensors mix IQ3_S, IQ3_XXS, Q3_K, Q6_K, and F32; the file is not made entirely from one “three-bit” tensor format.

Activating only a subset of experts reduces computation. It does not mean that only 4B parameters need to be loaded. The quality results below belong to this exact GGUF and runtime configuration and should not be treated as measurements of the official BF16 model.

Integration step one: K2 cannot be treated as ordinary GQA

zLLM does not bake K2 dimensions into Metal kernels. Model dimensions, layer counts, normalization, attention geometry, expert counts, and Top-K values live in a dedicated K2HorizonConfig. When a GGUF is opened, zLLM reads and validates the metadata field by field.

The validation goes beyond hidden size. The current implementation confirms sigmoid expert routing, normalization of selected expert weights, one shared expert per sparse layer, matching key and value head dimensions, and MoE in every layer after the dense prefix. A structurally incompatible GGUF fails during loading instead of carrying an invalid shape into GPU execution.

The model schedule lives in runtime/k2_horizon/. It describes the actual flow of one K2 layer:

hidden
  → grouped RMSNorm
  → query + attention gate
  → key
  → dense value (first 3 layers) or 4 of 64 value experts (last 45 layers)
  → RoPE + GQA + KV append
  → attention gate + output projection + residual
  → grouped RMSNorm
  → dense FFN (first 3 layers) or 8 of 100 routed experts + shared expert
  → residual

The value path is the easiest part to wire incorrectly. In ordinary GQA, value is a single linear projection. From layer four onward, K2 first applies a sigmoid router and bias, chooses 4 of 64 value experts, computes their outputs, and combines them with normalized routing weights. The result has 8 KV heads of 128 dimensions each, for a 1,024-dimensional value vector, and only then enters attention.

The FFN has a separate router. It selects 8 of 100 routed experts, executes a shared expert, normalizes the selected weights, and applies a scaling factor of 2.5. The two routers own different weights and serve different dataflows. Treating both as one generic “expert” branch would hide a real dependency.

The third-party GGUF quantizes the small but sensitive MoVA router as IQ3_S. zLLM decodes only that router to resident F32 during loading to keep routing scores stable. The much larger value-expert weights remain quantized and are consumed directly by Metal. This keeps the exception local instead of weakening the general weight-loading contract or expanding all expert weights.

Integration step two: IQ3_XS is not one kernel

Opening the tensor directory reveals five storage types: 322 IQ3_S tensors, 240 IQ3_XXS tensors, 3 Q3_K tensors, 1 Q6_K tensor, and 232 F32 tensors. A single generic “IQ3” entry point is not enough.

To execute the whole file, the Metal path needs ordinary quantized GEMV, fused gate/up projection, expert-indexed gate/up/down projection, the 4-of-64 MoVA value operation, and a Q6_K LM head over a 250,624-token vocabulary. These kernels read packed GGUF weights directly. Fully dequantizing the weights before every generated token would throw away much of the capacity and bandwidth advantage of the 14.62 GiB file.

Prefill and decode also need different execution shapes. Prefill processes multiple token rows, routes the batch, groups work by expert, and then executes each group. Decode has one row and uses device-resident Top-K results with indexed expert kernels. The last 45 layers contain 100 FFN experts each: 4,500 expert groups totaling about 10.00 GiB. The zero-configuration run preloads them into Metal resources visible through unified memory so that generation does not stop for per-layer disk I/O.

Integration step three: a 24 GiB machine still needs room for KV

A 14.62 GiB weight file does not leave every other byte available for context. Execution also needs resident expert resources, activations, scratch space, Metal objects, and memory for the operating system. At startup, zLLM reads the model geometry and machine budget, then computes KV cost per token. The M5 run reported:

Weights                  14.62 GiB
KV budget                 0.96 GiB
KV cost per token       ~101,376 bytes
Selected context length   9,216 tokens

For K2 on a 24 GiB machine, zLLM permits a working-set ceiling of three quarters of physical memory while preserving at least 2 GiB for the system. KV cache defaults to Q8g64, with one quantization group per 64 values. Short contexts use direct attention. Beyond 256 tokens, split-KV divides history into parallel segments and merges the partial results. This is why the model may advertise 524,288 tokens while the zero-configuration entry point chooses 9,216 on this machine.

Flattening complete decode into replay

One decode round crosses all 48 layers. An early profile showed roughly 1,100 Metal commands per token. Rebuilding and submitting that graph from the CPU for every round would repeat substantial scheduling work.

zLLM therefore implements complete single-token decode replay for K2. The first pass records the sequence from an uploaded token embedding through 48 layers of attention, both expert routers, MoE, final norm, LM head, and argmax. Later rounds update the token embedding, position, and KV write slots, then replay the same plan. Q8 attention replay selects its direct or split-KV pipeline from the current position, so crossing the 256-token boundary does not silently preserve the short-context path.

There is an important limitation behind the performance figures. Greedy generation can replay the complete round. Temperature/top-p sampling must retrieve logits and maintain sampling state, so the current K2 path does not enter the same replay fast path. The roughly 30 tok/s greedy result therefore does not describe the sampled writing tests later in this article.

Prove the numerical path before judging the text

Integration did not stop when the model began emitting tokens. The validation record includes:

• K2 Q8 direct and split KV append comparisons against the reference;
• a BF16 split-KV vectorized-kernel argument-binding regression, fixed and rechecked against the CPU oracle;
• Q6_K GEMV compared with a dot product over decoded weights;
• Metal top-p sampling over a 250,624-token vocabulary checked point by point against the CPU reference;
• full real-weight loading, prefill, decode, Q8 KV cache, and replay on the M5.

Under low-reasoning greedy settings, zLLM and llama.cpp produced the same four wrong answers for the same GGUF. With high reasoning and enough token budget, the archived llama.cpp test answered all four questions correctly across three seeds, for 12/12. This comparison does not prove that every numerical path is identical, but it rules out the simple explanation that those four mistakes occurred only in zLLM.

The console exposed another problem that was subtler than a kernel bug. The shared client initially sent enable_thinking=false explicitly. K2's chat template interprets that as an empty thinking fence, forcing a direct answer and making mixed-language degradation more likely. The corrected client leaves that field unset for K2, preserving the template's default high-thinking mode. MiniCPM5 and Qwen retain their separately validated settings. Model support includes template semantics as well as matrix math.

Current speed and the remaining bottlenecks

In an archived run with the same 55-token prompt, greedy decoding, and a 256-token completion, llama.cpp used F16 KV and measured 0.322 seconds for prefill plus 7.800 seconds for decode, or 32.691 tok/s. zLLM used Q8 KV and its best recorded run measured 5.999 seconds for prefill plus 8.623 seconds for decode, or 29.690 tok/s, 9.18% slower.

This is not a strict same-KV-format comparison, but it makes the current position clear: decode is close, while short-prompt prefill remains much slower. In a single-token profile, the two sigmoid routers—the 8-of-100 FFN router and the 4-of-64 value router—ranked among the largest GPU costs. Split-KV beyond 256 tokens and replay submission also leave room for improvement.

The machine eventually thermal-throttled during the last stress run, increasing direct decode from about 30 ms/token to 80–100 ms/token. The article therefore uses the reproducible pre-throttling measurements instead of presenting an overheated transient result as normal performance.

Give it enough reasoning budget, then inspect the answer

The quality test used high reasoning, temperature 1.0, and top-p 0.95, matching IFM's recommended settings. Each request had a 9,216-token context limit and up to 4,096 generated tokens. The questions were independent and did not share chat history.

Reasoning mode is not merely a display option. The local template changes its thinking marker based on the selected effort; disabling thinking inserts an empty thinking fence. If a run spends a tiny completion budget reasoning and ends with finish=length, it should be recorded as incomplete rather than graded as if its intermediate text were the final answer.

Both reasoning problems below reached a normal stop:

Problem	Correct answer	Observed answer	Generated tokens, including reasoning
A snail climbs 3 m by day and slips 2 m at night. On which day does it leave a 10 m well?	Day 8	Day 8	1,344
Three cats catch three mice in three minutes. How long do nine cats need for nine mice?	3 minutes	3 minutes	700

The final climb has no following slip, which makes day eight correct. The cats work in parallel, so scaling both cats and mice by three leaves the time unchanged. The model got both conclusions right.

Its Chinese explanation of the snail problem nevertheless included this sentence:

每天 daytime 爬升 3 米， nighttime 回滑 2 米

It means “climbs three meters in the daytime and slips two at night,” but inserts the English words daytime and nighttime into an otherwise Chinese sentence. It also produced the malformed phrase 白天下爬到. A correct numeric answer did not make the explanation naturally written.

The market essay makes the problem obvious

The first writing prompt, in Chinese, asked for a restrained 400–600-character slice-of-life essay titled The Market after the Rain, organized around concrete people, actions, sounds, and smells. It explicitly requested only the title and body, in Chinese.

The generated body began:

雨刚停，菜市场还不全是人。薄荷黑布遮在丑毛さらに码头上，粪水从竹筐的缝隙里滴着Becoming一条条银白色的短河。

The sentence starts in Chinese, inserts Japanese さらに and English Becoming, and is semantically broken even if those words are removed. Later text includes Granny, Japanese へえ, Arabic السجائر, and ends with ポリ袋ákááááááááá. The runtime stopped it with finish=repetition. Counting only Han characters after the title, it produced 88—far short of the requested essay.

The second prompt requested a 400–600-character explanatory essay titled If a Small Model Can Solve Problems, Can It Write Well? It asked the model to distinguish answer correctness, reliable reasoning, and natural language, with one concrete example.

This time the output formed complete paragraphs, but still contained lines such as:

Such表述语言自然。

最后把推理过程写成一段连贯的文字 describing the solving journey, 语言自然。

And the conclusion inserted the word fingernail where it had no meaningful role. The argument also treated natural language as exclusive to creative writing and tried to infer writing ability from one correct equation solution. Its finish=stop means generation stopped normally; it does not mean the essay passed review.

How much does temperature zero help?

I kept high reasoning and the same two prompts, changed temperature to 0, and ran each writing task once more.

Task	temperature=1.0	temperature=0
Market essay	Mixed languages, repetition stop; 88 Han characters	Normal stop, 489 Han characters; still contains English
Evaluation essay	Normal stop, 481 Han characters; meaningless English insertions	Normal stop, 652 Han characters; much better Chinese, but over the requested length

The counts include only Han characters in the body after the title, excluding punctuation, Latin letters, and Markdown markers.

The zero-temperature market essay has a complete scene, but still writes freshly 捕来的鲫鱼, Somehow 却不显得刺鼻, and 女孩 grabs 住鱼. It also renders the sound of bargaining as a camera-like “click.” The evaluation essay improves much more and has no comparable meaningless foreign-language insertion; AI is an ordinary abbreviation. It still misses the requested length.

The result should not be compressed into “the model cannot write Chinese.” A more accurate conclusion is that under these prompts and this runtime configuration, completion, language consistency, and instruction following in Chinese writing are unstable, and both genre and generation settings affect the result.

One engineering question remains unresolved: where does the degradation originate? The writing prompts were not repeated on the BF16 checkpoint or another runtime, so this test cannot separate quantization loss, runtime numerics, template behavior, and the model itself. Temperature zero also changes sampling state and eligibility for the current zLLM replay path, so it is not a clean one-variable ablation.

The evidence is enough to describe this local configuration. It is not enough to judge the entire model family.

Running K2-Horizon with zLLM

From the zLLM repository, pass the GGUF path to the compiled release client:

./target/release/zllm-metal \
  /Volumes/ORICO/models/K2-Horizon-MoVA-36B-A4B-IQ3_XS.gguf

--release is a Cargo build option and is not passed to zllm-metal. Once loading completes, enter a message directly; /reset clears the conversation and /exit quits. On this machine the client selected a 9,216-token context automatically. The advertised 512K model limit is not the usable capacity of this 24 GiB configuration.

This integration gives zLLM a complete executable path for a MoVA model. The writing tests add the other half of the story: an engine can run every layer correctly while a particular quantization and generation setup still produces poor prose. After looking at reasoning scores, it is worth asking a model for one concrete slice-of-life essay. Tasks without a single correct answer often reveal whether the language holds together, whether details make sense, and whether the result can actually be handed to a reader.

Test record

The tests ran on September 6, 2026, on an Apple M5 with 24 GiB of unified memory, using the IQ3_XS GGUF, Metal, and Q8g64 KV cache. There were six independent requests: four at temperature 1.0 without a fixed seed and two zero-temperature writing comparisons. Every request allowed up to 4,096 generated tokens, including reasoning and answer text. The sampled runs were not replay performance measurements.

The production entry point is zllm-metal, which identifies and loads K2 from the supplied GGUF path. Quality tests reused the same production inference path and an existing release build. The recorded source HEAD describes the workspace at test time and is not a verified build commit for that binary. The runtime returned a combined text stream rather than separate reasoning and content fields. Essay excerpts begin after an explicit title; problem grading uses the final answer at the end. English inside hidden reasoning was not counted as an error in the Chinese essay body.

The JSON record contains every prompt, all six raw outputs, finish reasons, and a runtime fingerprint. The sampled log, zero-temperature log, sampled configuration, and zero-temperature configuration are also available. The raw model output remains in Chinese and has not been rewritten. Paths in the configuration files belong to the test machine and must be changed for reproduction elsewhere.

Operator validation and earlier performance measurements are preserved in the integration report. Its measurements and the writing tests above are separate records.