edu-ganda-gemma-e2b-v4 (EXPERIMENTAL)

Experimental research checkpoint — not a production model. A Gemma-4-E2B Luganda primary-education assistant from Crane AI Labs' July GRPO lineage (this is the QAT W8A8 serving checkpoint, internally ganda-e2b-v4 / e2b-v4-qat-serving).

Good at

• Instruction-following — much stronger than the a065 Luganda-vocab lineage.
• Fluent Luganda prose.

Bad at

• Luganda content accuracy. It follows instructions well but makes Luganda content errors (wrong glosses, invented rules). In a blind A/B with 10 teachers, teachers preferred the untuned base Gemma-4-E2B 61/39 over this lineage — the deficit driver was Luganda content correctness, not fluency. Do not trust its Luganda factual/grammatical claims without review.
• Guard-free repetition / doom-loops — degenerates into repeated text without decoding guards.

Required serving settings

• Repetition guard to suppress loops: repetition_penalty=1.15 + no_repeat_ngram_size=3 (or the lighter repetition_penalty=1.0 + no_repeat_ngram_size=4).
• eos_token_id=[<eos>, <end_of_turn>] (multi-EOS; Gemma-4's <end_of_turn> is a separate stop token from the tokenizer default EOS).
• Prepend <bos> (id 2) before the chat-templated prompt — omitting it yields garbage logits on this Gemma-4-E2B lineage.
• Greedy decoding recommended over temperature sampling.

Known technical note (benign)

On load, transformers reports self_attn.k_proj/k_norm/v_proj MISSING for language-model layers 15–34 (60 keys) and randomly-inits them. Verified benign — generation is coherent and on-fingerprint despite the warning; a historical Unsloth save_pretrained_merged save artifact from this lineage's build, present identically across the whole local e2b-* merge lineage.

Intended use

Research on low-resource (Luganda / East African) education assistants; as a merge/distillation donor (see the edu-ganda-gemma-e2b-v5 merge, which combines this model's instruction-following with a Luganda-vocab-repaired base). Not for direct production serving without the guards above, and Luganda content should be reviewed.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL = "CraneAILabs/edu-ganda-gemma-e2b-v4"
tok = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForCausalLM.from_pretrained(MODEL, torch_dtype=torch.bfloat16, device_map="auto").eval()
eos_ids = [tok.eos_token_id] + tok.convert_tokens_to_ids(["<end_of_turn>"])

def chat(prompt, max_new_tokens=256):
    text = tok.apply_chat_template([{"role": "user", "content": prompt}],
                                   add_generation_prompt=True, tokenize=False)
    inputs = tok(text, return_tensors="pt").to(model.device)
    out = model.generate(**inputs, max_new_tokens=max_new_tokens, do_sample=False,
                         repetition_penalty=1.15, no_repeat_ngram_size=3,   # guard: prevents loops
                         eos_token_id=eos_ids, pad_token_id=tok.pad_token_id or tok.eos_token_id)
    return tok.decode(out[0, inputs.input_ids.shape[1]:], skip_special_tokens=True).strip()

print(chat("Nnyonnyola engeri y'okunaaba engalo mu Luganda."))

Recommended decoding: greedy with repetition_penalty=1.15, no_repeat_ngram_size=3, eos_token_id=[<eos>, <end_of_turn>]. The repetition guard is needed to prevent loops.