MoroLMEvaluation
How v0.6 is being evaluated, what its generations mean, and what remains unproven.
Work in progress: this page describes MoroLM-56M v0.6 and will change as the project moves.
What v0.6 proves#
v0.6 proves that the corrected pipeline can learn a causal language model from random initialization. The tiny overfit test learned exact next-token continuation; the full run then produced recognizable structures in English, French, Arabic-script Darija and Latin Darija.
Representative raw generations#
| Prompt | Observed behavior |
|---|---|
| Hello, my name is | Grammatical English structure, but repetitive semantic loops. |
| The capital of Morocco is | English sentence patterns learned, factual answer incorrect. |
| سلام خويا | Greedy often stops; sampling can produce fluent Moroccan locality prose. |
| 3lach hadchi | Sampling produced a common Latin-Darija phrase from the corpus. |
| I have 8GB RAM, wach | Can switch into Latin Darija, but relevance is unreliable. |
| Le Maroc est | French syntax learned; factual/topic drift remains. |
Greedy vs sampling#
Greedy decoding exposes high-probability repetition loops strongly. Sampling with temperature 0.8, top-k 50 and top-p 0.9 reveals a richer learned distribution, including full Arabic-script Darija passages. This does not make the model smarter; it shows that the probability distribution contains more language than the single greedy path suggests.
Memorization audit#
One generated Latin-Darija phrase was found verbatim multiple times in training, confirming corpus recall. An Arabic locality paragraph did not exist as one exact string, while its geographic and stylistic fragments occurred many times across different records. The current interpretation is template recombination with heavy memorization pressure, not factual reasoning.