Home / Docs / Overview

Overview

What MoroLM is, why it exists, and the current research milestone.

Work in progress: this page describes MoroLM-56M v0.6 and will change as the project moves.

Mission#

MoroLM is a small-language-model research project centered on Moroccan Darija. The long-term direction is a model family that can run on local or institutional infrastructure, understand Arabic script and Arabizi, handle Moroccan code-switching, and support privacy-sensitive applications without sending user data to a third-party inference provider.

Design principles#

Darija-first: Moroccan language behavior is the reason the project exists.
Local-first: the target runtime is fully local through llama.cpp/GGUF.
From scratch: custom tokenizer, custom model implementation and custom training pipeline.
Traceable: preserve provenance, holdouts, checkpoints and failed experiments.
Small before large: prove the pipeline on a 56M model before moving toward much larger parameter counts.

Current milestone#

MoroLM-56M Base v0.6 Cleanstart is the first clean base lineage trained from random initialization after the causal-label alignment bug was corrected. It used a 24k frozen tokenizer, 512-token training blocks, batch size 2, gradient accumulation 8 and 12,500 optimizer updates — approximately 102.4M token-position exposures.

What it is not: v0.6 is not an instruction model, not a factual knowledge system and not a production chatbot. Its current value is proving that the language-model pipeline is functioning and that meaningful Darija structure can be learned.

Public naming vs internal paths#

The public project is MoroLM. Some repository modules and file paths still use the historical Atlas name for compatibility. Those internal names are intentionally left unchanged until a controlled refactor.