Home / Docs / Runtime & export

Runtime & export

The local llama.cpp/GGUF runtime path and application architecture.

Work in progress: this page describes MoroLM-56M v0.6 and will change as the project moves.

Target runtime#

Browser UI
   ↓ streaming HTTP
Go backend
   ↓ local API
llama-server
   ↓
MoroLM GGUF

SQLite stores application memory / metadata.

llama.cpp parity work#

Earlier export work established a Llama-compatible Hugging Face-style export, F16 GGUF conversion, SentencePiece parity and matching greedy token IDs between PyTorch FP16 and llama.cpp F16 for the same checkpoint. That proved inference/export consistency, although those historical weights were later invalidated as clean base-model foundations by the training-objective bug.

Local binary#

The development environment uses a Windows llama.cpp server binary. The eventual product goal remains a lightweight local server that can run quantized MoroLM checkpoints without a Python dependency.

Quantization#

Planned formats include F16 reference GGUF and practical Q8/Q4 variants. Quantization should happen only after a checkpoint has passed PyTorch evaluation and export parity tests.

Application layer#

The broader application design is a Go backend, streaming web UI and local SQLite memory. The model itself should remain separable from the app so later 1B/2B models can replace the 56M research model without rewriting the product.