what fires together, wires together
entries: 1 · counting
May — Jul 2026
High End Computing AI & DL Lab, SMVDU
Researching self-supervised visual representation learning for open-world perception, exploring methods that improve model reliability when encountering unfamiliar objects, environments, and behaviors.
built from first principles
in development · data pipeline stage
A production-grade 150M-parameter LLaMA-inspired language model, built from scratch — not to reproduce an architecture, but to understand every component of a modern LLM and engineer it like software that has to ship.
custom tokenizer · RoPE · RMSNorm · SwiGLU · pretraining · instruction fine-tuning · inference optimization · FastAPI serving · Docker
complete · trained on a single T4
GPT-2 (124M) implemented by hand in PyTorch — no Hugging Face model classes anywhere. Tokenization, causal multi-head attention, the full transformer block, and an end-to-end pretraining loop, built up notebook by notebook.
preprocessing → attention → architecture → pretraining
// more models currently in training…
gradient descent, daily
inbox: open
Looking for someone who builds things from first principles? Reach out. Any channel works.
