Open this on your other devices on the same network and they pretrain a language model from scratch together — peer-to-peer, right in the browser, through the emulated GPU logic. This is pretraining, not fine-tuning: every run starts from random weights. Only devices on your network are grouped (like Snapdrop). To invite people across networks, everyone opens ?room=YOUR-CODE — the person who created the room approves each device before it can join.
Width or sequence above 64? Bring more devices — big settings on 1–3 devices mean slow steps and a small effective batch. 4+ devices recommended.
Any public dataset with a text column works — each device streams its own random slice. Whoever presses Start picks it for the whole group; if streaming fails, the built-in corpus is used.
This is pretraining, not fine-tuning — every run trains a brand-new model from random weights. You are watching a language model learn from scratch.
A mini transformer language model — attention, MLP blocks, next-character prediction — every multiply through the verified INT8 units. Training text is streamed from FineWeb-Edu (HuggingFace); offline devices fall back to a built-in corpus. Whoever presses Start sets the settings for the whole group; total batch scales with every device that joins.
Works solo — every device that joins adds its batch to the group.
Enabled after training finishes or a checkpoint is loaded. The “inference kit” below downloads a single HTML file with these weights baked in — open it anywhere to run generations offline.
Loading a checkpoint applies it here and pushes it to every connected device — use it to recover the group after a failure.
Needs a secure context (localhost or HTTPS) for WebGPU + cross-device WebRTC. No WebGPU? The same verified INT8 units run on CPU — old machines (e.g. via Supermium) still join, just slower. Every training step goes through the Neural Units; there is no plain-float path, and if the units fail to load, training is disabled.
Heads up: peers connect directly (WebRTC), so devices in your group can see each other's IP address, and there's no gradient authentication — only train with devices/people you trust. Proof of concept.