Dokitek
Software and intelligent-systems engineering company based in Toulouse, France, and publisher of clinical software. Here, we share what we learn training small language models from scratch, mostly for fun.
Models
- Dokichat-100M (from doki doki, the heartbeat in Japanese): a 100M-parameter French language model trained from scratch on a single RTX 4080. On our French benchmark, it matches models three to five times larger, and leads all models under 1B in grammar and vocabulary.
- Dokichat-100M-Instruct: its chat version, fine-tuned on synthetic French conversations.
Coming soon
- French synthetic conversations: the datasets we built to fine-tune it.
Our models are small, learning-oriented, and will happily make facts up.
dokitek.com ยท LinkedIn