The second in a series of posts about questions I get about GEITje.
“Why the name GEITje?”
Muppets, Cows, and Seals
The name “GEITje” had actually been in the back of my head for a long time as the name for a Dutch large language model.
Naming in the world of language models is subject to interesting trends. In 2017, the Muppet generation of language models started with Allen Institute for AI’s ELMo, followed by Google’s breakthrough BERT. Naturally, ERNIE, Grover, and BigBIRD soon followed. They were succeeded by Facebook’s improved variant of BERT: RoBERTa.
What name should you give to the Dutch variants of these language models? Wietse de Vries et al. simply chose “BERTje” for their Dutch BERT model. Simple, but effective: the diminutive “-je” is immediately recognizable as Dutch, and it adds an element of cuteness. The accompanying logo, a quintessential Dutch cow, is also a shot on target.
Pieter Delobelle et al. chose a different strategy for their Dutch variant of RoBERTa.
They had their model name itself, by having the model predict the mask in <mask>BERT for themselves.
This resulted in the typically Dutch name RobBERT.
With a corresponding cute logo of a seal (rob) dressed as Bert from Sesame Street, of course.
Llamas and Other Camelids
After the arrival of ChatGPT at the end of 2022 (sadly without a cute logo), open-source alternatives quickly followed. Meta started with LLaMA (no logo), followed by a series of finetunes that made LLaMA a chatbot: including Stanford’s Alpaca (fancy alpaca with sunglasses) and Vicuña (abstract logo by Stable Diffusion 2.1).
Unfortunately, the llama doesn’t have a very extensive family. After the alpaca and vicuña, you have the Guanaco, but after that, the lamini are exhausted. Names of other camelids never really caught on.
Falcons and Distant Lands
The animal theme lingered for a while. For a moment, the model du jour was Falcon, from the Technology Innovation Institute (kind Abu Dhabi’s TNO). Logo: a polygonal drawing of a falcon.
The next open-source model to capture a place in the zeitgeist was Mistral 7B, created by the somewhat mysterious French startup Mistral AI. Model and company are named not after an animal, but after the mistral: the powerful northern wind in eastern France. The accompanying logo has nothing to do with a wind, but it does have significant nineties WordArt vibes.
A Large Dutch Language Model Based on Mistral
So: you tell me. You’re training a Dutch language model, based on Mistral 7B. What are you going to name it?
I considered using a variant on Mistral. After all, in the Netherlands, we also have a thing for wind. But somehow, the models Westenwind 7B and Noordwester 7B just didn’t sound right.
So, back to the animal theme. A goat (geit), an animal you will find on every petting farm. An animal that greedily devours everything you feed it and then starts bleating loudly. What could be more appropriate for a language model? A little goat, “geitje”, because 7 billion isn’t as big as some other models. And it fits nicely with BERTje.
So, GEITje it is. With a cute logo, conceived by ChatGPT and DALL·E 3.
And those capital letters? Good question. If you really want to, you can probably think of a sentence where the letters G, E, I, and T appear in order. GEneratIve Transformer is the best I’ve come up with so far, but you can probably do better.
This post was translated from the original Dutch with the help of GPT-4.