Yotsuba

Yotsuba is an attempt at training an improved variant of the original GPT-4chan model, utilizing the more modern Qwen 2.5 platform and a structured training flow.

The current loaded model at the live demo is the 3B variant.

It has been trained on a heavily cleaned version of the Raiders of the Lost Kek dataset, which contains 3.5 years worth of data from 4chan's /pol/ board.

The model outputs in a structured pseudo-XML style format intended to make it easy for applications or other models to parse.

Disclaimer

The dataset used is known to contain offensive, disturbing, explicit, hateful or otherwise objectionable content. It also contains heavy political opinions, some of which are targetted at real world public figures. The outputs generated do not reflect the views or opinions of the developers and are intended solely for research and experimentation.

Training Procedure

The original dataset contains a huge number of threads, with almost 1/3 of them being low quality dead threads. These threads were cleaned out, together with extra metadata not useful for this model.

Formatting was converted to a simplified pseudo-XML style format:

<thread>

<post=1>
...

<post=2>
...

<post=3>
>>2
...a reply to post 2...

</thread>

The LoRA was trained with a Rank of 64, and an Alpha (α) of 128. Learning rate was set to a conservative 2e-5, running for 1 epoch.

Model Performance

Even the 3B variant adapted remarkably well to both the specialized vocabulary of the dataset and the structural constraints of the pseudo-XML format.

Format & Linguistic Replication

The model achieves near-perfect compliance with the <thread> and <post> tag structures. Broken tags are extremely rare, even when generating long, multi-post threads. The model successfully learned to utilize the >>[id] syntax to reference prior posts and maintain cohesive reply chains within the generated context window.

The distinct tone, slang, and formatting idiosyncrasies like greentexting are accurately and consistently replicated.

The Image Problem

A huge amount of threads from /pol/ have images in context, be it the main post or following replies. Since Yotsuba was trained purely on text, a good number of the threads it learned from don't make sense without the image in context.

We didn't remove these threads in order to keep the corpus huge, but this results in the model learning a lot of discussions that it can't make sense of, and is probably a strong source of coherence drops in its generations.

Open Weights?

Yotsuba is closed source for the time being. The model is inclined to output hateful content targetted at vulnerable/minority groups and public figures. GPT-4chan's release was controversial enough to be eventually removed from Hugging Face.

We are working towards running a public fair-use API endpoint in the foreseeable future, but for now there is no intention to release the weights publicly.