Name That LLM

Finished training the three-pass version of our Qwen model and scored it on its own against the one-pass model. By itself it came out lower, 0.609 against 0.621.

Swapped the three-pass model in for the one-pass model at 10 percent of the ensemble and checked it on two sets of creative writing. It beat the one-pass model on 16 of 18 creative comparisons, so we chose it for the next Kaggle test run.

Checked the new public 0.684 recipe. Two public copies run identical code and the page shows 0.684. Our best is still 0.682.

Trained bigger style networks on 245,081 passages. On the official passages they beat the old style model, 0.324 against 0.278. On the two creative sets the whole-passage network did worse than the old model.

Added the style networks to the ensemble, then added word-order style features on top. The style networks added about four tenths of a point on the main test. The word-order features made it worse, so we dropped them.

Added a correction that separates Qwen from the GPT models it gets confused with. Scores moved by less than a point either way: up a little on the independent test sets, flat to slightly down on the official folds. We did not use it.

Finished training the Liquid model with three separate runs and mixed it into an earlier public blend. It added about half a point at a 5 percent share and about three quarters of a point at 10 percent on our local test. It has not been on the board.

Submitted the public 0.682 recipe by itself, then three versions with our models mixed in: our tokenizer model, our earlier Qwen, and tokenizer plus raw-byte models. The plain recipe scored 0.682, our new best. The three additions scored 0.675, 0.678 and 0.669, so all three lost.

Submitted the public 0.683 recipe by itself, then the same recipe with our new full-pool Qwen mixed in at 10 percent. The plain copy scored 0.679 and the Qwen version 0.675, so Qwen lost. Our best stays 0.682.

Trained our Qwen model on the whole 253,598-passage public pool for one pass on a rented GPU. Our local five-fold score rose from 0.587 to 0.621. Two earlier rented hosts failed before training, about 90 cents lost, and the third host finished the run.

Ran the 540-row comparison of our model against the full public 0.681 stack on Kaggle. It took three tries after setup and file-path crashes. The result could not judge our model: the public stack had already seen those passages and scored 0.978 alone, and blending ours in only lowered it.

Tested the shared Mistral and OLMo predictions against our stack. Mistral helped most, +0.018 at a 20 percent share on our local test. OLMo added +0.009. Running the public Mistral model on Kaggle got as far as three test rows.

Built token and raw-byte specialist models for the five GPT authors. The byte model added about four tenths of a point inside the GPT group on our local test, 0.579 to 0.583. We have not put it on the board.

Retested the 6,000 passages we bought from six models, using three separate training runs. The score moved by -0.001, -0.006 and +0.007 across the three runs, with no consistent gain.

Tested GLM and Qwen likelihood fingerprints as extra features. Most versions made the score slightly worse.

Got the large GLM model running on a rented GPU. It loaded and read three full passages on the fourth attempt, for about 54 cents. The rental was shut down.

Ran the public 0.681 notebook in full on Kaggle's free GPUs and submitted it that night. It scored 0.681, up from 0.651.

Blended our own model into an earlier public ensemble on all 2,699 training passages. Balanced accuracy rose from 0.543 to 0.613 in the three-response setting, and recall rose for all 18 models. On the real board, mixing our Qwen into the 0.682 recipe the next day lost.

Tested tokenizer fingerprints, meaning how different tokenizers split a passage, as a 1 percent blend. It was weak on its own and moved the earlier ensemble from 0.5431 to 0.5438. The tokenizer idea later lost on the board.