AR
AUTORESEARCH ARENA / RESEARCH DESK

NAME THAT LLM / TERMINAL

11 November 2026 · day 17 of the project · board 2026-11-11 20:25 ET
DAILY ARCHIVE · DATA · @82deutschmark on X
0.68212 submissions
437of 5,558 · -103 since 11 Nov
top 7.9%top-10% line 0.682
0.768NguyenThanhNhan
-0.086to top 10: -0.065
5224 worked · 10 lost · 10 flat
$32.45GPU $19.50 · data $12.95
0.684+0.002 vs our best

01 / Board race

OUR ENTRIES (S#, SEE LEDGER) VS LEADER AND TOP-10 CUTOFF
0.550.600.650.700.750.8026 Oct28 Oct30 Oct1 Nov3 Nov5 Nov7 Nov9 Nov11 NovDate (Eastern)Public leaderboard scorescores below 0.55 sit on the floor lineLeader 0.768Top 10 0.747Top 1% 0.705Top 10% 0.682Top 25% 0.665S1 Public baseline, first try: crashedS1S2 Public baseline, second try: crashedS2S3 Our own CPU classifier: 0.348S3S4 First public neural ensemble: 0.651S4S5 Our own Qwen model alone: 0.585S5S6 Public 0.681 ensemble run: 0.681S6S7 Tokenizer model added to the 0.682 recipe: 0.675S7S8 Earlier Qwen added to the 0.682 recipe: 0.678S8S9 Public 0.682 recipe, control (our best): 0.682S9S10 Tokenizer and byte models on the 0.682 recipe: 0.669S10S11 Full-pool Qwen added to the 0.683 recipe: 0.675S11S12 Public 0.683 recipe, control: 0.679S12
Our best so farOur entryEntry below its controlCrashed (no score)Board leaderTop-10 cutoff

02 / Where the field sits

ALL 5,558 TEAMS · 2026-11-11 20:25 ET
0501001502002503003504004505005506006507007500.550.600.650.700.750.800.550 to 0.555: 70 teams0.555 to 0.560: 59 teams0.560 to 0.565: 42 teams0.565 to 0.570: 41 teams0.570 to 0.575: 38 teams0.575 to 0.580: 28 teams0.580 to 0.585: 32 teams0.585 to 0.590: 34 teams0.590 to 0.595: 87 teams0.595 to 0.600: 51 teams0.600 to 0.605: 85 teams0.605 to 0.610: 52 teams0.610 to 0.615: 31 teams0.615 to 0.620: 35 teams0.620 to 0.625: 79 teams0.625 to 0.630: 45 teams0.630 to 0.635: 59 teams0.635 to 0.640: 123 teams0.640 to 0.645: 120 teams0.645 to 0.650: 43 teams0.650 to 0.655: 127 teams0.655 to 0.660: 89 teams0.660 to 0.665: 384 teams0.665 to 0.670: 361 teams0.670 to 0.675: 143 teams0.675 to 0.680: 125 teams0.680 to 0.685: 711 teams0.685 to 0.690: 18 teams0.690 to 0.695: 15 teams0.695 to 0.700: 11 teams0.700 to 0.705: 11 teams0.705 to 0.710: 8 teams0.710 to 0.715: 8 teams0.715 to 0.720: 8 teams0.720 to 0.725: 4 teams0.725 to 0.730: 3 teams0.730 to 0.735: 4 teams0.735 to 0.740: 4 teams0.740 to 0.745: 2 teams0.745 to 0.750: 5 teams0.750 to 0.755: 3 teams0.755 to 0.760: 1 teams0.765 to 0.770: 2 teamsmedian 0.595top 10% 0.682top 1% 0.705top 10 0.747US 0.682 · #437Public leaderboard score (0.005-wide bins)Teams

03 / Board ledger

EVERY SUBMISSION · CHANGE VS ITS MATCHED CONTROL
#DateWhat went inScorevs control
S1210 NovPublic 0.683 recipe, control
The public 0.683 recipe run unchanged on our account. The author reported 0.683 and we got 0.679, 0.003 under our best.
0.679—
S1110 NovFull-pool Qwen added to the 0.683 recipe
The public 0.683 recipe with our new full-pool Qwen model mixed in at ten percent. It lost to the same recipe without it.
0.675-0.004
S1010 NovTokenizer and byte models on the 0.682 recipe
The public 0.682 recipe with ten percent tokenizer models and twenty percent raw-byte models mixed in. It lost.
0.669-0.013
S910 NovPublic 0.682 recipe, control (our best)
The public 0.682 recipe run unchanged on our account. It matched the author's score and is our best entry.
0.682—
S810 NovEarlier Qwen added to the 0.682 recipe
The public 0.682 recipe with ten percent of our earlier Qwen3-8B model mixed in. It lost.
0.678-0.004
S710 NovTokenizer model added to the 0.682 recipe
The public 0.682 recipe with ten percent of our tokenizer model mixed in. It lost.
0.675-0.007
S69 NovPublic 0.681 ensemble run
A full run of the public 0.681 ensemble on our account. It beat our old best of 0.651 by 0.030.
0.681—
S531 OctOur own Qwen model alone
Our own trained Qwen3-8B model submitted by itself to test the pipeline end to end. It scored 0.585, close to its 0.588 local score and well below the public ensemble.
0.585—
S427 OctFirst public neural ensemble
The public neural ensemble, the 0.651 recipe, run in full on our account. Our first strong board score.
0.651—
S327 OctOur own CPU classifier
Our own character, word and style classifier, trained on the official labels, to prove an entry of ours runs on the hidden test.
0.348—
S226 OctPublic baseline, second try
A second try at the public 0.651 notebook, still with our extra wrapper code. It crashed on the hidden test and earned no score.
crashed—
S126 OctPublic baseline, first try
The first try at the public 0.651 notebook, with our extra wrapper code. It crashed on the hidden test and earned no score.
crashed—

04 / Board watch

TOP 15 TEAMS AND US · GAP TO OUR BEST
#TeamScorevs usSubsLast entry (UTC)
1NguyenThanhNhan0.768+0.086132026-11-11 13:48
2James Liebel0.765+0.083352026-11-11 13:37
3You’re Absolutely Right!0.756+0.074752026-11-11 06:50
4Le Viet0.752+0.0701062026-11-11 00:05
5Meni David Gottesman0.752+0.0701022026-11-10 00:08
6Keanan0.750+0.068652026-11-11 14:04
7kulshekhar0.748+0.066792026-11-11 00:04
8Team Aillis0.747+0.0651262026-11-12 00:05
9Hod Carriers0.747+0.065662026-11-04 22:38
10Lian Matsuo0.747+0.065442026-11-11 00:10
11Harveen Singh Chadha0.746+0.064562026-11-11 11:58
12Søren Ravn Andersen0.743+0.061752026-11-11 12:29
13Civitasmass0.740+0.058682026-11-11 15:34
14ymg_aq0.737+0.055512026-11-11 18:00
15吃白饭的大肥鱼0.737+0.05522026-11-10 20:18
437Mark Barney (us)0.682—122026-11-11 00:00

Movers since last snapshot

TOP 25
First board snapshot taken 2026-11-11 20:25 ET; moves appear from the next daily snapshot.

05 / Discussion board

KAGGLE FORUM · NEWEST FIRST · OUR TAKE IN GREY
Posted (UTC)TopicCommentsVotes
11-11 23:47Reason for qwen performing so well in classifying :/
Gaurav Rawat
Gaurav Rawat asks why Qwen classifies so well. The saved file has only the title and no replies.
00
11-11 13:02Late post-processing cell never seems to affect the score in the hidden rerun — known differences vs. commit runs?
vishu phalswal
somerandomguy says the time limit counts from kernel start, not queue, use the competition-rerun flag to detect scoring runs, and keep about half an hour spare. Check our notebooks.
40
11-11 00:08Notes from a solo pipeline: grouped CV, why prompt-group tricks didn't transfer, typographic invariants
pedrohbenetton
Only the title was saved: pedrohbenetton's notes on fold splits by prompt, prompt-group tricks that did not carry over to test, and punctuation clues. Body missing; fetch it before relying on it.
02
11-10 18:08Can we use Kaggle Benchmarks model access to generate training data?
Joanna C.
Only reply: Darragh1Walsh links a dataset he made with Kaggle Benchmarks model access and invites use. A free extra data source, but check it matches the host format first.
11
11-09 13:55Can a CatBoost-based approach be competitive in this competition?
Maxim Stepanov
Maxim Stepanov asks if a CatBoost approach can compete; the only reply is 'wondering the same'. No answer.
16
11-09 07:38Running GLM-5.3 / GLM-5.3-Flash inside the 9h limit — has anyone got the public GGUF segments working?
vishu phalswal
Vijay Konda says GLM-5.3 likelihood only fits the time limit by streaming one layer at a time over all test passages; John K shows GLM and Claude labels heavily confused. Worth a test.
55
11-09 02:09Model Parameterizations
James Bedichek
Title only (James Bedichek, model parameterizations), no body or replies saved. Nothing to act on.
01
11-07 20:46Under-18 participation and consent procedure
cora zeng
Under-18 consent procedure question, no replies. Not relevant to us.
01
11-06 14:00Random K-fold is pessimistic here: 0.11 vs 0.35 with grouped folds (reverse leakage)
Renan Pinheiro
Renan Pinheiro says random folds score 0.11 but folds grouped by prompt score 0.35 (board 0.355); most errors stay in the right family. Use grouped folds and try a family-then-model step.
22
11-06 12:55Clarification on external datasets built by sending competition text to third-party LLM services
subuta_umee
subuta_umee asks if outside data made by sending competition text to other LLM services is allowed; no official answer, one unhelpful reply. Rule still unclear.
11
11-05 22:00Looking to merge: unique Mistral/OLMo LoRAs with full OOF + fresh Gemini data (public 0.663)
Emmanuel Ocheme
Emmanuel Ocheme offers Mistral and OLMo LoRAs (public 0.663). Anto Pil says they add only 0.004 to his 0.666 blend and declines to merge. His and Emmanuel's fold predictions are public; test them in our blend.
4-3
11-05 11:09Same ablations on two bases: what each part is worth on the public LB (0.638 vs 0.659)
HALX2158
Empty in our copy: title promises per-part public LB ablations on two bases (0.638 vs 0.659), but no body or comments were saved. Worth fetching the post itself.
01
11-05 03:07 Prize Winner Requirements – Name That LLM
radidani
radidani's post on prize winner requirements. The saved file has only the title, no body or replies.
01
11-05 00:14Stylometric baseline + grouped CV for Name That LLM (notebook)
Matin shirazi
Matin shirazi shares a style baseline with grouped cross-validation notebook; no comments. Our style models already cover this.
01
11-04 17:28Public Score: 0.333
Mohammed Hasan Abdullah
Title only (Mohammed Hasan Abdullah, public 0.333), no body or replies saved. Nothing useful for us.
00
11-04 13:26Prompt Group Sizes: Is it always exactly 18 texts per prompt?
ナハト・ ファウスト
John K says train has 18 answers on 149 of 150 prompts, but test groups are not full and model counts differ. Never force one-of-each-model per prompt at test time.
53
11-04 03:34Name That LLM: grouped validation, stylistic baselines and unsuccessful neural experiments
cylovelife
cylovelife's write-up on grouped validation, style baselines and failed neural runs. The saved file has only the title, no body or replies. Nothing to act on.
01
11-03 23:51only 3 rows in the test data
Marie Victoire IPOUK
Newcomer asks why test has 3 rows; somerandomguy says that is a sample and the hidden test has about 4000 rows. Known already.
20
11-03 17:34My approach for DistilBERT fine-tuning (0.2515 OOF)
Qhaleesh
Qhaleesh posts a DistilBERT baseline at 0.25. OsakanaTeishoku's only reply says to split folds by prompt, since test prompts are unseen. We already do that; nothing new for us.
1-2
11-03 11:49Will self-generated extra data help or hurt here?
Ruslan Akhmetzianov
somerandomguy says his self-generated data lowered val even after matching structure and length; Soham Acharya fears topic bias. Supports not paying for our own generation.
43
11-03 07:38My classifier thinks GLM-5.3 is more Claude than GLM
AyoGG
AyoGG says their classifier calls GLM-5.3 Claude; commenters blame distillation or routing to Claude. This confusion pair is a likely error spot to check in our results.
45
11-03 01:22Any real humans left in here?
Civitasmass
Joke thread about LLM-written posts on the forum; one user says he is stuck around 0.66. Nothing useful for us.
1116
11-02 04:23What score your best single model can achieve?
Jak Ma
Best single models: Horikita Saku and Xiao say 0.65 to 0.68 with Qwen3-14B and own data; GoPain says Qwen3-8B LoRA 0.628, ensemble adds under a point. Most gains came from data, so more clean data beats more blending.
2615
10-29 23:25New external dataset: 9,719 OpenAI responses with exact prompts and provenance
Enai
Comments are just a team-up request (bosprimigenious, LB 0.349) and an upvote. Nothing about whether the 9,719 OpenAI-response dataset helps.
216
10-29 09:45[BIG UPDATE - please try it] Same data, +0.027 accuracy: 1 in 4 passages in the public NTL pool was junk or the wrong genre. I cleaned it (Open Pool v9)
Carrot Production
Carrot Production says a quarter of the public pool was junk or wrong genre, and cleaning it (Open Pool v9) gave +0.027. Nobody has replied. Try that cleaning on our full-pool Qwen training.
08
10-27 11:52Retrained the Qwen2.5-14B member of nlm-v27 on the 120k pool: OOF 0.569 → 0.605, public LB 0.651 → 0.647. Why?
Harshita Kumari
Harshita: swapping old Qwen2.5-14B for a higher-score retrain cost board points; HALX2158 says diverse weak members matter. GoPain: centred-head weight over 1.0 lost. Add diversity, not clones.
78
10-26 07:59[NOW UPDATED]## Open Pool v14: real `kimi-k3` was written with reasoning on, `glm-5.3` is fine, and 284 mislabelled rows are gone
Carrot Production
Carrot Production updates the Open Pool to v14, with Kimi relabelled and wrong rows removed. GoPain shows a board gap of 0.002 between two models is noise. Train on v14 and judge gains by our own cross-validation.
1885
10-22 16:08Anyone tried a Hierarchical Approach? (Predicting Provider first, then Specific Model)
Ghaith Mechi
Only answer: Cloud9 links an arXiv paper (ATLAS) that predicts the provider well. No one reports results for provider-then-model here. Nothing to act on.
58
10-20 13:59Section 2.6 question: training data generated from the candidate models
Adisak Sukul
Adisak Sukul asks six questions on training data made from the candidate models; rw-nue asks about a spending cap and release deadline. No host answer, so the rule stays unclear.
110
10-19 12:29What the punctuation already tells you about the 18 models (and what it can't)
Masood Ahmed
HALX2158 says Gemini never em-dashes in train but proxy-made Gemini text does (13-43%); Nikhil sees Claude and GLM confused. Punctuation rules may flip if we add outside Gemini data.
1019
10-18 12:02Early Qwen layers know the author better than late ones: 0.39 vs 0.31
Aleksey Osipenko
Aleksey Osipenko says early Qwen layers predict the author better than late ones, and top public work cuts Qwen3-8B to 24 layers and pools several. About half the errors are within one model family.
79
10-18 04:55New here? Grouped CV that tracks the LB, CPU baseline code (LB 0.353), pitfalls, 2xT4 tips
Adisak Sukul
GoPain says grouped CV reads 0.03-0.04 high vs LB once the pool is in training, and class balancing gave +0.011 on LB. Our pool-trained local gains are inflated; keep balancing.
524
10-17 14:17Useful helper dataset
prvsiyan
prvsiyan shares a helper dataset; Darragh1Walsh adds one he updates daily. Commenters doubt its labels because it came through an unofficial proxy. Use only if checked against official data.
1355
10-17 10:56[Dataset] Extra Claude passages on new prompts, and what has worked with them so far
Seyit kaan Gunes
Zbigniew Kirchner says extra Claude passages in a side Claude head on a Qwen3-8B LoRA lifted val 0.456 to 0.493, and a lab-then-version hierarchy gave +2.4. Cheap Claude-split idea to try.
310
10-16 11:18Could the Organizers Provide More Official Training Data?
Yan Zhou
Yan Zhou asks the hosts for more official training data. buysider asks whether self-generated text from model providers is even allowed; Yan doubts hosts will answer. No ruling given.
210
10-15 16:2718-model wide response additional corpus
Oleg Zholobov
Oleg Zholobov built the 18-model corpus from Hugging Face and GitHub eval logs. HALX2158 says its Markdown and line breaks hurt CV even flattened. Filter to short single paragraphs.
532
10-15 15:39Clarification on self-generated data from model APIs
Efe Karamese
Efe Karamese asked if self-made API data is allowed; haritch asks whether public API-made datasets on Kaggle are fine. No host answer. Unresolved rule risk if we use those datasets.
18
10-15 07:08Has anyone tried the PersonalLLM dataset for this competition? (namkoong-lab/PersonalLLM)?
YePing Lin
YePing Lin asks whether anyone tried the PersonalLLM dataset; no replies. Just an open question, nothing to act on.
09
10-15 00:50 Reconstructed train prompts for all 150 prompt_ids (dataset + method)
pjmathematician
Only the title was saved: pjmathematician says he rebuilt the train prompts for all 150 prompt groups and shared them as a dataset. No body or replies. Worth a look as a source for new training text.
022
10-14 23:17Will the winner just be whoever can get the most amount of extra training data?
CoreyJamesLevinson
HALX2158 says the public 18-model corpus lowered grouped CV (format mismatch). James Warner puts extra data at about +0.11. Only data shaped like the host's helps us.
69146

06 / Public notebooks

CLAIMED SCORE FROM TITLE · GREEN = ABOVE OUR BEST
NotebookClaimedVotesLast run (UTC)
Name That LLM 0.684: All Public + What Failed
alex chilton
0.68422011-11 20:02
0.684 | I'M BETTER !
Areeb Ali
0.6849411-11 05:38
0.683 | THIS ACTUALLY WORKS
Areeb Ali
0.6833411-11 13:21
0.683 | Qwen3 + WeMM Embedding Experts
Rohan
0.683111-11 13:59
0.682 | 0.681 + Qwen3.8-27B GSQ 12% + Mix 0.50
Omer Faruk Merey
0.6828011-10 05:44
0.681 | Qwen3-Embedding LoRA + Qwen3.8-27B
Vladislav Lavrentev
0.68123811-08 08:55
0.681 | LoRA + Qwen3.8-27B Starter Notebook
Ertuğrul Özer
0.68112011-10 01:04
0.681 | Five-Fold Embeddings + Qwen3.8-27B LoRA
Vladislav Lavrentev
0.68110011-12 00:26
0.680 | ModernBERT + LoRA + Sparse Ensemble
Vladislav Lavrentev
0.68029011-07 18:31
0.675 | Are We Satisfied?
Areeb Ali
0.6757211-08 21:42
0.672+ | Dual-SVC Residual Refinement Apex
Ertuğrul Özer
0.6725811-11 17:49
0.668 | Verifier + Matched Rows + Pool Prompts
Omer Faruk Merey
0.66811811-04 06:24
ntl fuse33 (0.668 fork + response corpus)
Chunyu Wei
0.668911-12 01:24
Testntl fuse33 (0.668 fork + response corpus)
m-toshi desu
0.668011-11 23:17
0.664 [Hyperparameter Tuned] | Who are U?
Shiner
0.6649910-31 04:24
0.663 | 0.660 V31 + ATLAS 20%
Omer Faruk Merey
0.66312610-29 12:27
0.661 | Qwen3.8-27B LoRA + Spectral Ensemble
Vladislav Lavrentev
0.6617010-29 00:06
0.642 | Teach More, Keep the Signature
Kirill Tymchishin
0.6426910-26 01:47
Two-Stage LoRA + Gaussian + Full Hungarian [0.604+
Sangam Gautam
0.6048410-19 17:14
0.600 | Teach the Model, Remove the Prompt
Kirill Tymchishin
0.60011310-19 14:24
0.552 | Adaptive Centering + LoRA + Claude Gates
Zbigniew Kirchner
0.55210810-17 09:39
Starter Notebook for Name That LLM Competition
MartynaPlomecka
—63010-14 18:27
Name That LLM | Who am I?
Roman Tamrazov
—16410-28 15:57
Teach the Model + 10% ATLAS
Joseph Adamski
—15910-20 10:13
nlm-v27
Dhruv Tibarewal
—14510-25 04:47
Two-stage LoRA on the public pool
Nishant Dahal
—13510-19 01:50
LoRA x Adaptive Centering
Lucas Boesen
—11510-17 11:57
The Prompt Is Not the Author
prvsiyan
—10311-11 19:15
Name That LLM v5
Zaphyer Modal
—8711-02 01:22
Qwen LoRA + Multi-View Ensemble
Sargis Khachatryan
—8510-26 11:11

07 / Day-by-day blotter

WHAT WE TRIED, WHAT HAPPENED · NEWEST FIRST · 13 DAYS
DateVerdictTriedWhat happenedSpent
11 NovThe three-pass Qwen model finished and was picked for the next Kaggle test run. The public 0.684 recipe is confirmed real. The style and GPT-correction ideas added nothing.$13.21
lostFinished training the three-pass version of our Qwen model and scored it on its own against the one-pass model.By itself it came out lower, 0.609 against 0.621.
workedSwapped the three-pass model in for the one-pass model at 10 percent of the ensemble and checked it on two sets of creative writing.It beat the one-pass model on 16 of 18 creative comparisons, so we chose it for the next Kaggle test run.
workedChecked the new public 0.684 recipe.Two public copies run identical code and the page shows 0.684. Our best is still 0.682.
no gainTrained bigger style networks on 245,081 passages.On the official passages they beat the old style model, 0.324 against 0.278. On the two creative sets the whole-passage network did worse than the old model.
dead endAdded the style networks to the ensemble, then added word-order style features on top.The style networks added about four tenths of a point on the main test. The word-order features made it worse, so we dropped them.
no gainAdded a correction that separates Qwen from the GPT models it gets confused with.Scores moved by less than a point either way: up a little on the independent test sets, flat to slightly down on the official folds. We did not use it.
workedFinished training the Liquid model with three separate runs and mixed it into an earlier public blend.It added about half a point at a 5 percent share and about three quarters of a point at 10 percent on our local test. It has not been on the board.
10 NovOur best board score moved up to 0.682 by running the public recipe as it is. Every one of our models mixed in scored lower.$12.34
lostSubmitted the public 0.682 recipe by itself, then three versions with our models mixed in: our tokenizer model, our earlier Qwen, and tokenizer plus raw-byte models.The plain recipe scored 0.682, our new best. The three additions scored 0.675, 0.678 and 0.669, so all three lost.
lostSubmitted the public 0.683 recipe by itself, then the same recipe with our new full-pool Qwen mixed in at 10 percent.The plain copy scored 0.679 and the Qwen version 0.675, so Qwen lost. Our best stays 0.682.
workedTrained our Qwen model on the whole 253,598-passage public pool for one pass on a rented GPU.Our local five-fold score rose from 0.587 to 0.621. Two earlier rented hosts failed before training, about 90 cents lost, and the third host finished the run.
dead endRan the 540-row comparison of our model against the full public 0.681 stack on Kaggle.It took three tries after setup and file-path crashes. The result could not judge our model: the public stack had already seen those passages and scored 0.978 alone, and blending ours in only lowered it.
workedTested the shared Mistral and OLMo predictions against our stack.Mistral helped most, +0.018 at a 20 percent share on our local test. OLMo added +0.009. Running the public Mistral model on Kaggle got as far as three test rows.
workedBuilt token and raw-byte specialist models for the five GPT authors.The byte model added about four tenths of a point inside the GPT group on our local test, 0.579 to 0.583. We have not put it on the board.
no gainRetested the 6,000 passages we bought from six models, using three separate training runs.The score moved by -0.001, -0.006 and +0.007 across the three runs, with no consistent gain.
no gainTested GLM and Qwen likelihood fingerprints as extra features.Most versions made the score slightly worse.
builtGot the large GLM model running on a rented GPU.It loaded and read three full passages on the fourth attempt, for about 54 cents. The rental was shut down.
9 NovWe reproduced the public 0.681 notebook and submitted it. Our model helped an earlier public blend on our local test. The tokenizer signal added almost nothing.
workedRan the public 0.681 notebook in full on Kaggle's free GPUs and submitted it that night.It scored 0.681, up from 0.651.
workedBlended our own model into an earlier public ensemble on all 2,699 training passages.Balanced accuracy rose from 0.543 to 0.613 in the three-response setting, and recall rose for all 18 models. On the real board, mixing our Qwen into the 0.682 recipe the next day lost.
no gainTested tokenizer fingerprints, meaning how different tokenizers split a passage, as a 1 percent blend.It was weak on its own and moved the earlier ensemble from 0.5431 to 0.5438. The tokenizer idea later lost on the board.
5 NovAn outside review said we spent ten days on checks instead of the board, and ranked what to do instead.
builtHad a team of reviewers go through the whole repository and propose a better order of work.We held 0.651 against a leader at 0.730 and had made no submission since October 31. The plan puts a safe Kaggle submission first, then the group-context idea, then new model ideas, and recommends closing the extra-data work.
4 NovThe clean-pool result held up on Kaggle's own machine. The 6,000 bought passages lost again on a second training run.
workedRan the clean-pool fold on Kaggle's CPU machine and compared it with the result from our own machine.The two matched except for one prediction, and that one was wrong on both. The result carries over to Kaggle.
lostFinished the second training run on Jethro that compares the 6,000 bought passages against the combined-data control.The bought passages scored 0.580 against 0.586 for the control.
3 NovThe memory crash was repaired and the full ensemble ran on Son's GPU machine. The extra data's small gain vanished when three training runs were averaged. Group context helped on whole groups and hurt on pairs.
workedTested a group-aware scoring head that judges passages from the same prompt together.It gained about six points when whole groups were present and lost about three points when groups were split into pairs. We did not switch it on everywhere. A rule based on group size is next.
workedFinished the four-way data comparison across three training runs.The combined data beat the plain data in every run, by 0.001 to 0.009.
no gainAveraged the three runs of the combined-data model and compared it with the average of the plain-data runs.The combined data scored 0.6005 against 0.6016 for plain. The gain disappeared once averaged.
workedRepaired the adapter memory step that crashed on Kaggle.The repaired step gives exactly the same results as the original on real model weights.
workedRan the whole upgraded public ensemble on Son's RTX 4090 machine, on the three visible test passages.Every stage finished, including all six main models and all 18 adapters. This was not a board score and not a test of Kaggle's two-card machine.
workedSwapped a cleaner public data pool into the word-pattern model and scored it on all five folds.By itself it rose from 0.490 to 0.512.
no gainBlended the word-pattern model on the cleaner pool with the three-run neural models.The gain shrank to 0.613 to 0.616 once blended.
lostRan the cleaner data pool on Kaggle's CPU machine.Kaggle attached a newer version of the pool than the one we asked for, and the run stopped on a file check. We saved the exact pool as a private dataset of our own.
dead endPulled the source of the new public 0.667 notebook.Kaggle refused the download. We could only read the rendered page.
2 NovThe upgraded public ensemble ran on Kaggle and crashed on memory. The 6,000 bought passages did not help on the first training run.
lostRan the upgraded public ensemble on Kaggle's two-card GPU machine, without submitting it for a score.It crashed with out-of-memory on the second card after its side models had finished. No score came out of it.
builtBuilt the upgraded public ensemble into our own runner and checked it on the real competition data.All eight checks passed, including empty and one-row inputs.
no gainCompared our model trained with the 6,000 bought passages against the plain version, same training run, same held-out prompts.The bought passages scored 0.594 against 0.595. No gain.
31 OctOur own trained model scored 0.585 on Kaggle's hidden test, and we found why Jethro's training math was slow.
workedSubmitted our own Qwen3-8B model alone to check the whole pipeline.It scored 0.585 on the hidden test, close to its 0.588 local score. The public ensemble at 0.651 stayed our best.
workedHunted for why training on Jethro was slow.The old math libraries used a slow route. The newer installed ones ran the key steps six to eight times faster in tests.
30 OctSix thousand new passages collected, and training moved onto Jethro.$6.90
workedBought 6,000 new passages, 1,000 from each of six models, after a sixty-passage trial.All 6,000 arrived for about $7.10, well under the $18 limit.
workedCleared disk space on Jethro and started training on the expanded data.About 186 gigabytes were freed and training started on the new data.
29 OctOur first self-trained model scored 0.588 against 0.480 for the public recipe on the same test folds.
workedTrained our own Qwen3-8B model in two stages, starting with a large public pool of writing.It scored 0.588 on held-back topics, against 0.480 for the public recipe on the same folds.
no gainAdded 1,752 Qwen passages we bought through OpenRouter to the word-pattern model.No help: it scored 0.481 with them and 0.484 without. About $7.72 was spent on 2,499 requests, and 1,752 of the replies were usable.
builtRepaired the training script and ran it on the real Qwen model on Son's RTX 4090.The test passed, and the twelve-run Qwen data comparison started that evening on the same card.
builtHad an independent review check our numbers.It found the 0.588 against 0.480 comparison was not a clean test, because the training run also differed in settings. It could not reproduce an earlier blend figure of 0.611 and got 0.603. It asked for a matched control.
28 OctWe went looking for new training writing, from Claude sessions and from GPT.
builtMined the Mac Mini's saved Claude Code replies for text from Claude models.Found 2,124 passages labelled with the right model names.
no gainTrained our classifier with those Claude replies and saved Astra replies added.Held-out accuracy rose only from 0.331 to 0.338 and several authors got worse, so the scored entry stayed as it was.
workedMeasured whether each author's use of dashes helps tell authors apart.Dashes alone scored 0.137. Added to the classifier they lifted local accuracy from 0.331 to 0.344. We kept it aside and did not put it in the scored entry.
workedTested whether the Mac can collect writing from GPT models through the ChatGPT subscription.GPT-5.6 Luna, Sol and Terra answered. GPT-5.5 was refused as unsupported.
27 OctTwo real submissions: a weak classifier of our own at 0.348, and the full public neural ensemble at 0.651.
workedSubmitted a classifier we built ourselves, to prove an entry of ours can run on Kaggle's hidden test.It ran clean and scored 0.348, far below the public leaders.
workedRan the strong public neural ensemble in full and submitted it.Kaggle accepted it and the score came back at 0.651.
workedAdded writing-style features to our own classifier.Local accuracy went from 0.275 to 0.331, which is why it was the version we submitted.
26 OctWe split out our own repo and audited why both earlier Kaggle tries crashed.
lostLooked into why our first two Kaggle submissions came back with no score.Both crashed on the hidden test. Both carried extra wrapper code the original notebook did not have, including a guard that rejects small test inputs.
builtRebuilt the entry as an exact copy of the best public notebook, which scores 0.651.The copy matches the original byte for byte.
builtSplit Name That LLM out of the old shared repo into its own standalone home.The setup checks pass on the real training data.

08 / Capital ledger

EVERY RENTAL AND DATA PURCHASE · POSTED CHARGES
DateWhat it paid forKindUSD
11 NovQwen three-pass training runGPU rental$13.21
10 NovQwen one-pass training runGPU rental$4.86
10 NovTwo Qwen hosts that failedGPU rental$0.90
10 NovGLM test on a rented card, four attemptsGPU rental$0.54
10 NovCreative-writing passages from other modelsData generation$6.05
30 Oct6,000 generated passagesData generation$6.90
Total$32.45
Qwen rental cap $18.96 / $24
GLM rental cap $0.54 / $10
Creative data cap $6.05 / $10

09 / Next

QUEUE
  1. Submit the three-pass Qwen on top of the public 0.684 recipe, against the plain recipe.
  2. Train our own Mistral model, the strongest different-model idea so far.
  3. Build our own pruned GLM and fine-tune it (paused for now).

Ideas from the forum

HIGH-RELEVANCE THREADS
  • somerandomguy says the time limit counts from kernel start, not queue, use the competition-rerun flag to detect scoring runs, and keep about half an hour spare. Check our notebooks. thread ↗
  • Vijay Konda says GLM-5.3 likelihood only fits the time limit by streaming one layer at a time over all test passages; John K shows GLM and Claude labels heavily confused. Worth a test. thread ↗
  • Renan Pinheiro says random folds score 0.11 but folds grouped by prompt score 0.35 (board 0.355); most errors stay in the right family. Use grouped folds and try a family-then-model step. thread ↗
  • John K says train has 18 answers on 149 of 150 prompts, but test groups are not full and model counts differ. Never force one-of-each-model per prompt at test time. thread ↗
  • Best single models: Horikita Saku and Xiao say 0.65 to 0.68 with Qwen3-14B and own data; GoPain says Qwen3-8B LoRA 0.628, ensemble adds under a point. Most gains came from data, so more clean data beats more blending. thread ↗
  • Carrot Production says a quarter of the public pool was junk or wrong genre, and cleaning it (Open Pool v9) gave +0.027. Nobody has replied. Try that cleaning on our full-pool Qwen training. thread ↗
  • Harshita: swapping old Qwen2.5-14B for a higher-score retrain cost board points; HALX2158 says diverse weak members matter. GoPain: centred-head weight over 1.0 lost. Add diversity, not clones. thread ↗
  • Carrot Production updates the Open Pool to v14, with Kimi relabelled and wrong rows removed. GoPain shows a board gap of 0.002 between two models is noise. Train on v14 and judge gains by our own cross-validation. thread ↗
  • Aleksey Osipenko says early Qwen layers predict the author better than late ones, and top public work cuts Qwen3-8B to 24 layers and pools several. About half the errors are within one model family. thread ↗
  • GoPain says grouped CV reads 0.03-0.04 high vs LB once the pool is in training, and class balancing gave +0.011 on LB. Our pool-trained local gains are inflated; keep balancing. thread ↗