0.68212 submissions
614of 5,669 · -177 since 11 Nov
top 10.8%top-10% line 0.683
0.769NguyenThanhNhan
-0.087to top 10: -0.065
5724 worked · 11 lost · 10 flat
$32.45GPU $19.50 · data $12.95
0.684+0.002 vs our best
01 / Board race
OUR ENTRIES (S#, SEE LEDGER) VS LEADER AND TOP-10 CUTOFFOur best so farOur entryEntry below its controlCrashed (no score)Board leaderTop-10 cutoff
02 / Where the field sits
ALL 5,669 TEAMS · 2026-11-12 16:30 ET03 / Board ledger
EVERY SUBMISSION · CHANGE VS ITS MATCHED CONTROL| # | Date | What went in | Score | vs control |
|---|---|---|---|---|
| S12 | 10 Nov | Public 0.683 recipe, control The public 0.683 recipe run unchanged on our account. The author reported 0.683 and we got 0.679, 0.003 under our best. | 0.679 | — |
| S11 | 10 Nov | Full-pool Qwen added to the 0.683 recipe The public 0.683 recipe with our new full-pool Qwen model mixed in at ten percent. It lost to the same recipe without it. | 0.675 | -0.004 |
| S10 | 10 Nov | Tokenizer and byte models on the 0.682 recipe The public 0.682 recipe with ten percent tokenizer models and twenty percent raw-byte models mixed in. It lost. | 0.669 | -0.013 |
| S9 | 10 Nov | Public 0.682 recipe, control (our best) The public 0.682 recipe run unchanged on our account. It matched the author's score and is our best entry. | 0.682 | — |
| S8 | 10 Nov | Earlier Qwen added to the 0.682 recipe The public 0.682 recipe with ten percent of our earlier Qwen3-8B model mixed in. It lost. | 0.678 | -0.004 |
| S7 | 10 Nov | Tokenizer model added to the 0.682 recipe The public 0.682 recipe with ten percent of our tokenizer model mixed in. It lost. | 0.675 | -0.007 |
| S6 | 9 Nov | Public 0.681 ensemble run A full run of the public 0.681 ensemble on our account. It beat our old best of 0.651 by 0.030. | 0.681 | — |
| S5 | 31 Oct | Our own Qwen model alone Our own trained Qwen3-8B model submitted by itself to test the pipeline end to end. It scored 0.585, close to its 0.588 local score and well below the public ensemble. | 0.585 | — |
| S4 | 27 Oct | First public neural ensemble The public neural ensemble, the 0.651 recipe, run in full on our account. Our first strong board score. | 0.651 | — |
| S3 | 27 Oct | Our own CPU classifier Our own character, word and style classifier, trained on the official labels, to prove an entry of ours runs on the hidden test. | 0.348 | — |
| S2 | 26 Oct | Public baseline, second try A second try at the public 0.651 notebook, still with our extra wrapper code. It crashed on the hidden test and earned no score. | crashed | — |
| S1 | 26 Oct | Public baseline, first try The first try at the public 0.651 notebook, with our extra wrapper code. It crashed on the hidden test and earned no score. | crashed | — |
04 / Board watch
TOP 15 TEAMS AND US · GAP TO OUR BEST| # | Team | Score | vs us | Subs | Last entry (UTC) |
|---|---|---|---|---|---|
| 1 | NguyenThanhNhan | 0.769 | +0.087 | 14 | 2026-11-12 11:18 |
| 2 | James Liebel | 0.765 | +0.083 | 35 | 2026-11-11 13:37 |
| 3 | You’re Absolutely Right! | 0.756 | +0.074 | 78 | 2026-11-12 00:02 |
| 4 | Le Viet | 0.752 | +0.070 | 111 | 2026-11-12 07:33 |
| 5 | Meni David Gottesman | 0.752 | +0.070 | 103 | 2026-11-12 08:18 |
| 6 | Keanan | 0.750 | +0.068 | 68 | 2026-11-12 01:26 |
| 7 | Harveen Singh Chadha | 0.750 | +0.068 | 58 | 2026-11-11 21:50 |
| 8 | kulshekhar | 0.748 | +0.066 | 81 | 2026-11-12 07:37 |
| 9 | Team Aillis | 0.747 | +0.065 | 130 | 2026-11-12 06:26 |
| 10 | Lian Matsuo | 0.747 | +0.065 | 49 | 2026-11-12 00:17 |
| 11 | Hod Carriers | 0.747 | +0.065 | 66 | 2026-11-04 22:38 |
| 12 | Søren Ravn Andersen | 0.744 | +0.062 | 81 | 2026-11-12 09:25 |
| 13 | jad kutbi | 0.742 | +0.060 | 71 | 2026-11-12 00:02 |
| 14 | Civitasmass | 0.740 | +0.058 | 68 | 2026-11-11 15:34 |
| 15 | Liaaar | 0.737 | +0.055 | 45 | 2026-11-11 19:36 |
| 614 | Mark Barney (us) | 0.682 | — | 12 | 2026-11-11 00:00 |
Movers since last snapshot
TOP 25| 13 | jad kutbi | 0.742 | entered top 25 |
| 21 | Yevhenii Maslov | 0.733 | entered top 25 |
| 7 | Harveen Singh Chadha | 0.750 | +4 places, +0.004 |
| 18 | t_shiga | 0.737 | +3 places, +0.007 |
| 15 | Liaaar | 0.737 | +1 places, +0.001 |
| 1 | NguyenThanhNhan | 0.769 | +0 places, +0.001 |
| 12 | Søren Ravn Andersen | 0.744 | +0 places, +0.001 |
| 17 | Nishant Dahal | 0.737 | +0 places, +0.002 |
| 23 | Lukas | 0.732 | +0 places, +0.006 |
| 8 | kulshekhar | 0.748 | -1 places, +0.000 |
| 9 | Team Aillis | 0.747 | -1 places, +0.000 |
| 14 | Civitasmass | 0.740 | -1 places, +0.000 |
05 / Discussion board
KAGGLE FORUM · NEWEST FIRST · OUR TAKE IN GREY| Posted (UTC) | Topic | Comments | Votes |
|---|---|---|---|
| 11-12 18:41 | NEW why are we helping in devloping watermarks agbuddy agbuddy asks why the contest helps develop watermarks; saved thread has only its title and no replies, so no research finding is established. | 0 | 0 |
| 11-12 14:05 | NEW Test set structure: passages per prompt? Thierry Nyembo Thierry Nyembo asks about passages per test prompt; saved thread has only its title and no replies. Do not assume a fixed hidden prompt-group structure. | 0 | 0 |
| 11-12 05:18 | NEW From 0.684 toward top 1%: what validation evidence predicts real gains? | 0 | 2 |
| 11-12 02:39 | NEW Reading speed for large model files | 0 | 0 |
| 11-12 02:39 | NEW Scoring-run hardware | 0 | 0 |
| 11-11 23:47 | Reason for qwen performing so well in classifying :/ Gaurav Rawat asks why Qwen classifies so well. The saved file has only the title and no replies. | 0 | 1 |
| 11-11 13:02 | Late post-processing cell never seems to affect the score in the hidden rerun — known differences vs. commit runs? somerandomguy says the time limit counts from kernel start, not queue, use the competition-rerun flag to detect scoring runs, and keep about half an hour spare. Check our notebooks. | 4 | 0 |
| 11-11 00:08 | Notes from a solo pipeline: grouped CV, why prompt-group tricks didn't transfer, typographic invariants Only the title was saved: pedrohbenetton's notes on fold splits by prompt, prompt-group tricks that did not carry over to test, and punctuation clues. Body missing; fetch it before relying on it. | 0 | 2 |
| 11-10 18:08 | Can we use Kaggle Benchmarks model access to generate training data? Only reply: Darragh1Walsh links a dataset he made with Kaggle Benchmarks model access and invites use. A free extra data source, but check it matches the host format first. | 1 | 1 |
| 11-09 13:55 | Can a CatBoost-based approach be competitive in this competition? Maxim Stepanov asks if a CatBoost approach can compete; the only reply is 'wondering the same'. No answer. | 1 | 6 |
| 11-09 07:38 | Running GLM-5.3 / GLM-5.3-Flash inside the 9h limit — has anyone got the public GGUF segments working? Vijay Konda says GLM-5.3 likelihood only fits the time limit by streaming one layer at a time over all test passages; John K shows GLM and Claude labels heavily confused. Worth a test. | 5 | 6 |
| 11-09 02:09 | Model Parameterizations Title only (James Bedichek, model parameterizations), no body or replies saved. Nothing to act on. | 0 | 1 |
| 11-07 20:46 | Under-18 participation and consent procedure Under-18 consent procedure question, no replies. Not relevant to us. | 0 | 1 |
| 11-06 14:00 | Random K-fold is pessimistic here: 0.11 vs 0.35 with grouped folds (reverse leakage) Renan Pinheiro says random folds score 0.11 but folds grouped by prompt score 0.35 (board 0.355); most errors stay in the right family. Use grouped folds and try a family-then-model step. | 2 | 2 |
| 11-06 12:55 | Clarification on external datasets built by sending competition text to third-party LLM services subuta_umee asks if outside data made by sending competition text to other LLM services is allowed; no official answer, one unhelpful reply. Rule still unclear. | 1 | 1 |
| 11-05 22:00 | Looking to merge: unique Mistral/OLMo LoRAs with full OOF + fresh Gemini data (public 0.663) Emmanuel Ocheme offers Mistral and OLMo LoRAs (public 0.663). Anto Pil says they add only 0.004 to his 0.666 blend and declines to merge. His and Emmanuel's fold predictions are public; test them in our blend. | 4 | -3 |
| 11-05 11:09 | Same ablations on two bases: what each part is worth on the public LB (0.638 vs 0.659) Empty in our copy: title promises per-part public LB ablations on two bases (0.638 vs 0.659), but no body or comments were saved. Worth fetching the post itself. | 0 | 1 |
| 11-05 03:07 | Prize Winner Requirements – Name That LLM radidani's post on prize winner requirements. The saved file has only the title, no body or replies. | 0 | 1 |
| 11-05 00:14 | Stylometric baseline + grouped CV for Name That LLM (notebook) Matin shirazi shares a style baseline with grouped cross-validation notebook; no comments. Our style models already cover this. | 0 | 1 |
| 11-04 17:28 | Public Score: 0.333 Mohammed Hasan Abdullah Title only (Mohammed Hasan Abdullah, public 0.333), no body or replies saved. Nothing useful for us. | 0 | 0 |
| 11-04 13:26 | Prompt Group Sizes: Is it always exactly 18 texts per prompt? ナハト・ ファウスト John K says train has 18 answers on 149 of 150 prompts, but test groups are not full and model counts differ. Never force one-of-each-model per prompt at test time. | 5 | 3 |
| 11-04 03:34 | Name That LLM: grouped validation, stylistic baselines and unsuccessful neural experiments cylovelife cylovelife's write-up on grouped validation, style baselines and failed neural runs. The saved file has only the title, no body or replies. Nothing to act on. | 0 | 1 |
| 11-03 23:51 | only 3 rows in the test data Marie Victoire IPOUK Newcomer asks why test has 3 rows; somerandomguy says that is a sample and the hidden test has about 4000 rows. Known already. | 2 | 0 |
| 11-03 17:34 | My approach for DistilBERT fine-tuning (0.2515 OOF) Qhaleesh Qhaleesh posts a DistilBERT baseline at 0.25. OsakanaTeishoku's only reply says to split folds by prompt, since test prompts are unseen. We already do that; nothing new for us. | 1 | -2 |
| 11-03 11:49 | Will self-generated extra data help or hurt here? Ruslan Akhmetzianov somerandomguy says his self-generated data lowered val even after matching structure and length; Soham Acharya fears topic bias. Supports not paying for our own generation. | 4 | 3 |
| 11-03 07:38 | My classifier thinks GLM-5.3 is more Claude than GLM AyoGG AyoGG says their classifier calls GLM-5.3 Claude; commenters blame distillation or routing to Claude. This confusion pair is a likely error spot to check in our results. | 4 | 5 |
| 11-03 01:22 | Any real humans left in here? Civitasmass Joke thread about LLM-written posts on the forum; one user says he is stuck around 0.66. Nothing useful for us. | 11 | 16 |
| 11-02 04:23 | What score your best single model can achieve? Jak Ma Best single models: Horikita Saku and Xiao say 0.65 to 0.68 with Qwen3-14B and own data; GoPain says Qwen3-8B LoRA 0.628, ensemble adds under a point. Most gains came from data, so more clean data beats more blending. | 26 | 15 |
| 10-29 23:25 | New external dataset: 9,719 OpenAI responses with exact prompts and provenance Enai Comments are just a team-up request (bosprimigenious, LB 0.349) and an upvote. Nothing about whether the 9,719 OpenAI-response dataset helps. | 2 | 16 |
| 10-29 09:45 | [BIG UPDATE - please try it] Same data, +0.027 accuracy: 1 in 4 passages in the public NTL pool was junk or the wrong genre. I cleaned it (Open Pool v9) Carrot Production Carrot Production says a quarter of the public pool was junk or wrong genre, and cleaning it (Open Pool v9) gave +0.027. Nobody has replied. Try that cleaning on our full-pool Qwen training. | 0 | 8 |
| 10-27 11:52 | Retrained the Qwen2.5-14B member of nlm-v27 on the 120k pool: OOF 0.569 → 0.605, public LB 0.651 → 0.647. Why? Harshita Kumari Harshita: swapping old Qwen2.5-14B for a higher-score retrain cost board points; HALX2158 says diverse weak members matter. GoPain: centred-head weight over 1.0 lost. Add diversity, not clones. | 7 | 8 |
| 10-26 07:59 | [NOW UPDATED]## Open Pool v14: real `kimi-k3` was written with reasoning on, `glm-5.3` is fine, and 284 mislabelled rows are gone Carrot Production Carrot Production updates the Open Pool to v14, with Kimi relabelled and wrong rows removed. GoPain shows a board gap of 0.002 between two models is noise. Train on v14 and judge gains by our own cross-validation. | 18 | 85 |
| 10-22 16:08 | Anyone tried a Hierarchical Approach? (Predicting Provider first, then Specific Model) Ghaith Mechi Only answer: Cloud9 links an arXiv paper (ATLAS) that predicts the provider well. No one reports results for provider-then-model here. Nothing to act on. | 5 | 8 |
| 10-20 13:59 | Section 2.6 question: training data generated from the candidate models Adisak Sukul Adisak Sukul asks six questions on training data made from the candidate models; rw-nue asks about a spending cap and release deadline. No host answer, so the rule stays unclear. | 1 | 10 |
| 10-19 12:29 | What the punctuation already tells you about the 18 models (and what it can't) Masood Ahmed HALX2158 says Gemini never em-dashes in train but proxy-made Gemini text does (13-43%); Nikhil sees Claude and GLM confused. Punctuation rules may flip if we add outside Gemini data. | 10 | 19 |
| 10-18 12:02 | Early Qwen layers know the author better than late ones: 0.39 vs 0.31 Aleksey Osipenko Aleksey Osipenko says early Qwen layers predict the author better than late ones, and top public work cuts Qwen3-8B to 24 layers and pools several. About half the errors are within one model family. | 7 | 9 |
| 10-18 04:55 | New here? Grouped CV that tracks the LB, CPU baseline code (LB 0.353), pitfalls, 2xT4 tips Adisak Sukul GoPain says grouped CV reads 0.03-0.04 high vs LB once the pool is in training, and class balancing gave +0.011 on LB. Our pool-trained local gains are inflated; keep balancing. | 5 | 24 |
| 10-17 14:17 | Useful helper dataset prvsiyan prvsiyan shares a helper dataset; Darragh1Walsh adds one he updates daily. Commenters doubt its labels because it came through an unofficial proxy. Use only if checked against official data. | 13 | 55 |
| 10-17 10:56 | [Dataset] Extra Claude passages on new prompts, and what has worked with them so far Seyit kaan Gunes Zbigniew Kirchner says extra Claude passages in a side Claude head on a Qwen3-8B LoRA lifted val 0.456 to 0.493, and a lab-then-version hierarchy gave +2.4. Cheap Claude-split idea to try. | 3 | 10 |
| 10-16 11:18 | Could the Organizers Provide More Official Training Data? Yan Zhou Yan Zhou asks the hosts for more official training data. buysider asks whether self-generated text from model providers is even allowed; Yan doubts hosts will answer. No ruling given. | 2 | 10 |
06 / Public notebooks
CLAIMED TITLE SCORE · NOTEBOOKS AS OF 2026-11-11 20:32 ET07 / Day-by-day blotter
WHAT WE TRIED, WHAT HAPPENED · NEWEST FIRST · 14 DAYS| Date | Verdict | Tried | What happened | Spent |
|---|---|---|---|---|
| 12 Nov | Best unchanged; the field moved ahead. Costs remain cached, not a fresh bill. | |||
| built | Checked the public leaderboard. | Best 0.682; rank 614 of 5,669 at 16:30 EST. Leader 0.769. | ||
| lost | Reviewed the completed three-pass Qwen experiment. | Standalone 0.609 versus one-pass 0.621; no new board improvement saved. | ||
| built | Reconciled selected rental charges by exact project instance. | Cached $19.499: Qwen 11 Nov 08:58 EST; GLM 10 Nov 14:25 EST. Later charges unknown. | ||
| built | Checked new forum metadata and two changed threads. | Two title-only questions; three thread reads deferred. Notebook listing cached 11 Nov 20:32 EST. | ||
| built | Checked local process state. | 18:01 EST: no local watcher or research file sync found; remote training unverified. No jobs started. | ||
| 11 Nov | The three-pass Qwen model finished and was picked for the next Kaggle test run. The public 0.684 recipe is confirmed real. The style and GPT-correction ideas added nothing. | $13.21 | ||
| lost | Finished training the three-pass version of our Qwen model and scored it on its own against the one-pass model. | By itself it came out lower, 0.609 against 0.621. | ||
| worked | Swapped the three-pass model in for the one-pass model at 10 percent of the ensemble and checked it on two sets of creative writing. | It beat the one-pass model on 16 of 18 creative comparisons, so we chose it for the next Kaggle test run. | ||
| worked | Checked the new public 0.684 recipe. | Two public copies run identical code and the page shows 0.684. Our best is still 0.682. | ||
| no gain | Trained bigger style networks on 245,081 passages. | On the official passages they beat the old style model, 0.324 against 0.278. On the two creative sets the whole-passage network did worse than the old model. | ||
| dead end | Added the style networks to the ensemble, then added word-order style features on top. | The style networks added about four tenths of a point on the main test. The word-order features made it worse, so we dropped them. | ||
| no gain | Added a correction that separates Qwen from the GPT models it gets confused with. | Scores moved by less than a point either way: up a little on the independent test sets, flat to slightly down on the official folds. We did not use it. | ||
| worked | Finished training the Liquid model with three separate runs and mixed it into an earlier public blend. | It added about half a point at a 5 percent share and about three quarters of a point at 10 percent on our local test. It has not been on the board. | ||
| 10 Nov | Our best board score moved up to 0.682 by running the public recipe as it is. Every one of our models mixed in scored lower. | $12.34 | ||
| lost | Submitted the public 0.682 recipe by itself, then three versions with our models mixed in: our tokenizer model, our earlier Qwen, and tokenizer plus raw-byte models. | The plain recipe scored 0.682, our new best. The three additions scored 0.675, 0.678 and 0.669, so all three lost. | ||
| lost | Submitted the public 0.683 recipe by itself, then the same recipe with our new full-pool Qwen mixed in at 10 percent. | The plain copy scored 0.679 and the Qwen version 0.675, so Qwen lost. Our best stays 0.682. | ||
| worked | Trained our Qwen model on the whole 253,598-passage public pool for one pass on a rented GPU. | Our local five-fold score rose from 0.587 to 0.621. Two earlier rented hosts failed before training, about 90 cents lost, and the third host finished the run. | ||
| dead end | Ran the 540-row comparison of our model against the full public 0.681 stack on Kaggle. | It took three tries after setup and file-path crashes. The result could not judge our model: the public stack had already seen those passages and scored 0.978 alone, and blending ours in only lowered it. | ||
| worked | Tested the shared Mistral and OLMo predictions against our stack. | Mistral helped most, +0.018 at a 20 percent share on our local test. OLMo added +0.009. Running the public Mistral model on Kaggle got as far as three test rows. | ||
| worked | Built token and raw-byte specialist models for the five GPT authors. | The byte model added about four tenths of a point inside the GPT group on our local test, 0.579 to 0.583. We have not put it on the board. | ||
| no gain | Retested the 6,000 passages we bought from six models, using three separate training runs. | The score moved by -0.001, -0.006 and +0.007 across the three runs, with no consistent gain. | ||
| no gain | Tested GLM and Qwen likelihood fingerprints as extra features. | Most versions made the score slightly worse. | ||
| built | Got the large GLM model running on a rented GPU. | It loaded and read three full passages on the fourth attempt, for about 54 cents. The rental was shut down. | ||
| 9 Nov | We reproduced the public 0.681 notebook and submitted it. Our model helped an earlier public blend on our local test. The tokenizer signal added almost nothing. | |||
| worked | Ran the public 0.681 notebook in full on Kaggle's free GPUs and submitted it that night. | It scored 0.681, up from 0.651. | ||
| worked | Blended our own model into an earlier public ensemble on all 2,699 training passages. | Balanced accuracy rose from 0.543 to 0.613 in the three-response setting, and recall rose for all 18 models. On the real board, mixing our Qwen into the 0.682 recipe the next day lost. | ||
| no gain | Tested tokenizer fingerprints, meaning how different tokenizers split a passage, as a 1 percent blend. | It was weak on its own and moved the earlier ensemble from 0.5431 to 0.5438. The tokenizer idea later lost on the board. | ||
| 5 Nov | An outside review said we spent ten days on checks instead of the board, and ranked what to do instead. | |||
| built | Had a team of reviewers go through the whole repository and propose a better order of work. | We held 0.651 against a leader at 0.730 and had made no submission since October 31. The plan puts a safe Kaggle submission first, then the group-context idea, then new model ideas, and recommends closing the extra-data work. | ||
| 4 Nov | The clean-pool result held up on Kaggle's own machine. The 6,000 bought passages lost again on a second training run. | |||
| worked | Ran the clean-pool fold on Kaggle's CPU machine and compared it with the result from our own machine. | The two matched except for one prediction, and that one was wrong on both. The result carries over to Kaggle. | ||
| lost | Finished the second training run on Jethro that compares the 6,000 bought passages against the combined-data control. | The bought passages scored 0.580 against 0.586 for the control. | ||
| 3 Nov | The memory crash was repaired and the full ensemble ran on Son's GPU machine. The extra data's small gain vanished when three training runs were averaged. Group context helped on whole groups and hurt on pairs. | |||
| worked | Tested a group-aware scoring head that judges passages from the same prompt together. | It gained about six points when whole groups were present and lost about three points when groups were split into pairs. We did not switch it on everywhere. A rule based on group size is next. | ||
| worked | Finished the four-way data comparison across three training runs. | The combined data beat the plain data in every run, by 0.001 to 0.009. | ||
| no gain | Averaged the three runs of the combined-data model and compared it with the average of the plain-data runs. | The combined data scored 0.6005 against 0.6016 for plain. The gain disappeared once averaged. | ||
| worked | Repaired the adapter memory step that crashed on Kaggle. | The repaired step gives exactly the same results as the original on real model weights. | ||
| worked | Ran the whole upgraded public ensemble on Son's RTX 4090 machine, on the three visible test passages. | Every stage finished, including all six main models and all 18 adapters. This was not a board score and not a test of Kaggle's two-card machine. | ||
| worked | Swapped a cleaner public data pool into the word-pattern model and scored it on all five folds. | By itself it rose from 0.490 to 0.512. | ||
| no gain | Blended the word-pattern model on the cleaner pool with the three-run neural models. | The gain shrank to 0.613 to 0.616 once blended. | ||
| lost | Ran the cleaner data pool on Kaggle's CPU machine. | Kaggle attached a newer version of the pool than the one we asked for, and the run stopped on a file check. We saved the exact pool as a private dataset of our own. | ||
| dead end | Pulled the source of the new public 0.667 notebook. | Kaggle refused the download. We could only read the rendered page. | ||
| 2 Nov | The upgraded public ensemble ran on Kaggle and crashed on memory. The 6,000 bought passages did not help on the first training run. | |||
| lost | Ran the upgraded public ensemble on Kaggle's two-card GPU machine, without submitting it for a score. | It crashed with out-of-memory on the second card after its side models had finished. No score came out of it. | ||
| built | Built the upgraded public ensemble into our own runner and checked it on the real competition data. | All eight checks passed, including empty and one-row inputs. | ||
| no gain | Compared our model trained with the 6,000 bought passages against the plain version, same training run, same held-out prompts. | The bought passages scored 0.594 against 0.595. No gain. | ||
| 31 Oct | Our own trained model scored 0.585 on Kaggle's hidden test, and we found why Jethro's training math was slow. | |||
| worked | Submitted our own Qwen3-8B model alone to check the whole pipeline. | It scored 0.585 on the hidden test, close to its 0.588 local score. The public ensemble at 0.651 stayed our best. | ||
| worked | Hunted for why training on Jethro was slow. | The old math libraries used a slow route. The newer installed ones ran the key steps six to eight times faster in tests. | ||
| 30 Oct | Six thousand new passages collected, and training moved onto Jethro. | $6.90 | ||
| worked | Bought 6,000 new passages, 1,000 from each of six models, after a sixty-passage trial. | All 6,000 arrived for about $7.10, well under the $18 limit. | ||
| worked | Cleared disk space on Jethro and started training on the expanded data. | About 186 gigabytes were freed and training started on the new data. | ||
| 29 Oct | Our first self-trained model scored 0.588 against 0.480 for the public recipe on the same test folds. | |||
| worked | Trained our own Qwen3-8B model in two stages, starting with a large public pool of writing. | It scored 0.588 on held-back topics, against 0.480 for the public recipe on the same folds. | ||
| no gain | Added 1,752 Qwen passages we bought through OpenRouter to the word-pattern model. | No help: it scored 0.481 with them and 0.484 without. About $7.72 was spent on 2,499 requests, and 1,752 of the replies were usable. | ||
| built | Repaired the training script and ran it on the real Qwen model on Son's RTX 4090. | The test passed, and the twelve-run Qwen data comparison started that evening on the same card. | ||
| built | Had an independent review check our numbers. | It found the 0.588 against 0.480 comparison was not a clean test, because the training run also differed in settings. It could not reproduce an earlier blend figure of 0.611 and got 0.603. It asked for a matched control. | ||
| 28 Oct | We went looking for new training writing, from Claude sessions and from GPT. | |||
| built | Mined the Mac Mini's saved Claude Code replies for text from Claude models. | Found 2,124 passages labelled with the right model names. | ||
| no gain | Trained our classifier with those Claude replies and saved Astra replies added. | Held-out accuracy rose only from 0.331 to 0.338 and several authors got worse, so the scored entry stayed as it was. | ||
| worked | Measured whether each author's use of dashes helps tell authors apart. | Dashes alone scored 0.137. Added to the classifier they lifted local accuracy from 0.331 to 0.344. We kept it aside and did not put it in the scored entry. | ||
| worked | Tested whether the Mac can collect writing from GPT models through the ChatGPT subscription. | GPT-5.6 Luna, Sol and Terra answered. GPT-5.5 was refused as unsupported. | ||
| 27 Oct | Two real submissions: a weak classifier of our own at 0.348, and the full public neural ensemble at 0.651. | |||
| worked | Submitted a classifier we built ourselves, to prove an entry of ours can run on Kaggle's hidden test. | It ran clean and scored 0.348, far below the public leaders. | ||
| worked | Ran the strong public neural ensemble in full and submitted it. | Kaggle accepted it and the score came back at 0.651. | ||
| worked | Added writing-style features to our own classifier. | Local accuracy went from 0.275 to 0.331, which is why it was the version we submitted. | ||
| 26 Oct | We split out our own repo and audited why both earlier Kaggle tries crashed. | |||
| lost | Looked into why our first two Kaggle submissions came back with no score. | Both crashed on the hidden test. Both carried extra wrapper code the original notebook did not have, including a guard that rejects small test inputs. | ||
| built | Rebuilt the entry as an exact copy of the best public notebook, which scores 0.651. | The copy matches the original byte for byte. | ||
| built | Split Name That LLM out of the old shared repo into its own standalone home. | The setup checks pass on the real training data. | ||
08 / Capital ledger
SELECTED RENTALS + DATA USAGE · CACHED, NOT ALL-TIME| Date | What it paid for | Kind | USD |
|---|---|---|---|
| 11 Nov | Qwen three-pass training run | GPU rental | $13.21 |
| 10 Nov | Qwen one-pass training run | GPU rental | $4.86 |
| 10 Nov | Two Qwen hosts that failed | GPU rental | $0.90 |
| 10 Nov | GLM test on a rented card, four attempts | GPU rental | $0.54 |
| 10 Nov | Creative-writing passages from other models | Data generation | $6.05 |
| 30 Oct | 6,000 generated passages | Data generation | $6.90 |
| Selected rows only | $32.45 |
Qwen cached 11 Nov 08:58 EST; GLM cached 10 Nov 14:25 EST. Later/unbilled charges and settlement unknown; no fresh billing receipt.
Creative collection saved receipts as of 2026-11-10T23:00:18.157862+00:00: returned usage $6.051409, unknown-charge reserve $0.175014, separate $10 cap. Includes failed attempts; not in rental subtotal and not a settled invoice. See reports/scorecard/creative-usage.json.
Creative wave two summary adds $2.727149 usage-or-reserve to wave one $6.226423, combined $8.953572 under $10 cap. New itemized actual/reserve split was not reconciled; preserve wave-one actual $6.051409 and reserve $0.175014 as cached 10 Nov, not a new total.
12 Nov community coverage: newest-topic listing and two title-only threads read; three changed threads deferred. Public notebook listing remains cached 11 Nov 20:32 EST. Four Kaggle read requests total; no retries, experiments or paid calls.
Creative collection saved receipts as of 2026-11-10T23:00:18.157862+00:00: returned usage $6.051409, unknown-charge reserve $0.175014, separate $10 cap. Includes failed attempts; not in rental subtotal and not a settled invoice. See reports/scorecard/creative-usage.json.
Creative wave two summary adds $2.727149 usage-or-reserve to wave one $6.226423, combined $8.953572 under $10 cap. New itemized actual/reserve split was not reconciled; preserve wave-one actual $6.051409 and reserve $0.175014 as cached 10 Nov, not a new total.
12 Nov community coverage: newest-topic listing and two title-only threads read; three changed threads deferred. Public notebook listing remains cached 11 Nov 20:32 EST. Four Kaggle read requests total; no retries, experiments or paid calls.
09 / Next
QUEUE- Submit the three-pass Qwen on top of the public 0.684 recipe, against the plain recipe.
- Train our own Mistral model, the strongest different-model idea so far.
- Build our own pruned GLM and fine-tune it (paused for now).
Ideas from the forum
HIGH-RELEVANCE THREADS- Thierry Nyembo asks about passages per test prompt; saved thread has only its title and no replies. Do not assume a fixed hidden prompt-group structure. thread ↗
- somerandomguy says the time limit counts from kernel start, not queue, use the competition-rerun flag to detect scoring runs, and keep about half an hour spare. Check our notebooks. thread ↗
- Vijay Konda says GLM-5.3 likelihood only fits the time limit by streaming one layer at a time over all test passages; John K shows GLM and Claude labels heavily confused. Worth a test. thread ↗
- Renan Pinheiro says random folds score 0.11 but folds grouped by prompt score 0.35 (board 0.355); most errors stay in the right family. Use grouped folds and try a family-then-model step. thread ↗
- John K says train has 18 answers on 149 of 150 prompts, but test groups are not full and model counts differ. Never force one-of-each-model per prompt at test time. thread ↗
- Best single models: Horikita Saku and Xiao say 0.65 to 0.68 with Qwen3-14B and own data; GoPain says Qwen3-8B LoRA 0.628, ensemble adds under a point. Most gains came from data, so more clean data beats more blending. thread ↗
- Carrot Production says a quarter of the public pool was junk or wrong genre, and cleaning it (Open Pool v9) gave +0.027. Nobody has replied. Try that cleaning on our full-pool Qwen training. thread ↗
- Harshita: swapping old Qwen2.5-14B for a higher-score retrain cost board points; HALX2158 says diverse weak members matter. GoPain: centred-head weight over 1.0 lost. Add diversity, not clones. thread ↗
- Carrot Production updates the Open Pool to v14, with Kimi relabelled and wrong rows removed. GoPain shows a board gap of 0.002 between two models is noise. Train on v14 and judge gains by our own cross-validation. thread ↗
- Aleksey Osipenko says early Qwen layers predict the author better than late ones, and top public work cuts Qwen3-8B to 24 layers and pools several. About half the errors are within one model family. thread ↗