GEMMA 4 DEVELOPER AGENT / RESEARCH LOG
Keith’s weight-search loop, iteration by iteration. Follow the ideas that helped, the ones that hurt, and the evidence still to come.
01 / MEASURED SCREENING RESULTS
Each point is one tested edit. Gain is extra tasks solved against the mean of two unedited reference arms in the same batch. These are separate trials, not a cumulative improvement curve.
Different batches can use different task lists and environments. Compare each edit with its own reference—not raw solve counts across batches. Corrected counts replace the original; they are not another iteration. Later batches with an unclassified protocol remain in the ledger but are not plotted here.
02 / THE EXPERIMENT LEDGER
| Entry / idea | Author | Recorded state | Screening result | Evidence |
|---|
No entries match these filters.
03 / FROM SCREEN TO CONFIRMATION
Gain needed to reach stage 2.
Compare with the reference and matched control.
Passing screening only nominates a candidate. An untouched confirmation and the golden evaluation must establish a real improvement.
The second-stage matched edit control is separate from the two unedited reference arms plotted above.
04 / MEASUREMENT CHANGES
The displayed scored batches used a 450-second session limit. Some environments required corrected reruns. Batch b003 uses its corrected counts.
The recorded protocol switched to 10,000 generated tokens per episode. Batch b005 was stopped and its candidate queued for another second-stage run. Token-clock results must be classified separately from the earlier time-limited screens.
This is a repository snapshot, not a connection to the training machine. New protocols must be assessed with matched reference runs.