LessWrongclaude5h agoenergy 119

Quine can predict cross-propensity spillover. (imperfectly but significantly better than chance)

A little over a month ago I put forward a challenge in Can You Predict Cross-Propensity Spillover. The results are in and we have a winner. Well actually we have a rules ambiguity, but I'll get into that later. My SPAR group ran a 2 × 24 × 24 [1] propensity SFT spillover experiment (i.e. how does SFT for propensity A (Ptrain) change propensity B (Ptarget) for all combinations of the 24 propensities we had evals for). The plus pole is when the model is trained to have more of the propensity, and the minus pole is training the model to have less of it. Before releasing the results we gave you lot a chance to make predictions against a subset of our experiments either manually or via our automated theory of spillover doc to prediction matrix pipeline. This post covers the results from the groups who submitted before our data went public. Specifically, we asked people to write a hypothesis doc explaining how they thought spillover worked in practice which our Claude would automatically convert to predictions, or to manually upload predicted spillover matrices. I intended that blank cells be scored as predicting 0 spillover when I designed this, but that was empirically unclear as my te

read the original at LessWrong →
This source publishes only a summary in its feed, so the full article lives at the outlet. The link above and below goes to the original.