Conference
COLM 2026
Small Foundation Models of Human Cognition and Behaviour
Centaur showed that a 70-billion-parameter language model fine-tuned on Psych-101 — 10.7 million choices from 60,092 participants — predicts human behaviour better than the domain-specific cognitive models psychologists build by hand. We asked what that result actually rests on — and answered with fourteen open models, from 135M to 14B parameters, fine-tuned to predict what real people do in psychological experiments, trial by trial.
Seventy billion parameters are not what cognitive foundation models rest on.
Across four base-model families, whether a model has been fine-tuned on behaviour matters far more than how large it is. Our 0.6B Qwentaur already edges the original Centaur-70B (0.69 vs 0.71 mean negative log-likelihood over the 46 held-out-participant tasks) and matches the cognitive-model baseline (0.69).
Accuracy on familiar paradigms is cheap. What remains expensive — and unsolved — is breadth: the models saturate within the paradigms they have seen and remain far from saturated outside them. Full results in the paper; every adapter and training script is public.
Adapter rank against model size on Psych-101. Mean negative log-likelihood over the 38 of 46 held-out-participant tasks with a published domain-specific cognitive model (dotted line), one side panel per adapter rank (r = 4–64). Lower is better.
Which model should you use?
Every model is released as a LoRA adapter on a public base. These six cover the sensible operating points; the collection holds the full rank and data-fraction grid behind the paper’s ablations.
| Model | Base | Params | Mean NLL ↓ | Pick it when |
|---|---|---|---|---|
| Qwentaur-14B | Qwen3 | 14B | 0.64 | Accuracy is all that matters. Best in-distribution, and the strongest transfer to unseen experiments. |
| Llama-Centaur-8B | Llama-3.1 | 8B | 0.64 | Ties the flagship at half the size — and shares Centaur’s own base family for direct comparisons. |
| Olmotaur-7B | OLMo-3 | 7B | 0.68 | Provenance matters as much as prediction: the base is open end to end — weights, data, and training recipe. |
| Qwentaur-4B | Qwen3 | 4B | 0.66 | A single consumer GPU is the budget. Near-flagship accuracy in a laptop-class footprint. |
| Qwentaur-0.6B | Qwen3 | 0.6B | 0.69 | Matches the cognitive-model baseline and edges reproduced Centaur-70B at about 1% of the parameters. |
| Smoltaur-0.1B | SmolLM2 | 135M | 0.89 | The floor of the sweep — yet still ahead of the untuned, 60×-larger Llama-3.1-8B (0.92). |
| Centaur-70B (reproduced) | Llama-3.1 | 70B | 0.71 | Reference: the original model, re-evaluated under our library and CUDA versions. |
| Cognitive models | — | — | 0.69 | Reference: the best published domain-specific cognitive model per task. Chance sits at 0.98. |
Mean negative log-likelihood across all 46 held-out-participant Psych-101 tasks; bf16 inference, LoRA rank 16, full training data. Lower is better. Each model link opens that size’s full adapter collection. Per-task tables, the out-of-distribution Psych-201 results, and 4-bit numbers are in the paper appendix and the results directory.
Run one
Every adapter loads with stock transformers + peft — or with unsloth, the loader our training and evaluations use. The base checkpoint resolves automatically from the adapter config. These are completion models over experiment transcripts, not chat assistants: give them a session in the Psych-101 format and read off the distribution over what this participant does next.
# pip install torch transformers peft import torch from peft import AutoPeftModelForCausalLM from transformers import AutoTokenizer model_id = "socius/Qwentaur-8B-LoRA-r16" # any adapter from the table model = AutoPeftModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto") tokenizer = AutoTokenizer.from_pretrained(model_id) # Psych-101 transcripts: task instructions, then one line per trial with # the participant's choice between << >> markers. Stop the prompt before a # choice and the model predicts what this participant presses next. prompt = ( "In this task, you have to repeatedly choose between two slot machines " "labeled O and G. You can choose a slot machine by pressing its " "corresponding key. When you select one of the machines, you will win " "or lose points. Machine O will not always give you the same points " "when you select it again, but machine G will always give 0 points " "when you select it. Your goal is to choose the slot machines that " "will give you the most points. You will receive feedback about the " "outcome after making a choice. You will play 20 games in total, each " "with a different pair of slot machines. Each game will consist of 10 " "trials.\n\n" "Game 1:\n" "You press <<O>> and get -1 points.\n" "You press <<G>> and get 0 points.\n" "You press <<" ) inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.no_grad(): probs = model(**inputs).logits[0, -1].softmax(-1) for key in "OG": token = tokenizer.encode(key, add_special_tokens=False)[0] print(f"P({key}) = {probs[token]:.2f}")
Right for the right reasons?
Accuracy alone would settle nothing; an instrument must also respond to the right things. Two stress tests probe what the models actually read from an experiment’s transcript — and whether the original 70B behaves any differently.
Masking content collapses prediction. Share of learned information retained as prompt components are removed, over the 27 ablation experiments with a defined chance floor (mean ± SEM). Dropping instructions costs little; masking the content of stimuli and feedback — or leaving only the bare choice history — erases essentially everything, for every model. Centaur-70B (diamonds) is indistinguishable from models a hundredth its size.
Order matters only where it should. Variance of each participant’s predictions across 50 shufflings of context-trial order. On THINGS odd-one-out (top), where trials are exchangeable, fine-tuned models are near-invariant — 8.4× lower variance than their bases. On intertemporal choice (bottom), an adaptive staircase whose ordering encodes the participant’s discount rate, the same models turn strongly order-sensitive — exactly where order genuinely carries information.
What these models are — and aren’t
Instruments, not theories
They estimate how predictable behaviour is within paradigms they have seen — a noise ceiling for experimental data. Whether fine-tuning moves these models any distance toward human mechanism is a question for representational analysis, and we treat it as open.
Bound by coverage, not capacity
Training spans 160 laboratory paradigms. Models saturate within the paradigms they have seen and remain far from saturated outside them: the binding constraint is breadth of paradigms, not parameters and not more participants per experiment.
Content is what they read
Prediction collapses when the content of stimuli and feedback is masked, and responds to trial order only where the experimental design makes order informative — a useful sanity check against shortcut explanations of what the models learned.
Low-rank, dense, supervised
Adaptation is LoRA throughout (r = 4–64), on dense decoder-only transformers, trained with supervised fine-tuning on one epoch. Full fine-tuning at small scales, other architectures, and RL-based post-training could all move the picture — none are tested here.
Artifacts
@inproceedings{oh2026smallcogfm,
title = {Small Foundation Models of Human Cognition and Behaviour},
author = {Oh, Nick and Gobet, Fernand},
booktitle = {Third Conference on Language Modeling (COLM)},
year = {2026},
note = {arXiv:2608.05224}
}
