Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior
Paper • 2609.39827 • Published • 12
This model is a pre-trained checkpoint from the paper Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior. It is a causal language model based on the SmolLM3 architecture and is part of the verify-ppt project.
Paper: Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior
Project page: https://huggingface.co/verify-ppt
Code: https://github.com/gucci-j/verify-ppt-at-scale
For additional training, evaluation, and artifact details, see the GitHub repository.