A new study by researchers from Tel Aviv University and Stanford University shows that AI can develop general predictive skills using only self-generated training data, challenging a fundamental assumption about how artificial intelligence learns.

Almost every artificial intelligence system we know begins in the same way: it is exposed to vast amounts of information from the real world – books, websites, images, and recordings and trained to identify patterns within it. A new study involving researchers from Tel Aviv University and Stanford University raises another possibility: Can a system develop general learning capabilities even when all the data it trains on is generated entirely from scratch within the system itself? The researchers have shown that it can, marking a significant step toward a goal, long considered the holy grail of AI training.

The study was conducted under the supervision of Dr. Yoav Levine of the Blavatnik School of Computer Science at Tel Aviv University, as part of the postdoctoral research of Dr. Kfir Dolev, in collaboration with researchers from Stanford University. The study is available as a preprint on arXiv.

Two AI Models Learning Together

In the new study, the researchers presented a training method in which two models start from scratch and learn together. The first generates short computer programs that produce sequences of symbols. The second is trained to predict the next symbol in each sequence. The model generating the exercises receives feedback on how much they contribute to learning and gradually becomes better at selecting challenges suited to the learning model’s developing capabilities.

The decisive test came after training: Would what the model had learned from these artificial sequences also help it predict real-world data it had never encountered during training? The researchers evaluated its performance on text, images, sounds, music, code, and DNA sequences. They found that as more computational resources were devoted to the training process, the model’s predictive capability improved consistently across many domains. The model was also able to infer rules from examples presented to it during testing, without any additional training.

What Happens When AI Creates Its Own Lessons?

One of the most intriguing findings concerned the learning material the system generated for itself. During training, familiar mathematical patterns appeared in this material, including the Fibonacci sequence. In other words, the system learned to find ways to generate structured sequences from which the second model could benefit. In another experiment, the researchers found that pretraining with the new method helped the model learn faster when it was subsequently exposed to real-world data.

What Could This Mean for the Future of AI?

The study offers an initial answer to fundamental questions in artificial intelligence research: Does intelligence have a “universal” component that does not depend on any external data? Does part of a model’s learning capacity stem from general patterns that can be acquired without prior exposure to real-world information? The findings indicate that this is indeed possible. However, the researchers emphasize that the method cannot teach a system facts about reality without exposing it to that reality. Rather, it may eventually help AI systems learn from real-world data more efficiently.

Dr. Yoav Levine concludes: “For years, the prevailing assumption was that for an AI model to learn anything useful, it first had to be fed enormous amounts of real-world data. In this study, we disproved this fundamental assumption and showed that a model can acquire general predictive skills even when it is trained only on data generated within the learning process itself. This is still a proof of concept involving small models, but the findings raise fascinating questions for future research: What universal mathematical patterns will emerge as we strengthen AI using these methods? How far can we take AI without exposing it to information from the outside world? And can we prepare models to learn more effectively from reality through experiences they have generated for themselves?”