Sam Altman, the chief of OpenAI, has a different perspective on the current state of artificial intelligence (AI). He believes that there is no limit to scaling AI and that it is more important than ever to focus on scaling the right things. However, former OpenAI co-founder and Safe Superintelligence (SSI) founder Ilya Sutskever disagrees. In an interview with Reuters, he stated that the 2010s were the age of scaling, but now we are back in the age of wonder and discovery. Sutskever is reportedly working on an alternative approach to scale large language models (LLMs) and eventually build safe superintelligence.
Based on his academic and research interests, it is likely that Sutskever is advancing artificial general intelligence (AGI) by scaling transformer architectures with a focus on reinforcement learning and self-supervised methods. These methods allow models to learn from vast amounts of data with minimal human guidance and increase their adaptability to complex tasks. OpenAI is also following a similar path, with plans to scale test-time compute and utilize high-quality synthetic data generated by previous models.
Another former OpenAI co-founder, Andrej Karpathy, believes that LLMs lack thought process data and that enough high-quality thought process data can help achieve AGI. He coined the term “jagged intelligence” to describe this lack of cognitive self-knowledge. These developments come at a time when reports suggest that traditional scaling may be reaching its limits, with some models underperforming despite scaling efforts. The focus is now shifting towards quality synthetic data and scaling test-time compute.
Yann LeCun, the chief AI scientist at Meta, couldn’t resist criticizing OpenAI’s new approach. He stated that he had been working on “the next thing” for a while now at FAIR and added, “I don’t want to say ‘I told you so’, but I told you so!” Earlier this year, Meta merged two major AI research efforts, FAIR and the GenAI team, in pursuit of AGI. Under the guidance of LeCun, the company is developing a “world model” with reasoning capabilities.
