Yann LeCun, Meta’s Chief Scientist for Artificial Intelligence, has been honored with the TIME100 Impact Award for his contributions to the field of artificial intelligence. Prior to receiving the award, LeCun sat down with TIME magazine to discuss topics related to achieving “artificial general intelligence” (AGI), the benefits of Meta’s open approach, and the controversial view of existential threats posed by artificial intelligence.
Many contemporary technology experts believe that training large language models (LLMs) using more computational power and larger datasets will lead to the attainment of artificial general intelligence. Do you agree with this?
We talk about how remarkably well LLM models perform when trained on large amounts of data, but they do have their limitations. Currently, we can see that these systems produce hallucinations and do not understand reality. They require massive amounts of data to achieve a level of intelligence that is ultimately not so great. And they are unable to think rationally. They cannot plan anything beyond what they have been trained on. So, they are not the path to what we call “AGI.” I don’t like that term. They have their applications, there’s no doubt. But they are not the way to achieve human-level intelligence.
You mentioned that you don’t like the abbreviation “AGI.” This is a term that Mark Zuckerberg used in January when announcing that Meta is focused on building artificial general intelligence as one of its main goals.
There is a lot of misunderstanding here. The mission of our Fundamental AI Research (FAIR) team at Meta is to achieve human-level intelligence. That ship has already sailed; it’s a battle I’ve lost, but I don’t like calling it AGI because human intelligence is not truly general. There are certain characteristics that intelligent beings possess, which AI systems do not have, such as understanding the physical world, planning sequences of actions to achieve a goal, and reasoning in a time-consuming manner. Humans and animals have a specialized part of our brain that we use as working memory. LLM models do not have that.
A child learns how the world works in the first few months of their life. We don’t know how to do that for artificial intelligence. When we have techniques for “world models” learning, just by observing the world, combined with planning techniques and perhaps short-term memory systems, then maybe we’ll have a chance, not at general intelligence, but let’s say intelligence at the level of a cat. Before we reach the human level, we will have to go through simpler forms of intelligence. And we are still very far from that.
In a sense, this metaphor makes sense because a cat can look at the world and learn things that even the most advanced LLM model simply cannot understand. However, the entire condensed history of human knowledge is not available to the cat. How limited is this metaphor?
Here’s a very simple calculation. A large language model is trained on all publicly available text on the Internet, roughly. That typically amounts to 10 trillion tokens. Each token is about two bytes. So that’s two times 10 to the power of 13 bytes of training data. And you say, “Oh my gosh, it would take a human 170,000 years to read that.” It’s a tremendous amount of data. But then you talk to developmental psychologists, and they tell you that a four-year-old child has been awake for 16,000 hours in their life. And then you try to estimate how much information has made it into their visual cortex in those four years. The optic nerve carries about 20 megabytes per second. So, 20 megabytes per second times 60,000 hours times 3,600 seconds per hour gives you 10 to the power of 15 bytes, which is 50 times more than 170,000 years of text.
Yes, but text encodes the entire history of human knowledge, whereas the visual information a four-year-old child receives contains basic 3D information about the world, basic language, and things like that.
But what you’re saying is not true. The vast majority of human knowledge is not expressed in text. It happens in the subconscious part of the mind that you learn in the first year of your life before you learn to speak. Most knowledge is truly related to our experience of the world and how it works. That’s what we call common sense. LLM models don’t have that because they don’t have access to it. And that’s why they can make really stupid mistakes. That’s where hallucinations come from. Things that we take for granted turn out to be incredibly difficult to replicate in computers. That’s why AGI, or human-level AI, is not just around the corner; it requires some very deep perceptual changes.
Let’s talk about open-source software. Throughout your career, you have been a strong advocate for open research, and Meta has adopted a policy of effectively sharing its most powerful language models, most recently Llama 2. This strategy sets Meta apart from Google and Microsoft, who do not share the “weights” of their most powerful systems. Do you think Meta’s approach will still be suitable as its AI becomes more advanced, even approaching the level of human intelligence?
First and foremost, the answer is yes. And the evidence for that is that in the future, all interaction with the digital world and the world of knowledge will be mediated by AI systems. They will serve as assistants that will be with us constantly. We won’t be using search engines anymore. We will simply be asking our assistants, and they will help us in our daily lives. Our entire knowledge will be based on these systems. They will become the repository of all human knowledge. In such a situation, we cannot rely on closed and proprietary systems. That is why an open approach is so important and will continue to be appropriate, even as our AI becomes more powerful, approaching human-level intelligence.
Questions and Answers about Artificial Intelligence and the TIME100 Award for Yann LeCun
1. What honor did Yann LeCun from Meta receive?
Yann LeCun, Meta’s Chief Scientist for Artificial Intelligence, was honored with the TIME100 Impact Award for his contributions to the field of artificial intelligence.
2. What did Yann LeCun say about achieving “artificial general intelligence” using large language models?
Yann LeCun noted that large language models have their limitations. Although they are incredibly effective when trained on large datasets, they cannot achieve true general intelligence. They do not understand reality, cannot think rationally, and are limited to what they have been trained on.
3. How does Yann LeCun feel about the term “AGI”?
Yann LeCun is not fond of the term “AGI” (artificial general intelligence) because he believes that human intelligence is not truly general. There are certain characteristics that intelligent beings possess, such as understanding the physical world and planning actions to achieve goals, that AI systems lack. Therefore, achieving human-level intelligence requires significant perceptual changes.
The source of the article is from the blog be3.sk