cream cat

With the explosion of AI technology, especially the rise of GPT, this term has been flashing in my mind repeatedly. According to OpenAI's explanation of their model's capabilities, they do not know why GPT has shown extraordinary performance since version 3.5, or they cannot explain this performance based on principles. In fact, this term has already existed.

What is the phenomenon of emergence? According to the professional definition, the phenomenon of emergence refers to the unexpected overall properties or behaviors that arise in a system due to numerous small interactions and interconnections. These overall properties or behaviors cannot be simply understood or predicted by analyzing the behaviors of individual components of the system, but require consideration of the system's overall interactions. A famous example is the behavior of ants. The behavior of an individual ant is simple, but when they interact in a certain way, the entire ant colony can exhibit very complex behaviors, such as building ant nests, searching for food, and protecting the ant colony.

The same principle applies to large AI models. With the increase in training data, a simple prediction model suddenly acquires the ability to understand logic. Because this ability can lead to better predictions, better predictions further enhance logical abilities, resulting in an emergent effect. Another example is social networks. When people constantly connect and influence each other in social interactions, in turn, the social network itself can also influence individuals. The mutual interactions lead to unexpected states or scenarios in the entire network.

Many domestic experts have always been skeptical about whether AI has logical abilities. They believe that language models cannot generate logic. The argument is also simple: logical abilities are only possessed by higher intelligent organisms, such as humans. However, humans do not have this ability from birth. When humans are born, they only have instinctive reactions, whether it is the ability to breathe or to eat. The development of brain neurons matures gradually as humans grow, and it is not considered complete until after the age of 20. So why is AI different?

In addition, AI has an advantage over humans in that both the number of cells and neurons in humans have theoretical limits and cannot grow and expand indefinitely. However, as electronic computers based on silicon, their computing chips can be infinitely expanded and upgraded. The latest A100 from NVIDIA uses 7nm technology, and TSMC will soon mass-produce technology below 5nm, which means that future high-performance GPUs will have even more powerful computing capabilities.

As for the future potential of GPT, there are different opinions in the industry, but based on the current iteration speed, I believe that within 2 years, generative models will be able to successfully complete video creation tasks. In addition, several companies in mainland China are also developing related products, such as Wenyan Yixin. However, considering the long-standing restrictions on the cultural industry in China, how can large language models be reviewed? After all, hot topics are dynamic, and censorship is also dynamic. If the model cannot perform self-censorship, it will not be promoted by the mainland government. Just look at Bitcoin and crypto.