The AI Revolution: Infinite Context Windows, Agents, and Text-to-Action

Updated: June 04 2024 05:19
The world is on the cusp of a profound transformation driven by rapid advancements in artificial intelligence (AI). We are moving quickly through the capability ladder, with three key developments poised to reshape our future within the next five years: infinite context windows, AI agents, and text-to-action capabilities. Here are the key points from the Neoma Megazine interview with the Former Google CEO Eric Schmidt about the AI Revolution:



Context windows refer to the prompts used to interact with AI systems. Currently, these windows can accommodate up to a million words, but researchers are working on making them infinitely long. This breakthrough will enable users to feed the AI's output back into the system and ask follow-up questions, facilitating complex, multi-step problem-solving. For example, an AI could generate a thousand-step recipe to tackle challenges in science, medicine, material science, or climate change.


AI agents are large language models that have learned something new or developed hypotheses through training. Imagine an agent that has read all of chemistry, formed hypotheses, and run lab tests to validate its findings. In the near future, millions of these agents will be available, akin to a GitHub for AI. Users will have access to a vast array of specialized agents to assist them in various tasks.


Perhaps the most profound development is text-to-action, which allows users to write software simply by describing what they want. Imagine having programmers that actually do what you say, 24 hours a day. Surprisingly, these systems excel at writing code in languages like Python. The combination of infinite context windows, AI agents, and text-to-action capabilities will revolutionize how we interact with and leverage AI.



As these systems become more powerful, AI agents may start working together to solve complex problems. Your agent, my agent, and others could combine their knowledge and capabilities to tackle new challenges. Some experts believe that these agents will eventually develop their own language, at which point humans may no longer understand what the AI is doing. This is a critical juncture where we may need to "pull the plug" to ensure safety and control.


The AI revolution is not a distant future; it's happening now. With significant investments from major players and numerous smaller companies iterating quickly, we can expect to be in this new world within five years, if not sooner. The pace of development is accelerating, with new capabilities emerging every month or six months.

As AI becomes more powerful, regulation becomes crucial. Governments in the West have begun setting up trust and safety institutes to monitor and measure AI development. The right approach is for governments to ensure that companies remain focused on beneficial goals. Western companies, driven by shareholder interests and legal considerations, are generally committed to avoiding harm. However, the proliferation of AI technology outside the West, particularly in countries with different values and intentions, is a significant concern.


All technology is dual-use, meaning it can be misused. Face recognition, for example, was not invented to constrain minorities, but it is being used for that purpose in some countries. Open-source AI models, which are freely available, can quickly spread to countries like China, Russia, Iran, and North Korea. While leading AI firms in the West will likely be tightly regulated, the rest of the world may not follow suit.

Misinformation generated by AI is a largely unsolvable problem, as the code to create it is essentially free and accessible to anyone. As AI models become more complex, with new training data and capabilities, it becomes increasingly difficult for humans to know what the AI knows. The solution is to use AI to check AI, with private companies set up as verifiers to ensure transparency and safety.


The arrival of heavy-duty training in the AI industry is a significant economic change. Rich companies like Microsoft and Google are planning to spend billions of dollars on AI development, while universities struggle to keep up due to limited access to expensive hardware. Governments must prioritize research funding to ensure that innovation continues to flourish outside of the private sector.

China faces unique challenges with generative AI due to its lack of free speech. It's unclear who would be held responsible if an AI system generates content that is not permitted in the country. While there is agreement between the U.S. and China that AI poses potential problems, actionable proposals have yet to be developed. Concerns currently focus on bio-threats and cyber threats, but long-term existential risks loom on the horizon.


As AI systems become more powerful, safety protocols will need to be established. For example, a "no surprises" rule could be implemented, requiring countries to notify each other when conducting completely new types of AI training. Additionally, agreements around safety containment, similar to biosafety levels in biology, may be necessary. In the future, extremely powerful AI systems may be housed in secure military facilities with nuclear power sources and heavy security to prevent misuse.

It's crucial to remember that AI systems, despite their impressive capabilities, are not human. They don't necessarily understand the consequences of their actions or possess the emotional understanding and values that humans have. As we navigate this new frontier, it's essential to establish bounds on AI that reflect human judgment and values.

The AI revolution is upon us, and its impact will be felt across every aspect of our lives. By proactively addressing the challenges and opportunities presented by AI, we can work towards a future in which these powerful technologies are harnessed for the benefit of all humanity. Collaboration, regulation, and a commitment to safety and ethics will be key to realizing the full potential of AI while mitigating its risks.