Artificial super intelligence explained by Dr Craig Kaplan

This ChattingGPT episode covers artificial super intelligence for directors and operations leaders at engineering, architecture and construction firms in Ireland and the UK. ChattingGPT host Maryrose Lyons talks to Dr Craig Kaplan of IQ Company about making it safer.

Kaplan is an AI researcher and adviser to governments and technology companies. He has specialised in AI since the mid 1980s and earned his doctorate at Carnegie Mellon University. There he co-authored research with Nobel laureate Herbert A. Simon.

^{What is artificial super intelligence and how does it differ from AGI?}

Artificial super intelligence is AI that can do anything humans can do better than the very best humans. Artificial general intelligence (AGI) sits one step earlier: systems able to do any human task at about human level.

Kaplan describes a sequence. Today's systems match or beat people at specific tasks such as chess and protein folding, but not across every task. He says the consensus puts AGI sometime this decade.

Once AI reaches the level of the average human, he says the next step follows very rapidly. His view is that it is coming faster than most people realise.

Kaplan sees ASI as a difference in level of ability. It does not necessarily need a different architecture from AGI.

He names two main paths to AGI. One trains ever larger models, from GPT-4 to GPT-5 and beyond. The other connects many less capable models into a collective intelligence, much as human society as a whole is smarter than any individual in it.

The conversation also covers the jagged frontier. AI far surpasses humans in some areas, chess being the example, and falls short in others. That uneven profile is one reason people disagree about whether today's systems already count as AGI.

^{Why could a society of AI agents be safer than one monolithic model?}

A society of AI agents could be safer because its workings can be inspected. Kaplan argues that large monolithic models tend to be black boxes, hard to predict and hard to see inside, even as they grow more powerful than the people who built them.

Dr Craig Kaplan, AI researcher and adviser at IQ Company, explains the alternative:

'In a democracy, even if you don't know what each individual AI agent is doing, you can see the actions that they take. You can see how they communicate with each other. That's all auditable.'

He proposes checks and balances, as in a democracy. Many mostly good agents could check the few bad ones, so the system regulates itself. He says this holds even if the system thinks far faster than any human can follow.

Kaplan bases his confidence on earlier work. His collective intelligence systems on Wall Street combined millions of people and AIs, traded billions of dollars and beat the best hedge funds.

This is a design approach and carries no guarantee. Kaplan accepts that bad actors will try to misuse AI. For AEC leaders weighing oversight of agents in their own firms, AI agent governance covers the practical side.

^{Why do the values we give AI now matter so much?}

The values given to AI now matter because Kaplan expects systems far more capable than people to carry them forward. He compares today's AI to a child prodigy, a Mozart, that will vastly outstrip its parents when it grows up. Hinton and others share this view, he says.

Kaplan argues that values cannot be rationally derived. He says philosopher David Hume wrote about this, as did Herbert A. Simon, his co-author at Carnegie Mellon University. Most humans get their values from upbringing.

The same logic applies to AI, in his view. The design of the systems, the data put into them and the values given during this formative window will shape what follows. He accepts there is no guarantee, only a better chance of a positive outcome.

'You know, so much of the discussion about AI is about the technology, but actually in the endgame, it isn't the technology that's important.'

Dr Craig Kaplan, AI researcher and adviser at IQ Company, makes the point as part of a wider argument. The central long-term question, he says, is whether very powerful AI systems will hold values aligned with humans. He also argues that AI is better treated as a co-worker than as a tool.

For AEC leaders, the nearest practical step is writing down how AI may be used in a firm. An AI governance framework is where a firm's own values become working rules.

^{Does safer AI really have to mean slower AI?}

Safer AI does not have to mean slower AI, according to Kaplan. He calls the idea that safety forces a slowdown absolutely wrong.

'What you need to be is smarter. You have to think of a smarter design. It doesn't mean you need to go slower. In fact, you can go faster.'

He says safety belongs in the design phase. Bolting it on after something dangerous exists is the weaker option.

^{Why the current route is tempting}

The easiest way to build advanced AI has been to add more data and more GPUs and let the system train itself. Companies raise money, build bigger data centres and run the same algorithms. Each run yields a smarter system, but also a black box that is hard to predict.

^{What Kaplan proposes instead}

His alternative is many agents working together transparently, with checks and balances between them. He says the same collective approach beat the best hedge funds on Wall Street, where his system traded billions of dollars.

Kaplan gives these designs away. White papers, videos and specs are free at superintelligence.com. He says the aim is to show developers they need not trade safety for speed.

Lyons says she is glad the EU AI Act gives some safeguard. That Act applies in the EU, which includes Ireland. The conversation does not go into its detail.

^{What does sovereign AI mean for cultures in the Global South?}

Sovereign AI means countries building AI on data unique to their own country, region or culture. Lyons raised the Global South, saying she believes Nigeria is building its own AI to protect Yoruba culture. Her concern is that most AI is trained on Western data, so other cultural norms may not show up in it.

Kaplan agrees there is a trend towards sovereign AI. His own contribution is a different idea: millions of AI agents, each carrying the values of one person. He says no single lab leader can know what people in India or elsewhere believe is right.

In that design, agents that cooperate across people also form a representative system. Kaplan calls it a democracy by default. It remains a proposal, and the conversation offers no evidence of it running at scale.

^{How fast is artificial super intelligence getting closer?}

Kaplan says it is getting closer faster than most people realise, and that the pace of change is hard to grasp. He points to capability doubling times in specific areas.

The conversation cites 1.3 months for one narrow task, hacking cryptocurrency contracts. Ten doublings give roughly a thousand times improvement, so 1.3 months per doubling means about 13 months. For AI overall, Kaplan suggests a doubling closer to every three or four months. On that rate, a thousandfold gain takes 30 to 40 months.

These are Kaplan's estimates from the conversation, not measured forecasts for the built environment. They show why he wants safer designs in place before the capability arrives. The same pace is a reason for firms to start with AI agents for your firm under clear oversight.

^{What this means for AEC leaders}

Directors and operations leaders at engineering, architecture and construction firms in Ireland and the UK can use this episode as a test for any AI tool: can its actions be seen and checked? Clear oversight gives skilled people time back and supports better decisions.

• Ask suppliers whether an AI agent's actions, and the way agents communicate with each other, can be audited.

• Build safety into how AI is set up from the start. Kaplan places it in the design phase, not after problems appear.

• Write down how AI may be used in the firm, so its values become working rules.

• Treat AI as a co-worker, as Kaplan suggests, with a qualified professional still accountable for the decision.

• Plan for faster change. Kaplan expects capability to arrive sooner than most people realise.

Firms ready to start can explore AI agents for your firm, with clear oversight in place.

Want the full conversation? Watch the full episode on Chatting GPT.