By Simon Cleobury, Head of Arms Control and Disarmament, Geneva Centre for Security Policy
Governing how states use their militaries, whether that means rules around conduct during conflict, treaties prohibiting certain categories of weapon, or agreements controlling how many arms of a certain capability they can deploy or possess, has focused diplomats’ minds for over a century. Their work has produced international humanitarian law (IHL or the laws of war), disarmament treaties, and arms control agreements. This governance architecture is not perfect. It was reactive, slow to negotiate, and failed to get every state on board. All three areas have experienced challenges: alleged violations of IHL a regular occurrence, the Nuclear Nonproliferation Treaty (NPT) is under huge pressure from a range of disarmament and non-proliferation issues, the Chemical Weapons Convention has been violated, attempts to strengthen the Biological Weapons Convention (BWC) have failed, states have withdrawn from the conventions on land mines and cluster munitions, and the final arms control agreement between the two countries with largest nuclear arsenals has expired.
Despite all this, the governance architecture still stands. However, with the introduction of autonomous weapons and the growing use of AI in weapon systems, diplomats face an even bigger challenge: agreeing a new architecture to govern these machines of war. Here are ten reasons why building a global governance architecture for military AI is so hard and why getting there will take time.
Streamlining the different processes
Everyone is talking about AI, and the disarmament community is no different. There are several different multilateral processes aimed at governing military AI. One, the group of governmental experts on lethal autonomous weapon systems (LAWS GGE), has been running for over ten years in the framework of a treaty. Others, like the Dutch initiative on the responsible use of AI in the military domain (REAIM), are more recent, sit outside any formal structure and aren’t open to everyone. Neither process is universal, so we are now seeing new processes created in the UN General Assembly (UNGA) too. All of these try to bring together governments with industry, academia, and civil society.
What is missing is a permanent, universal home for these discussions, somewhere that brings all the actors together on a regular basis. Some have suggested that a truly global governance architecture needs a dedicated multilateral AI institution, modelled on the International Atomic Energy Agency (IAEA), to provide coordination and legitimacy. Bringing all the discussions under one roof is a nice idea, but it would need to sit behind a treaty or be established through the UNGA. Both routes are highly ambitious and would require the major states to agree to pay for it. More realistic would be a permanent inter-governmental body, with civil society and industry as observers, something like the Committee on the Peaceful Uses of Outer Space. Call it the Committee on Military AI, based in Geneva.
Solving the definitions problem
Agreeing a definition for “autonomous weapon system” has proven so difficult that it’s one of the reasons why the LAWS GGE has never progressed to negotiations on a legally binding instrument. Part of the problem is technical. Even before politics enters the picture, these definitions are hard to draft, especially when the technology keeps developing. Then there’s the fact that definitions shape the scope of any instrument. If you want to prohibit use, then your definition may be narrower than if you just want to regulate. If you want to do both, it gets even more complicated. And if you don’t really want a legally binding instrument at all, because you don’t want your military constrained, it suits you to drag out the definitions debate to prove agreement is impossible.
This begs the question of whether you need a definition of what you are trying to govern. It sounds absurd to ask, but there is no definition of a nuclear weapon in the NPT and no definition of a biological weapon in the BWC. Despite that, no-one has questioned the fundamental objective of these treaties. Could the same be said for an instrument on military AI? Probably not. It is not as simple as banning a category of weapon that is universally understood. AI is not a weapon; it is an enabler. Try to govern “AI-enabled weapons’’ and you won’t find a shared understanding of what that even covers. From a negotiation standpoint, that’s a non-starter. From a compliance and verification standpoint, it would likely be unworkable too.
Rules without referees
Even if states can get past the definition challenge, a legally binding instrument still has to deal with compliance and verification. Verification has always been a headache for disarmament diplomats, and different domains have handled it differently. In the nuclear and chemical weapon domains, specific bodies were given the mandate to inspect and investigate, namely the IAEA and the Organisation for the Prohibition of Chemical Weapons (OPCW). In the biological weapon domain, negotiators simply didn’t address verification. Later attempts to negotiate a verification protocol failed.
Verifying compliance with the weapons of mass destruction treaties, where alleged violations are rare and what you are verifying is tangible, is a different job than verifying compliance with a treaty covering technology that states will be using and developing constantly. If you think agreeing on definitions will be difficult, wait until you try to agree on rules around inspecting targeting systems. These are highly sensitive and classified, meaning major military states would be highly unlikely to agree to any kind of inspections. There are ways of verifying compliance remotely and through open-source research, but these also would be heavily negotiated given concerns about reliability and legitimacy. Negotiators would probably have to satisfy themselves with a provision obliging states parties to consult with one another about any potential violation of the instrument.
The battlefield does not wait for Geneva
Given how AI has already been used on the battlefield in Gaza, Iran, and Ukraine, some would argue the window for regulation has closed. Developments in these live conflicts have simply outpaced the slow moving inter-governmental processes, so any legally binding instrument would arrive too late. States using AI on the battlefield have seen the advantages it brings, and they’re unlikely to want limits, either on what they are already using or on what they’re developing next.
In the short term, it is hard to see the gap between the battlefield and the Geneva conference room closing. States want to see where the technology takes them and want to keep a strategic advantage over potential adversaries. This is classic security dilemma territory: states fear others are getting ahead and feel they have to catch up. That said, there could come a point where states feel they have taken the technology as far as it can go, and where some kind of control over how these systems are used starts to look like it’s in their own interest.
Right now it looks like the battlefield is way ahead of Geneva, but that could change. Targeting errors, such as the US strike on an Iranian school, have brought the military AI into sharp focus and have left a lot of people seriously worried. So far these concerns have not been enough to shift the dynamics in negotiations. But if something goes catastrophically wrong, the calls for governance could generate the political will needed to agree on a global governance architecture.
The consensus rule
Processes like the LAWS GGE run on the consensus rule. It is a longstanding working method for the disarmament community, designed so no state has to agree to anything that damages its national security. It is also a protection that keeps major military states engaged with the process in the first place. Drop the consensus rule, and move to simple majority voting like the UNGA does, and you are much more likely to adopt an agreement. But if the major militaries never engaged with the process, how effective is the agreement really?
The argument for going this route is that you have to start somewhere. Getting some states to reach agreement encourages others, starts building norms, and eventually brings more states on board. The real test is how many states sign up at the outset. If it is a group of non-aligned countries that are not big users of military AI, it is hard to see the agreement having much effect.
The case for consensus is that it keeps everyone at the table. What you gain in universality, though, you can lose in substance, because reaching consensus often means watering down the instrument. Reach agreement that way and you are left with something that traded away substance for adherents.
Getting enough states to sign up
Given the frustrations of the consensus rule, it is tempting to head for the UNGA route instead. A group of countries is already on record wanting to negotiate a legally binding protocol on LAWS. If they don’t get what they want, they will likely push for the UNGA route. We could also see a new UNGA process launched on the use of AI in the military domain more broadly, beyond LAWS.
But going to the UNGA to negotiate a new legally binding instrument won’t necessarily be quick or easy. Military AI is not a simple yes-or-no choice. Most countries will want to explore the potential of AI-enabled military systems. Many will want such systems themselves one day, or hope to benefit economically from producing them, or both. So even these countries might hesitate to adopt an instrument that’s too constraining.
Any coalition pursuing this route would also need to decide what kind of instrument it wants: a high-level prohibition, or a tiered agreement combining prohibition and regulation. They would encounter all the same challenges that have dogged the LAWS GGE, particularly around definitions and verification. And any agreement reached would face the same legitimacy questions raised above, if none of the major militaries signed up.
Engaging industry
Much of the rapid development in military AI is coming from industry, not governments. Industry is designing the autonomy that the inter-governmental processes are trying to govern. Yet, it is governments that discuss global governance, with no formal role for industry in those processes. Yes, industry is invited to speak, and some engage with processes like REAIM, however, their input into the discussions is not that substantive. Without rules at the global level, this effectively means the industry is writing its own rules.
Good governance in any area begins at home. Good global governance of military AI will need to be built on good domestic governance. States need to work with their industries to ensure there are rules requiring them to act responsibly. Exchanges of best practice can feed into global rules. This should be in the industry’s interests too. Global rules level the playing field and bring certainty, and it was exactly these considerations that led industry to welcome the idea of an Arms Trade Treaty. If industry itself called for global governance on military AI, that would bring it into the discussions, giving it a chance to shape the outcome. If we don’t make them feel part of the process, we will have a much harder time agreeing on anything.
Capacity building
Responsible use of military AI matters for all nations, but many of the countries of the Global South lack the technical capacity to properly evaluate the proposals on the table at negotiations. That’s why it is significant to see a country from the Global South, Kenya, stepping up to host the Fourth REAIM Summit in Nairobi in April 2027. Kenya wants the Summit to focus on strengthening institutional capacity, developing human capital, and fostering collaboration between governments.
These are worthwhile aims and REAIM can certainly be a forum for helping the Global South states to build their capacity. There is plenty of room for states and industry to share best practices and deepen each other’s understanding of military AI. Still, this too will be a challenge. Major militaries will have proliferation concerns and will be cautious about sharing too much detail about what they are developing.
Another focus of the Summit will be strengthening regional capacities. States working together to understand military AI issues at the regional level could have a real positive impact on the global discussions. That will need leadership, which in itself could be a capacity issue. It will also take political will. Kenya has shown the way. The question is whether REAIM can become more than an annual roadshow for industry and think tanks.
Interoperability
Much of the current debate treats states as individual actors, but military AI deployment increasingly happens inside alliance structures with shared intelligence and interoperable systems. Whether different national systems can communicate, exchange data, and work together is a traditional operational issue. But there is also an important AI governance angle here: can different systems operate safely and responsibly together? This topic has not received much attention in the LAWS GGE, though has been raised in broader military AI discussions, where countries who are part of military alliances have recognized the challenges AI poses for interoperability.
How would systems operate together if they had different ways of selecting targets and different rules on human authorization? That question needs to be addressed, both operationally as well as for the sake of any future legal instrument working properly. Disarmament diplomats have dealt with interoperability before, notably in Article 21 of the Convention of Cluster Munitions (CCM). That provision let CCM states parties join military operations with non-state parties. It was a crucial provision in allowing many NATO states, who carried out joint operations with the US, to join the CCM. In an ideal world you would not need such a provision, but something similar may well be needed in any instrument governing military AI.
On the positive side, interoperability discussions could be an opportunity to promote good governance and push for uniform high standards, at least amongst states in the same military alliance. In the absence of any legally binding instruments, action by military alliances and regional groups to make sure their systems can operate together safely and responsibly would be an important way to raise governance standards.
Playing the long game
Prospects for agreeing a legally binding instrument on LAWS are not good, and setting up a universal process through the UNGA could take years. But there are other tools in the disarmament toolkit that can be and already are being used.
The fact that states are meeting to discuss these issues is itself an important trust and confidence building measure, so we need to keep that dialogue continuing. Transparency is also vital. Unilateral statements from states and military alliances help others understand their policies, which in turn should lead to more exchanges of best practice and military-to-military dialogue. Joint declarations, like the REAIM Summit outcomes, help establish norms of behavior.
There is clearly a key role for the major militaries. Even if none of them want a new legally binding instrument, they can still talk to each other about the security threats posed by irresponsible use of military AI. If they can agree on some guardrails, that would be hugely reassuring. The statement by the US and China on never allowing a machine to decide on the use of nuclear weapons is a good example.
Regional groups can also play their part. Agreements on guidelines and codes of conduct can provide good governance standards for others to follow. Processes such as REAIM could hold regional meetings between Summits to help with capacity building and build common regional understandings of the challenges.
The long game will pay off
The history of arms control and disarmament tells us that we need to play a long game here. The absence of legally binding instruments in this area does not mean there is no governance. Substantive dialogue is happening regularly on military AI and the knowledge being built up helps ensure these systems are developed responsibly. The time spent on these discussions, slow moving and frustrating as it is, won’t be wasted. The acquis that disarmament diplomats talk about, could one day translate into an effective, legally binding global governance architecture.
Simon Cleobury is Head of Arms Control and Disarmament at the Geneva Centre for Security Policy, a position he has held since July 2023. He served as the United Kingdom’s Deputy Permanent Representative to the Conference on Disarmament in Geneva from 2017-2023. Prior to that he was Senior Policy Advisor in the Security Council team at the United Kingdom Mission to the United Nations in New York from 2012-2016.
The views expressed in this article are those of the author and do not necessarily reflect the views of Techplomacy Magazine or the Techplomacy Foundation. Articles may be republished in full, without alteration, with credit to Techplomacy Magazine (magazine.techplomacyfoundation.org).
Series: Techplomacy Conversations™