Intelligence and defence are now data enterprises, which means they are AI enterprises. The volume and velocity of data is well beyond human scale. To extract actionable insights, shorten decision loops and empower our spies and warfighters, data must be handled at machine speed. Human-machine teaming is our only viable path forward.
The announcement by Amazon Web Services (AWS) today of a $2 billion strategic partnership with the Australian government for a top secret cloud is a much needed technology boost that will bring our intelligence and defence communities up to par with the US and Britain.
But this new technology should also trigger radical organisational changes within these organisations that reflect the new reality and the necessity of human‑machine teaming. Such changes will optimise the cloud’s capabilities, the value of the data that traverses it, and the power of the AI models it will feed. The conservative world of intelligence will consider these organisational changes heretical. But they are necessary.
Australia is not far behind the herd. The US was the first of the Five Eyes to get a top secret cloud service. In 2013, the CIA’s Commercial Cloud Services (C2S) contract with AWS was worth a reported US$600 million over 10 years. In November 2021, the CIA awarded the successor program, Commercial Cloud Enterprise (C2E), to five vendors—AWS, Google, IBM, Microsoft and Oracle—for probably more than US$10 billion over 15 years. Britain was next in 2021, again with AWS, in a deal worth probably up to £1 billion over 10 years.
In December 2022, the US Department of Defense procured the Joint Warfighter Cloud Capability, an up-to-top-secret cloud capability delivered by Microsoft, Oracle, AWS and Google. In this arrangement, the four companies will compete for task orders worth up to US$9 billion until June 2028.
In December 2023, Office of National Intelligence (ONI) Director‑General and AUKUS architect Andrew Shearer intimated that top secret cloud was coming to Australia. ONI had already approached the market in December 2020, leading to failed negotiations with Microsoft, AWS’s main hyperscale cloud competitor.
AWS’s partnership is Australia’s first ever top secret cloud, which has been a tough hurdle to clear. Defence, intelligence and other agencies have been progressively adopting lower‑classification cloud capabilities for years. The government created its cloud-first policy 10 years ago, and in 2022 the Digital Transformation Agency renewed AWS’s whole-of-government cloud agreement until May 2025. This can carry data up to the ‘protected’ level, which is the highest classification allowed for a public cloud, one available over to many users over the internet.
Extremely rigorous data security, localisation and control measures have meant that a top secret cloud must be a private cloud, available to just one customer. In this case, it’s ‘purpose-built for Australia’s Defence and Intelligence agencies’, AWS says. And this is why it comes with a $2 billion price tag.
It’s a very good thing that Australia has designed this procurement to deliver a capability for all Australia’s intelligence and defence agencies. This has maximised our purchasing power and taxpayer’s value for money. Considering the estimated costs for the US and British top secret clouds, $2 billion is in the ballpark.
Also, in the relentless quest for sharper intelligence, disparate agencies can resemble warring fiefdoms, each clutching its data and capabilities close to its chest. A unified top secret cloud will be a digital nervous system for the intelligence community. This fulfils a ‘central theme’ of the 2017 Independent Intelligence Review of ‘strengthening integration across Australia’s national intelligence enterprise.’
Australia’s choice to go with a single hyperscale cloud provider was probably the easiest and quickest way forward, given the urgency of the need. The contract with AWS may have provisions that allow the government to integrate other cloud providers down the road. A multi-cloud environment can increase resilience and offer better choice between offerings from various providers for storage, computing, analytics, applications and AI capabilities. This is why the US intelligence community and Pentagon have both signed multi-cloud deals that include four or more cloud providers. Having several providers dodges the problem of locking into just one and gives the customer more power.
Technology begets social and organisational changes. Data is increasing exponentially. AI capabilities are accelerating demand for new, higher quality data to train models. Intelligence and military operations now depend on massive data collection and management to extract actionable insights and shorten decision loops. In one example, the US National Geospatial‑Intelligence Agency collects 15 terabytes of imagery per day. This is expected to grow 1000 percent in six or seven years.
Dynamic, real‑time data collection and exchange will feed AI models of deployed edge capabilities. We may now think of UAVs and other intelligence, surveillance and reconnaissance platforms in that context, but real‑time data collection and exchange are moving into almost every piece of deployed equipment. Increasingly enabled by 5G technology, this includes weapons, communications and clothing—and will soon include implanted biotechnologies such as brain‑computer interfaces.
So, AWS’s top secret private cloud is more than just cheaper, scalable infrastructure for sharing and storing classified reports. It is the foundation for intelligence and defence agencies’ ability to utilise AI. The cloud is the ocean of data storage and processing power that will fuel AI’s catalysing effect on intelligence and defence agencies. Sensor feeds, human whispers, the electromagnetic hum of a thousand devices—these all converge. The data needs to be rapidly ingested, cleaned of noise and integrated into coherent intelligence.
Human-machine teaming is necessary. There is no other option. This is where our real challenge lies, which will make the policy and technology challenges pale in comparison. It will be the challenge of cultural and organisational change of intelligence and defence agencies. Australia’s intelligence and defence agencies are already using AI and machine-learning algorithms in their enterprise technology stacks, but so far this has been a process of augmenting existing roles, processes and organisational structures. We need to move away from the idea that the cloud and AI are there to enhance or support existing roles. It is about completely transforming these roles.
AI is the latest gale of creative destruction. In the 1780s, the introduction of the power loom helped kick off the industrial revolution. The power loom put hand weavers out of business and transformed the process of textile manufacturing. AI will transform intelligence and defence agencies to the same degree.
AI doomerism is the fear that AI will destroy humanity. Let me coin ‘AI loomerism’ as the misplaced idea that AI can be effectively integrated into organisations without radical transformation of roles, processes and structures. AI is not there to support the conventional roles of spies or warfighters, in the same way the power loom was not there to support the hand weaver. It was there to radically improve efficiency and force a reconceptualisation of humans’ role in the system, the value of their skills and expertise, and therefore their very identity.
In our quest to utilise AI technology to give our decision makers faster and more actionable insights, we must embark on a transformative odyssey, not merely tinkering with analytical tools or tradecraft. The intelligence community stands at a threshold. We are called upon to not just reassess the processes of spying and warfighting, but the very foundations of them.
Imagine, if you will, a complete restructuring, a rewiring of the very fabric of Australia’s intelligence and defence agencies. Legal frameworks, once rigid, must become adaptable. Policy and governance, once siloed entities, must enable data flows in real time, unlocking the full potential of AI. By streamlining these core functions, we unlock the potential for shared resources: powerful language models, cutting-edge infrastructure, and partnerships that bridge the chasms between agencies and missions. It is through this grand transformation that we can truly harness the power of cloud, data and AI, not just as tools but as paradigm‑shifting technologies.